NVIDIA Research Panel Takeaway

Research at NVIDIA

Q: What is it like to do research at NVIDIA?

Open, collaborative and flat. Groups working in different directions often collaborate, because the underlying technology is shared. Julius's generative-model group, for instance, has worked with the weather group. The policies on using tokens and AI tools are generous, and you can choose your own research direction. NVIDIA has many partner companies, and research results can directly help them improve their models. Here you are often not the smartest person in the room: for every topic there is a colleague who has made it their life's work.

Q: What is special about the culture?

"Mission is the boss." On Jiao's first day, Jensen said the company has no org chart. The company is big, yet as agile as a startup. For researchers, that means if a direction fails you can switch to another and still stay ahead; it is not a case of "the direction dies and the person goes out with it".

(I really like the idea of the mission. Great people and great companies are always driven by a vision, and always have a lofty sense of mission: the force of aspiration carries you forward to where you are meant to be. Being as agile as a startup matters a great deal too. It is the "iterations per unit of time" measure I mentioned before.)

Q: Why is NVIDIA betting big on open models?

NVIDIA publishes not only model weights but also the data and the training recipes, so that others can adapt the models to their own applications. Training a strong model now takes enormous resources, and very few places in the world can do it and also allow it to be published in full. He thinks NVIDIA may be the only one.

Jiao is also pushing for students to be able to use NVIDIA as a platform for their research.

(That's true. EE194 Scalable AI, which I'm taking this term, is sponsored by NVIDIA: each group of four gets an eight-GPU node to train on. It's an excellent course that teaches large models as an end-to-end engineering discipline, covering the whole pipeline from architecture, pre-training and post-training to efficient inference. The course website is https://scalable-ai.eecs.berkeley.edu/F2026/.)

What are NVIDIA's scientists working on?

Q: What is Jiao working on?

The core question: how should hardware be designed to run well the software workloads of the next few years, which don't exist yet? (This is also what he teaches in EE194.)

Specifically:

1. As context windows go from 1 million tokens to 10 million and then 100 million, what model architectures and what data will they need?
2. Some people are building "company-level harnesses" (systems that orchestrate a whole company's employees and agents). What kind of inference workload will these generate?
3. Inference is already a datacenter-scale problem: you have to think about the single chip, the interconnect within a rack, links across racks, and even across datacenters, all at once.

The hard part is that conclusions from small-scale experiments don't necessarily carry over to large scale, so you need to find out early which ones won't extrapolate (this can save a lot of money and time).

(This is a concrete and very big research problem: predicting future workloads to guide the co-design of hardware and models, and working out what architecture and data ultra-long context needs. It's a very good direction.)

Q: What is Jenny working on?

Using Transformers to simulate the atmosphere at kilometre-scale resolution. The difficulty is multiscale interaction, from small-scale turbulence all the way up to weather systems thousands of kilometres across. With the US National Severe Storms Laboratory (NSSL), they feed radar and satellite data straight into the model to predict about three hours ahead, for severe convective storm warnings. Here, higher resolution isn't "adding pixels": it brings in new physical interactions. So the scaling law here is a physics problem and an AI problem at the same time.

Q: What is the biggest obstacle to scaling RL up? (audience question)

The RL in a course runs to completion on a single GPU. At cluster scale, the problems become:

how to keep training stable when GPUs fail; how to coordinate the three kinds of agents (learner, environment and reward) so that none of them blocks the others; and how to get the highest throughput out of the whole system.

(Again, this is the RL infra problem I mentioned before. Training a toy model and training at industrial scale are two completely different things.)

Q: What is Julius working on?

The goal is to turn bidirectional video models (which generate 5–15 seconds at a time) into low-latency causal models that generate frame by frame, for generative games and for virtual environments to train agents in. There are three open problems:

1. Causalization: when you generate frame by frame, errors accumulate. This is exposure bias.
2. Memory: the KV cache for video is far larger than for a language model and quickly runs out of GPU memory, yet the model has to remember the scene to stay consistent. His example: if you leave a room and come back, the ice cream on the table should have melted.
3. Acceleration: sampling has to come down from 50–100 steps to 1–4, through step distillation. Existing methods either suffer mode collapse or are limited by the quality of the teacher model.

Their group's work is open source in the FastGen library, which has already been used for Cosmos, weather, protein generation and more.

Advice for students

Q: With AI this strong, is a PhD still worth doing?

Thomas: The value of a PhD lies not in publications but in the ability to verify, and in resilience. AI really is very strong now, and everyone at NVIDIA uses AI in their work; yet with the same compute and the same agents, some people get things done and others don't. That's the gap. And don't be frightened by "model X killed another field today": the media is far too excitable these days.

(I once asked Claude this question too: in the age of AI, which ability matters most? It told me the same thing, the ability to verify. Generating knowledge and code is now quite cheap, but verifying still takes deep expertise and judgment.)

Q: One piece of advice from each of you? (The audience was mostly PhD students.)

• Thomas: In your first or second year of a PhD, always check your results yourself before you send them to your advisor or to senior students.
• Julius: Use LLMs as a learning tool. Aim to "learn faster", not to "produce more that you can't understand yourself".
  (In research, widen the range of what you are able to verify, rather than mass-producing plausible-looking things that lie outside it.)
• Jenny: Many good ideas come from academia, and academia has enough compute for them. Focus on fundamental ideas.
• Jiao: Look ahead, and find the current work that will become big. (That is, find a good direction and do research of real value.)

My questions

Q: If my goal is to be a Research Scientist in industry, do I need a PhD?

If the goal is Research Scientist, yes, because at most companies a PhD is the bar for that role. But you could consider Research Engineer, which usually doesn't require a PhD. Doing a PhD now really is something to think hard about: the field is changing too fast, and there are too many PhDs now.

Q: And I see a dilemma: university labs have almost no compute; the compute is basically all in industry.

Yes, so as a PhD student you absolutely can't stay in the university lab the whole time. Typically you spend the first two years in the university lab working out a topic and doing research, and from the third year on you must go to industry. In the US there are many part-time internships in industry, but CPT has recently been tightened, which makes things awkward for international students. You can still find collaborators in industry, though, and have them provide compute.

Q: I'm an undergraduate, and taking my project further needs compute. Where can I get it?

Go and talk to your advisor directly, and ask them to introduce you to partners and provide compute. You can also try working with collaborators in industry, or look at the research programmes of Chinese companies such as Alibaba Cloud and Tencent, which offer some funding.

(2026.10.15, UC Berkeley Sky Computing Lab)