CUDA’s libraries are open. NVIDIA’s computers are not.

12patches sent to NVIDIA’s CUDA libraries

7now in NVIDIA’s code

On November 11, NVIDIA’s board accepted Jensen Huang’s resignation. Much of the coverage since has been about who decides where NVIDIA’s chips go. I wrote about that separately. This post is about the layer under the argument: CUDA, the software that makes those chips hard to replace.

I have a small first-hand record of it. I am not a CUDA engineer; I went in out of interest. In late September, working with AI coding agents, I sent twelve patches to three of NVIDIA’s open-source CUDA libraries. They are small fixes: an iterator in Thrust that lost its step, a build script in NCCL that read the wrong git repository, a test plugin in CUTLASS that grouped tests wrongly. Seven are now in NVIDIA’s code. Sending them showed me where that software sits, who can change it, and who decides what runs.

Where CUDA sits

CUDA turned twenty this year, and NVIDIA counts six million developers on it. The part I touched is the middle of the stack: the libraries that AI frameworks stand on.

The patches went into the layer the frameworks stand on
  1. Models and products

    What people use.

  2. Frameworks

    PyTorch, RAPIDS and others.

  3. Core libraries

    CCCL, NCCL, CUTLASS.

    12 sent, 7 landed

  4. CUDA itself

    Compiler, runtime and drivers.

  5. GPUs

    From data-centre racks to a laptop.

A simplified picture. Counts as of 2026-11-16.

Two examples of how much rests on that middle layer.

NCCL moves data between GPUs. PyTorch’s documentation says to use the NCCL backend for distributed training on CUDA GPUs, and lists it as the default for CUDA. By default, a model trained across many NVIDIA GPUs with PyTorch goes through it.

CCCL is the C++ foundation: Thrust, CUB and libcu++. Until late October, its CI built RAPIDS, MatX and PyTorch against proposed changes. Those builds now run nightly. I know because one of the twelve patches was about that CI.

A rival can match a chip’s speed and still not have this. Tom’s Hardware reported last year that China has no software ecosystem to rival CUDA, and that DeepSeek, after trouble training on Huawei’s chips and toolkit, reportedly went back to NVIDIA for training.

Twelve patches, seven landed

Seven of twelve landed, by library

CCCL: Thrust, CUB and libcu++3 of 6

NCCL: multi-GPU communication3 of 3

CUTLASS: linear algebra templates1 of 3

One square per patch. Amber means the change is in NVIDIA's code, as of 2026-11-16. Source: the pull requests.

None of the twelve is about speed. They are upkeep, the kind a twenty-year-old stack needs.

Of the five that did not land, two were overtaken by NVIDIA’s own decisions and three are still waiting.

The one real bug. Thrust has a counting_iterator that produces a sequence such as 10, 13, 16 without storing it. Converting one to another compatible system, host or device, kept the starting number and lost the step, so the next element depended on what happened to be in memory. The fix is three lines. Thrust can also run on a CPU, so it could be proven on a Mac: the new tests failed before the fix and passed after, 208 assertions in all. It was merged 39 hours after it was opened.

Closed does not mean rejected. All three NCCL patches show as closed on GitHub. All three are in NCCL, as commits with my name on them. A maintainer typed /mirror, and days later the change appeared and the pull request was closed. My own tracking had counted them as rejections.

Open code, gated computers

None of the twelve was run on an NVIDIA GPU before NVIDIA ran it. Eleven of the descriptions say so, and the twelfth only changes a comment.

Each patch to CCCL got the same first reply from a bot: it needs validation before any workflow can run on NVIDIA’s runners. The full CI starts only when a maintainer writes /ok to test and a commit hash. The approval covers that commit. Push an update and it has to be given again.

I write about controls on AI agents, so I noticed this one. These patches were written with an agent’s help. None of that code could reach NVIDIA’s machines until a maintainer had cleared one specific commit. That is a gate that holds. A paragraph in a contributing guide would not have been.

Once the gate opens, NVIDIA spends a great deal of machine time on a stranger’s patch.

Job time NVIDIA's CI spent on four of the patches

Thrust examples, 77 jobs10 h 45 m

Thrust fix, 154 jobs7 d

CUB reduction, 202 jobs9 d 21 h

CI change, two runs, 1,134 jobs11 d 19 h

The 'Total' the CI bot reports for a run, which I read as the run time of all jobs added together. The Thrust fix is 50 lines. Source: the bot's summary comments.

The other end of the chain is a laptop

I do own one NVIDIA GPU: an RTX 5070 Ti with 12 GB, in a gaming laptop at home. It was not reachable when the twelve patches were written, so they were tested on a Mac.

I have put it to work twice since. On October 4, an experiment of mine made 198 generation attempts with a 7-billion-parameter coding model on it. During the measured runs the GPU sat at a median of 96% utilisation. On October 12, I had an agent reach it over my private VPN and SSH and run 14 shader tests on the real GPU, for the AnyPS5 side of the fix in this post. That second job was Vulkan, not CUDA.

The racks the export bills are about and the laptop at home run the same vendor’s stack.

What I think this says

  1. The moat is upkeep. Twenty years of fixes like these, checked by hundreds of CI jobs against the frameworks above, is what a rival has to reproduce. A faster chip does not come with it.
  2. The libraries are open and the computers are not. I could change Thrust from a Mac. I could not run it on NVIDIA’s hardware until NVIDIA said yes, one commit at a time. On a small scale, that is the question Washington is now asking on a large one: not who may read the code, but who gets the machines.
  3. Agents make upkeep cheap, for everyone. Working with an agent, and with no GPU, I had a real bug in Thrust fixed and merged within two days of reporting it. The same way of working can port and maintain code for other chips. In the same week I sent one patch to AMD’s ROCm that way. It is still open, and one patch proves nothing. I do not know which side cheaper upkeep helps more.

What I did not do well

  • Nine of the twelve were opened within 53 minutes, across all three repositories. That is a lot to hand one company’s maintainers at once.
  • Six of the twelve descriptions say AI was used. Six do not. This post is that disclosure for the other six.

Sources

Jake WangSenior software engineer at Pinterest. I write about running AI agents like production systems, and I maintain Argon.About meFollow on X