Digital Event Horizon
Microsoft's NPU obsession has left Nvidia's AI PC ambitions uncertain. While Microsoft pushes for an NPU-based future, Nvidia continues to push for its own AI-enhanced solutions.
Nvidia's AI PC ambitions are being impacted by Microsoft's NPU obsession Microsoft requires NPU-based PCs for its Copilot+ feature, which Nvidia's GPU doesn't meet Nvidia is working to bring AI features to PCs, but faces competition from NPUs NPUs are likely to become a target for developers building AI apps in the future Nvidia's competitive edge may be compromised by its reliance on proprietary libraries and microservices
Microsoft's recent push into Artificial Intelligence (AI) augmented features and experiences has left a trail of unanswered questions about its stance on Neural Processing Units (NPUs). At the heart of this conundrum lies a contentious issue between Microsoft and its rival, Nvidia. The question remains, where does Microsoft's NPU obsession leave Nvidia's AI PC ambitions?
In early 2024, it became apparent that Microsoft's future for Windows would be deeply intertwined with AI-augmented features and experiences. Headline features included live captions and translation, image generation in MS Paint, and the somewhat dubious Recall feature that captures periodic screenshots and uses them to keep track of past activity. For these new features to be exclusive to so-called Copilot+ PCs, computers must meet Microsoft's minimum performance targets.
According to Windows titan documentation, Copilot+ PCs require a neural processing unit (NPU) capable of 40 or more TOPS, or 40-plus trillion INT8 AI operations per second, along with at least 16GB of RAM and 256GB of storage. When this all launched, only Qualcomm had a processor capable of meeting Redmond's NPU requirement, and so only PCs with that silicon were allowed as Copilot+ PCs to run the aforementioned AI-augmented features.
Since then, Qualcomm's qualifying Arm-compatible X chips were joined by Intel's Lunar Lake and AMD's Strix Point and Halo processor families as Copilot+ PC compliant. Yet, somehow, a $2,000 Nvidia RTX 5090, as announced at CES 2025 this month, with more than 3.3 petaFLOPS of AI compute still isn't good enough for Redmond. No matter how many FLOPS or TOPS your GPU can muster, it only matters to Microsoft if it's an NPU churning them out — for now anyway.
Nvidia hasn't been slacking on the AI PC front, content with lording its dominance over the datacenter, workstation graphics, and discrete gaming GPUs. In fact, Nvidia has been working to bring AI features to the PC for years, Jesse Clayton, who leads product marketing for Windows AI at Nvidia, told The Register.
"We kind of started the movement with AI on the PC back in 2018 when we launched the first GeForce GPUs and Nvidia GPUs with dedicated AI hardware — our tensor cores," Clayton said. "Along with that, we announced the first widely deployed PC AI, which was DLSS, which is used in games to accelerate frame rates by using AI to generate pixels and now generating frames for the games."
Since then, the GPU giant has rolled out the RTX AI Toolkit, a suite of tools and software for optimizing and deploying genAI models on Windows PCs, brought Nvidia Inference Microservices (NIMs) to PCs, and rolled out a number of blueprints for things like state-of-the-art image generation, and converting PDFs to podcasts.
"Our strategy is where we can deliver interesting and differentiated experiences, either as a gamer because it enhances your gameplay, or as a creator because it saves you time and reduces the repetitive, tedious work," Clayton explained.
"NPUs are going to be where you can run your lightweight AI workloads, and they're going to be really power efficient. A GPU is where you run your more demanding AI use cases, and that's where we've been pushing and focusing our efforts."
"For stuff that simply doesn't fit on a PC, you run those on GPUs in the cloud, where you have effectively unlimited performance," Clayton added.
Considering Windows' market share and Microsoft's ongoing efforts to shoehorn AI into every corner of its software, it's only a matter of time before NPUs trickle down to even the lowliest of budget builds. What's more, adoption of frameworks such as Microsoft's DirectML and ONNX Runtime have helped to simplify application development and allow code to run across a diverse set of hardware with minimal retooling.
Just as your LLM once again goes off the rails, Cisco, Nvidia are at the door smiling
READ MORE
This poses a potential problem for Nvidia. The Silicon Valley goliath may dominate the discrete graphics processor market, surrounded by its CUDA moat, yet its GPUs are only found in about 18 percent of PCs sold with the vast majority of systems using integrated graphics from Intel, AMD, or others.
The case can be made that, before too long, NPUs will become a much larger target for developers building AI apps. And while Nvidia won't necessarily be left out of the conversation as its accelerators also support many of the more popular software frameworks, at least some of its competitive edge revolves around convincing developers to use its libraries and microservices, which promise easier integration and higher performance and efficiency.
Ultimately, Clayton says, developers will have to make a decision on whether they want to bring their app to market quickly using something like NIMs or if they want to support the largest possible install base.
But, while Nvidia may face competition from NPUs eventually — AI PCs are still a pretty niche market — it isn't necessarily all bad news. Even if models don't end up running on Nvidia's PC hardware, it's highly likely they were trained on its GPUs.
Even then, Clayton makes the case that NPUs won't be appropriate for every workload. Forty TOPS is a decent amount of compute, but, as we mentioned earlier, it pales in comparison to the performance of high-end graphics silicon.
"NPUs are going to be where you can run your lightweight AI workloads, and they're going to be really power efficient," he said. "A GPU is where you run your more demanding AI use cases, and that's where we've been pushing and focusing our efforts."
"For stuff that simply doesn't fit on a PC, you run those on GPUs in the cloud, where you have effectively unlimited performance," Clayton added.
Related Information:
https://go.theregister.com/feed/www.theregister.com/2025/01/20/microsoft_nvidia_ai_pcs/
https://www.msn.com/en-us/news/technology/where-does-microsofts-npu-obsession-leave-nvidias-ai-pc-ambitions/ar-AA1xuOjH
https://forums.theregister.com/forum/all/2025/01/20/microsoft_nvidia_ai_pcs/
Published: Mon Jan 20 09:14:11 2025 by llama3.2 3B Q4_K_M
