Since the mid-to-late 2010s, the Neural Processing Unit (NPU) has revolutionized mobile devices thanks to the pioneering efforts of Apple, Huawei, Qualcomm, and Samsung. These compact powerhouses have transformed cameras, enabling them to recognize objects, adjust focus, and apply filters. They have also elevated voice assistants and augmented reality, showcasing their versatility and potential in various applications.
The PC ecosystem swiftly responded to the AI arms race last year, with AMD and Intel leading the charge. In January 2023, AMD unveiled the game-changing Ryzen PRO 7040 series processors with built-in NPUs designed for thin laptops. Not to be outdone, Intel launched its Core Ultra series of three-in-one (CPU, GPU, NPU) mobile processors in December 2023. The ripple effect was immediate. Acer, Asus, Dell, HP, Lenovo, MSI, and Samsung soon announced laptops featuring Intel’s or AMD’s new chips.
Microsoft declared 2024 “The Year of the AI PC” in January. At its build conference in May, the company announced Copilot+ AI-ready PCs and revealed how AI will be natively embedded into Windows. In April, AMD introduced Ryzen Embedded 8000 series processors optimized for industrial AI applications, machine vision, and robotics. AMD says its new XDNA 2 engine is designed for PC-based generative AI.
As the PC industry enters a transformative era, integrating NPUs as standard processing technology is not just a potential development but a vital one. This transition has the power to redefine the future of computing, underscoring the need for industry professionals to keep pace with these changes.
Nothing’s Faster Than the (Virtual) Human Brain
While CPUs handle sequential processing and GPUs provide parallel processing, NPUs stand out with their unique ability to simulate human neurons at the circuit layer. This design is specifically tailored to optimize the handling of complex mathematical computations, a crucial function that underpins the artificial neural networks (ANNs) on which machine learning and deep learning depend.
NVIDIA’s market share exploded as the company became a central, if not leading, player in the AI game with its line of GPUs. However, GPUs are not designed explicitly for AI. The more complicated the AI and the faster it must generate accurate results, the greater the space and power draw required by traditional GPUs.
NPUs can handle the same inference tasks at much smaller sizes and require much less power. They complement, not replace, CPUs and GPUs. Offloading AI tasks to an NPU frees up system resources for the CPU and GPU. Considering its more considerable headroom, the NPU’s performance advantage could pay out on a larger scale for PCs than smartphones.
What PC Applications Do NPUs Support?
NPU technology should greatly benefit industrial machine vision applications. Machine vision and deep learning algorithms depend on inference and pass image data through layers of neural circuits, an ideal example of an industrial application that NPU design supports. The small size and low power requirements of NPUs also make them well-suited for edge computing and IoT devices that run AI at the point of data capture, known as “the edge”. This could benefit MES (manufacturing execution systems) and ERP (enterprise resource planning) systems running NPU-enabled industrial workstations.
PC hardware that supports medical imaging, which requires rapid data acquisition and fast analysis with low latency tolerance and close-to-zero error tolerance, should soon leverage AI chips as standard technology. While not specific to industrial applications, companies have learned to use natural language processing (NLP) to support administrative and communication functions.
NLP combines computational linguistic models, machine learning, and deep learning, making it a prime application for NPU support. Allowing maintenance workers on plant floors to create context-specific incident reports via NLP and AI could save manufacturers time and increase productivity.
How Will PCs Evolve to Support NPUs?
Chip manufacturers have all taken the three-in-one approach so far. Intel’s Core Ultra processors, AMD’s Ryzen AI chips, and Qualcomm’s Snapdragon X-series chips integrate CPU, GPU, and NPU onto a single chip. Considering how NPUs support CPUs and GPUs, an integrated three-in-one chip design makes perfect sense. Additional PCIe slots or other accommodations for NPUs should not be necessary.
A big question is how the lack of standardization and unclear performance measurements in these early days will affect the choice of which NPU-enabled processor to deploy. Are some better geared toward machine vision than others? Industrial hardware manufacturers and vendors must pay close attention to who generally adopts AI PCs and which industries favor specific NPU chip families.
Why PCs Need NPUs Long-Term
If the new AI PCs offer superior all-around performance compared to hardware without NPU-enabled processors, or if new releases like Windows 12 require NPU support for full functionality, hardware manufacturers and vendors will be encouraged to adopt NPU-enabled chips. Beyond general performance benefits, there are a few arguments for why PCs must embrace NPU-enabled chips.
Most AI computing currently occurs in the cloud, which some experts argue will hinder AI development. That argument suggests that local AI may process data with lower latency and a better contextual understanding compared to AI running in the cloud. Anyone training AI models may want to do so on-device. The number of models being developed will grow exponentially as large language models (LLMs) become cheaper and easier to use.
NPUs offer tremendous advantages for AI cybersecurity software designed to observe and learn from regular data traffic patterns, identifying anomalies, threats, and intrusions. Consider whether a cybersecurity vendor protecting susceptible data would run its AI cybersecurity remotely in the cloud, depending on the cloud providers’ security guarantees, or on-premises for complete server control.
Hardware manufacturers and vendors may not have to go all in on the NPU in 2024. But looking forward to 2025 and beyond, the landscape for NPUs is rapidly advancing. The value of NPU-enabled processors, the investments made by major chip manufacturers in their development, and the rising number of AI-enabled PCs on the market support their long-term value.
Build Your Ideal PC with CoastIPC
Within the industrial automation space, AI advancements from niche companies and major conglomerates will continue to benefit businesses worldwide. CoastIPC will help you identify the best components for your AI application, from machine vision and robotics to autonomous vehicles and intelligent transportation applications.
NPUs will not be a perfect match for every job, but CoastIPC can work with you directly to build a rugged industrial PC tailored to your application needs. For 15 years, CoastIPC has built the brains for the smartest machines in the world. Contact us today to discuss GPUs, CPUs, and any other industrial computing technology that may help bring your project to life. Contact us anytime via email, phone, or chat. Check out our blog or follow us on LinkedIn to stay up to date.