Nearly two weeks after Arm made waves by launching its own CPU for agentic AI, Intel has joined hands with Google to advance AI designs, with Xeon processors powering inference and general-purpose workloads, as well as the co-development of custom infrastructure processing units (IPUs) to offload networking, storage, and security tasks from CPUs.

These two announcements affirm that CPUs are very much back in the AI game amid the shift from model training to complex agentic workflows. It also marks the resurgence of general-purpose compute in AI systems, with a twist: merchant silicon products, such as Xeon processors, alongside custom silicon devices such as IPUs.

The CPU reinvigoration also comes at a time when the AI world is moving beyond training models, where GPUs are predominant, to latency-sensitive inference at scale, which places sustained demands on CPU resources for orchestration, data pre-processing, and system management that training pipelines don’t.

Intel CEO Lip-Bu Tan is quick to point out that scaling AI requires more than accelerators. “It requires balanced systems.” While he is echoing the popular notion that GPU accelerators alone are insufficient to meet the demands of modern AI infrastructure, what does a balanced system stand for? Tan refers to a combination of CPUs and IPUs in a heterogeneous AI system.

The CPU-IPU equation

Intel’s partnership with Google is built around general-purpose Xeon CPUs and custom IPUs, in a multi-year collaboration aimed at boosting performance, efficiency, and total cost of ownership across heterogeneous AI systems. The general-purpose CPUs, combined with custom silicon, allow Intel to offer a competing narrative to GPU-centric solutions in AI deployments.

CPUs are becoming more relevant in AI as systems become more orchestrated, stateful, and complex; here, CPUs bolster workaround coordination, memory handling, and control. Currently, Google employs Intel Xeon 5 and Intel Xeon 6 processors for a variety of workloads in its cloud infrastructure. That encompasses both large-scale AI training coordination and latency-sensitive inference.

While CPUs handle orchestration, data processing, and system-level control, IPUs offload networking, storage, and security functions from CPUs. That improves utilization, increases efficiency, and enables more predictable performance across hyperscale AI environments. In other words, IPUs enable CPUs to focus entirely on application and AI workload processing.

An IPU is a custom ASIC that serves as a programmable accelerator and handles infrastructure tasks traditionally managed by CPUs. That, in turn, bolsters compute capacity and allows AI data centers to scale more efficiently without increasing overall system complexity. The announcement notes that Intel and Google will deepen the co-development of IPUs and thus match the prowess of Arm-based custom processors for AI workloads.

IPUs, alongside CPUs, form a tightly integrated platform that balances general-purpose compute with purpose-built infrastructure acceleration to deliver more efficient, flexible, and scalable AI systems. In other words, Intel is integrating Xeon CPUs with IPUs to enable a more balanced architecture that optimizes both general-purpose and specialized processing.

Arm vs. x86 in AI processors

The announcement of Intel’s CPU tie-up with Google came weeks after Arm launched its purpose-built CPU for agentic AI. Arm also claimed that its new CPU offers more than 2× performance per rack compared to x86, and that Meta is its lead partner and co-developer. Moreover, according to Arm, more than 50 companies are supporting its CPU initiative for AI systems.

Arm’s premise that its new CPU addresses the specific needs of agentic AI could be the main catalyst behind Intel’s collaboration with Google. Before the announcement of this partnership, Intel’s Data Center group chief Kevork Kechichian questioned Arm’s claim to be the first to address this AI workload shift. Kechichian also noted that Intel has CPUs with similar characteristics.

“This deal is Intel asserting that the next phase of AI infrastructure will still leave room for x86, especially where buyers want continuity more than a clean-sheet shift in architecture,” said David Harold, senior analyst at Jon Peddie Research. “Google’s continued use of Xeon 6 in C4 and N4 instances helps make that case.”

So, Arm with Meta matches Intel with Google. On the one hand, it shows the influence of hyperscalers on silicon development. At the same time, however, it demonstrates a strategic combination of merchant CPUs with custom silicon designs. For instance, Google, now reemphasizing the relevance of Intel Xeon processors, has deployed custom accelerators in its cloud infrastructure for years.

More importantly, Intel, which has been striving to enter the lucrative AI market for several years, seems to have made CPUs relevant at a time when the center of gravity in AI is shifting from model training to infrastructure development. And for that, Google could prove a valuable ally.

By: DocMemory
Copyright © 2023 CST, Inc. All Rights Reserved