2 min read

Broadcom Poised to Lead AI Inference with ASICs and What That Means for You

Broadcom Poised to Lead AI Inference with ASICs and What That Means for You
Photo by Brian Kostiuk / Unsplash

A quiet but powerful chip battle is unfolding behind the scenes of every chatbot and smart app you use. Broadcom, better known for networking gear than flashy AI headlines is being tipped to become the new leader in AI inference by 2026 thanks to its work on custom chips called ASICs. If that happens, the AI tools you rely on could get cheaper, faster, and greener.

What is AI inference and why it matters more than you think.

AI inference is the moment an AI model answers your question or performs a task in real time. Training builds the model; inference is everything that happens after, repeated for every user request. That repetition makes inference very cost-sensitive: small efficiency gains scale to big savings. Right now, Nvidia’s GPUs dominate, but inference values efficiency over raw, general-purpose power, opening the door for specialised chips.

ASICs vs. GPUs: the specialist vs. the Swiss Army knife  

Think of GPUs as a Swiss Army knife: flexible, powerful, and useful for many things. ASICs (application-specific integrated circuits) are the specialised kitchen tool; they do one job extremely well. That specialization lets ASICs:

  • Run inference faster for certain AI models  
  • Use far less electricity (big for cost and the environment)  
  • Lower the per-query price of AI services as usage scales

Broadcom’s history with custom chips - including helping build Google’s TPUs and large TPU orders from customers like Anthropic shows the company is betting big on this specialist approach.

The evidence that Broadcom could take the lead  

Here are the concrete signs fuelling analysts’ optimism:

  • Broadcom helped develop the custom chips that power Google’s AI (TPUs), which are now offered through Google Cloud.  
  • Anthropic reportedly ordered about $21 billion in TPUs tied to Broadcom this year.  
  • OpenAI and other big players are exploring alternatives (including AMD) for inference, signaling the start of a post-Nvidia diversification.  
  • Broadcom’s recent scale, roughly $64 billion in revenue last year and growing orders gives it the manufacturing and sales muscle to compete.

What this could mean for everyday users and businesses  

If ASICs win the "inference era," expect:

  • Lower costs for AI features in apps and services (more affordable or even free tiers)  
  • Faster responses from chatbots and real-time AI tools  
  • Greener AI thanks to better energy efficiency a real factor as usage grows  
  • More competition, which can spur innovation and better pricing
🫠
Caveat: Nvidia still leads training and remains strong in many areas. Software compatibility, model support, and industry adoption will determine how fast the shift happens.

Who knows where this goes.

Published by:

Karl Kloppenborg