Groq

Cassandra Unchained

@michaeljburry

As Nvidia pours $30 billion into OpenAI so OpenAI can spend 20+ billion on Nvidia chips, remember this story from last week. Reuters reported two weeks ago that OpenAI had become dissatisfied with the performance of Nvidia’s hardware for certain types of inference tasks, especially coding-related workloads and AI-to-software interactions. The company began exploring alternatives last year, including deals with AMD, Cerebras, and early discussions with Groq. It was reported that OpenAI was specifically concerned about speed, saying Nvidia’s GPUs were not fast enough for certain inference-heavy products such as Codex, OpenAI’s coding model. Inference, in contrast to training, relies heavily on memory access speed. Reuters reported that OpenAI was looking at architectures using large amounts of on-chip SRAM, which could accelerate real-time responses but differ from Nvidia’s conventional GPU designs. https://t.co/pCncenDr7y

Gavin Baker

@GavinSBaker

Interesting report from @theinformation that Nvidia will be able to make 1000 Vera Rubin racks per day, which is $630b per quarter. Actually a little hard for me to believe and haven’t checked the math, but wow if true. And Vera CPU racks and Groq LPU racks would be incremental to this. As would their new business model where they take a revenue share of neocloud revenue in return for guaranteeing offtake. I do think the latter point has not been well explored by analysts and is likely to be super important going forward.

Deedy

@deedydas

Every single startup working on next-gen AI chips (July 2026) Every approach attacks data movement differently to break Nvidia’s dominance: – eliminate DRAM (Groq) – eliminate the interconnect (Cerebras) – eliminate the compute/memory split (d-Matrix) – eliminate the server’s compute-centrism (Majestic) – eliminate generality (Etched, Taalas, MatX) – or eliminate the $400M litho machine (Substrate)

Chamath Palihapitiya

@chamath

Taken Sep 1, 2016 when @JonathanRoss321 convinced me we could take on the giants, build new silicon and that AI was coming. In typical SV fashion, we didn’t even have a company yet - just a term sheet from me to invest and the three of us. I spent the next month recruiting as many of the TPU team from Google Wisconsin as I could. “Welcome to chips”, I thought. The company, as with all important companies, went through its own trials and tribulations including promoting Jonathan from CTO to CEO and the inevitable falling out and repair of his and my relationship. Anyways, it all happened for a reason. Today we close this almost decade chapter and Jonathan starts a new one with nVidia. I can’t thank him enough. Sometimes it’s simply better to be lucky than good and be fortunate enough to work with great people and follow them into battle. That is me here. Jonathan was not only the father of TPU when he was at Google but he is a technical genius of biblical proportions. He also assembled a great team with folks like @sundeep and @GavinSherry to back him up. They will also do incredible things at nvidia. Separately, whenever something I’m involved in either crashes or wins, I reread my thoughts when I first did it. Was it luck? Was it skill? What did I learn? What do I do now? YMMV, but I am attaching the original investment memo I wrote a decade ago. I’ll do a more substantive view on why this deal matters and what my guess is about the future of AI silicon from here on an upcoming episode of the pod. In the meantime, Merry Christmas. I am very thankful. So thank you Jonathan, Sunny, Gavin and the entire Groq team. 🙏🏽🙏🏽🙏🏽

Matt Turck

@mattturck

My conversation with @andrewdfeldman, CEO of @cerebras. We started from "what is a wafer?" and built up to why the entire chip industry is reorganizing around inference speed. 00:00 Cold open & Intro 01:31 Why speed became the AI bottleneck 02:32 Tokens per second per user, explained 03:16 AI’s broadband moment and the Netflix analogy 04:35 The AI chip landscape: GPUs, TPUs, Trainium, ASICs 06:36 What is an ASIC? 08:08 Nvidia, Groq, and the fast inference war 09:16 OpenAI, Broadcom, and specialized silicon 12:10 China, power, and sovereign AI infrastructure 15:05 Is the AI infrastructure boom a bubble? 18:56 The hidden bottlenecks: HBM, CoWoS, and 3nm 22:57 Why agents are creating CPU demand 25:36 Andrew’s path from SeaMicro to Cerebras 26:13 Why Cerebras bet on AI in 2016 31:14 SRAM vs. HBM: why inference is a memory problem 33:19 What wafer-scale computing actually means 34:28 The deep-tech “Everest” problem 36:07 The moment the first Cerebras system worked 36:49 Ringing the bell and surviving deep tech 39:08 How a giant chip handles failure 41:22 Why GPUs struggle with decode 42:17 Prefill vs. decode explained 44:01 The “100 HD movies” problem in AI inference 45:04 How fast inference changes RL and training 48:08 Reasoning models and why they cost more compute 50:08 Verification, guardrails, and small models checking big models 52:37 Multimodal AI and the path to video 53:51 Cerebras’ business model: hardware, cloud, API 55:14 OpenAI’s 750MW inference deal 55:36 Why data centers are measured in megawatts 58:01 AWS Trainium + Cerebras decode 59:29 Fast tokens as a cloud product 01:00:52 Is CUDA still a moat? 01:03:53 How TSMC helped Cerebras build the giant chip 01:07:41 Why nobody cared in 2020 01:08:15 Why chip supply chains are hard to diversify 01:09:54 Why today’s AI models will be the worst you ever use 01:10:38 What fast AI could do to SaaS

Natasha Malpani 👁

@natashamalpani

the infrastructure war has just started. @etched raised $300M at $10.3B this week, less than a month out of stealth. $1B orders already booked. the most interesting investor on the cap table is SK Hynix. the world’s largest high-bandwidth memory supplier just co-invested in a chip designed to need dramatically less of what they sell. they plan to sell memory to serve the inference explosion that follows. this is a supply chain signal. the stack is splitting into three distinct hardware markets. -training compute consolidates at the frontier: @nvidia, massive clusters, a handful of labs. cloud inference bifurcates: @etched, @groq, @cerebras. serving fixed weights billions of times is a different engineering problem from training them. edge inference: https://t.co/dvDxAOfnIH is building chips for robots, cameras, and manufacturing equipment that cannot send data to a cloud server. intelligence is now abundant. what remains constrained is physical. power, memory, grid connections. as we saw with kimi K3 this weekend, every model hits these constraints the moment it tries to scale. generation is distributed. infrastructure is not.

Jonathan Ross

Founder & CEO

Creator of Google's original Tensor Processing Unit (TPU) and founder of Groq's LPU inference chips.