News broke Wednesday that Meta Platforms plans to start a cloud computing business, prompting speculation that the first signs of waning demand for computing capacity may be starting to appear on the horizon. Chipmaking stocks were down across the board on Wednesday, though they rebounded a bit in early trading Thursday. But technology analysts say that another development is likely more responsible for the midweek dip in chip stocks than any sign of excess computing capacity at Meta: an efficiency gain in the ability of artificial intelligence algorithms to answer particular questions, a process known as “inference.” Frontier model OpenAI made a deal with chipmaker Cerebras in January to use the company’s new AI chips in its inference stack, and OpenAI engineers are now seeing greatly reduced inference costs, according to a Monday report in The Information. Plentiful memory Cerebras’ efficiency-boosting chips use a more plentiful type of memory – known as SRAM – than other AI chipmakers, and its success with OpenAI could ease the memory bottleneck and reduce the huge amount of pricing power in that section of the hardware supply chain. “The Cerebras chip uses SRAM, and SK Hynix, Samsung and Micron [produce] high-bandwidth memory DRAM, so if these guys really make a splash in the industry, then Micron gets hurt,” Paul Meeks, head of technology research at Freedom Capital Markets, told CNBC Wednesday. Cerebras stock soared 19% on Monday, another 2% on Tuesday and was little changed on Wednesday. CBRS 5D mountain CBRS past five days. Efficiency gains at OpenAI could be the start of a trend, as hardware and software companies scramble to optimize their inference capabilities and find workarounds for the memory chip shortage. “We’ve got all these chip startups and small companies talking about new methods of doing inference with different chip architectures,” analyst Bob O’Donnell at Technalysis Research said. “What we’re seeing is significantly more diversity in the types of Silicon that’s being used to do inference and the sense that all those options are viable because the demand is so high.” Inference connection While the analysts CNBC spoke with didn’t think Meta’s cloud play was responsible for Wednesday’s chip and memory rout, some do see a connection to what’s happening in the inference field. Meta climbed 9% on Wednesday in anticipation of the hoped-for new revenue stream for the Instagram and WhatsApp parent, but some analysts are concerned that CEO Mark Zuckerberg’s company could be missing out on efficiency gains in its own core advertising business. “We’d much prefer that Meta develop core AI products, leverage them over its base of around 4 billion users, and require massive compute for its own inference rather than selling access to its infrastructure,” Doug Anmuth at JP Morgan wrote in a Thursday note to clients. Meta has been signaling its move into the cloud space for some time. At the company’s shareholder meeting in May, Zuckerberg said the reason Meta hadn’t started renting its servers is that it needed the computing power for itself. “It’s possible that Meta is selling compute that’s more suitable for inference, and its heavy spending on training will continue, but it still suggests to us that Meta’s AI product traction beyond advertising remains limited,” Anmuth wrote.
What’s behind the volatility in memory chip stocks this week?
