Menu
This is Jessie with Global League (G-L.Ventures) – a network for collaboration between investors to help the most impactful startups recommended by investors. We prioritize climate tech with high impact, AI infra, automation, and semiconductors. We share information for your reference, invite collaboration, and do not solicit or endorse any investments. We always ensure to have industry experts in evaluation and use a digital platform to support the process of avoiding cognitive bias.
AI Compute’s Bottleneck Lies in Connectivity
Among all IPOs (a full list here) in 2024 so far, Astrea Labs (ALAB) has created a return (current price/IPO price) of almost 100% within less than 3 weeks after IPO, among the top 3. The other two either have a tiny revenue or market cap. All other IPOs are in very bad shape or much less impressive, including Reddit. The valuation (near $11B) of ALAB is nearly 100x last year’s revenue of $115.8 million, demonstrating the high prices investors will pay for AI infra companies. (the company isn’t profitable yet)
Astrea Labs provides semiconductor-based connectivity solutions purpose-built to unleash the full potential of cloud and AI infrastructure. For example, one of their product aims at the famous memory wall problem by innovating the connectivity between computing and memory chips. Their Intelligent Connectivity Platformprovides (1) High-speed connectivity integrated circuits (ICs), modules, and boards, and (2) COnnectivity System Management and Optimization Software, called COSMOS.
Another notable funding news is about Celestial AI – $175 Million Series C Funding Round Led by U.S. Innovative Technology Fund (invests in tech with dual-use applications in commercial and defense industry) with participation from investors including AMD Ventures, Koch Disruptive Technologies (KDT), Temasek, Temasek’s wholly owned subsidiary Xora Innovation, IAG Capital Partners, Samsung Catalyst, Smart Global Holdings (SGH), Porsche Automobil Holding SE, Engine Ventures, M-Ventures and Tyche Partners.
According to their website: “Celestial AI’s Photonic Fabric is the industry’s only optical connectivity solution that enables the disaggregation of compute and memory, which allows each component to be leveraged and scaled most effectively. This breakthrough technology delivers over 25x greater bandwidth and memory capacity while reducing latency and power consumption by up to 10x compared to existing optical interconnect alternatives and copper.” The company partners with some of the largest hyperscalers and tier-1 partners. (designing in the Photonic Fabric optical chiplets, no software change needed)
AI model sizes grow exponentially (400x in 2 years) but GPU capacity grows only 2x in 2 years. Massive volumes of data must be rapidly transferred between processors, memory, and storage, today’s advanced AI models require exponentially more I/O bandwidth and memory capacity, otherwise low GPU utilization and increased power consumption will result. And it’s projected that in 3 years, we’ll need 1000’s to millions of GPU chips connected by fiber links with ultra-high bandwidth and low latency. The requirement of next-generation connectivity to push AI computing further is singled out by top companies.
Another AI infra company, Cerebras Systems, just announced its IPO plan in the second half of 2024 at its earliest. It builds AI chips as large as the whole 300 mm wafer. With over 4 trillion transistors – 57x more than the largest GPU on a chip, we can see why it has “Twice the speed at same power & cost” as Nvidia’s H100. (Source) But its approach has its manufacturing challenges.
Advanced Packaging and Si Photonics
Photonics uses light to transmit signals, the speed of light is several orders higher than electrons (and less heat). Optical cables and pluggables have been used to connect AI servers. To use photons to replace electrons to shorten the path of electrons, top semiconductor players are actively developing advanced chip packaging technologies, such as CPO (Co-Packaged Optics). CPOs are a natural replacement for the optical pluggables.
Photonics includes other semiconductor technologies than Si. But Si Photonics is desired because it leverages Si semiconductor economies of scale to create photonic integrated circuits (PICs) that use light to transmit and process data. It enables in-package chip-to-chip optical I/O communications and brings the benefits of light speed from the server and rack level to the board and chip level. It would allow thousands of GPUs to operate as one and communicate directly with disaggregated memory – imagine a giant GPU like Cerebreas’ chips. Ayar Lab and Intel are both working in this space. Nvidia has set a future goal of GPU interconnect performance, but both can’t achieve the goal yet.
Broadly, electronic photonic integration provides a wide range of unique advantages that can be exploited for a variety of applications, including quantum computing, telecommunications, and high-performance computing. The fundamental advantage of photons over electrons leapfrogs performance, energy efficiency and device sizes, etc.
All the above are breakthroughs for different applications. We have identified some startups pushing the boundaries in these frontiers, even achieving the goal set by Nvidia. Let us know if you’re interested in meeting them or learning more.
AI computing arms race has accelerated. The news of new AI funds has flooded the market for a while. From the largely oversubscribed round of Celestrial AI, and the high price investors give Astrea Labs, the demand for quality AI infra companies is much higher than the supply. Recommendations of companies in this AI infra category are welcome. (Email: Contact@G-L.Ventures)
Read more:
AI Infrastructure Hardware and Software Accrue the Most Value in AI Stack
Onboarding to DueDash
To support our networking, communications, workflow, and method, sign up to DueDash with our invitation link so you can use certain paid functionalities with us for free, and join deal evaluation collectively. Startups can sign up to update the Global League network of investors and build their data rooms for free.
Sign Up to Global League on DueDash
Email: Contact@G-L.Ventures for communications.
