Microsoft backed a tiny hardware startup that just launched its first AI processor that does inference without GPU or expensive HBM memory and a key Nvidia partner is collaborating with it
dMatrix Inc., a California-based startup backed by Microsoft, has launched its first AI processor, Corsair, designed for efficient AI inference without traditional GPUs or expensive memory. Corsair achieves high performance, processing up to 60,000 tokens per second for Llama3 models, with significant cost and energy savings. Initially set for late 2023, broader availability is expected in mid-2025. Notably, Micron Technology, a key Nvidia partner, is collaborating with dMatrix on this innovation.