Training neural networks is expensive and inefficient. The amount of computation required to train the largest models is estimated to be growing >5x every year, yet hardware performance per dollar is growing at only a fraction of that rate. As a result, the cost of training the latest neural networks has ballooned to millions of dollars, leaving these capabilities accessible to only the best resourced institutions. We believe that this inaccessibility is a core reason why we haven’t seen AI / ML positively impact more areas of our life.
Today, MosaicML comes out of stealth to address this gap. We believe that unfettered growth of computing is not a sustainable path towards a future powered by artificial intelligence. Our mission is to reduce the time, cost, energy, and carbon impact of AI / ML training so that better AI models and applications can be developed.
We tackle this problem at the algorithmic and systems level. MosaicML makes machine learning more efficient through composing a mosaic of methods that together accelerate and improve training.
Compute is expensive, and we provide methods that extract the most training out of every cycle by eliminating inefficiencies in the learning process. We also find the most efficient hardware, system architecture, and cloud on which to perform these computations. We provide this as a set of tools to allow practitioners to make rational trade offs between cost, time, and the quality of the resulting models.
Our amazing team of researchers and machine learning systems experts have thoughtfully crafted a foundation for our product offerings. Today, we are sharing with the community our initial methods and analysis tools in the form of an open source package (Composer) and a visualization tool (Explorer):
MosaicML has raised a total of $37 million from an amazing set of investors: Lux Capital, DCVC, Future Ventures, Playground Global, AME, Correlation, E14, and some very special angels. We would like to thank them for their support and belief in our vision.
This is only the beginning. We will be releasing more tools to help the ML research & engineering community with compute efficiency, costs, and environmental impact. Please stay tuned!
Naveen, Hanlin, Jonathan, and Mike