Cornstarch

Build, Train, and Run Your Own Multimodal Model

---

Cornstarch is a multimodal model training framework that allows you to create your own multimodal model from a set of HuggingFace unimodal models, train, or use it.

Documentation Organization

• Getting Started: Instructions on installation and setup.
• Creating a multimodal LLM: How to create a multimodal LLM from unimodal models.
• Preprocessing multimodal inputs: How to preprocess muiltimodal inputs to run a multimodal LLM.
• Training a multimodal LLM: How to train a multimodal LLM using Cornstarch.
• Multimodal LLM Parallelization: How to parallelize a multimodal LLM training.

Research Works

Below is a list of works powered by Cornstarch.

• Oobleck: resilient distributed training using pipeline template

Contact

Insu Jang (insujang@umich.edu)