A specific type of Base Model
The Stanford Institute’s Center of Human-Centered AI defines a foundation model as “any model that is trained on broad data (generally using self-supervision at scale) that can be adapted to a wide range of downstream tasks”
- Training:
Trained on vast amounts of text data (websites, books, etc.) to learn general language patterns and information. - Capabilities:
Good at general language understanding, but may not be effective at following specific instructions or performing specialized tasks without further training. - Example:
A foundation could be used as a starting point for developing a chatbot or a text summarization tool, but it would require fine-tuning to perform those tasks effectively.
Example foundation models: