Blog
Adept Fuyu-Heavy: A new multimodal model
January 24, 2024 — Adept Team
Adept Fuyu-Heavy is a new multimodal model designed specifically for digital agents.
Fuyu-8B: A Multimodal Architecture for AI Agents
October 17, 2023 — Rohan Bavishi, Erich Elsen, Curtis Hawthorne, Maxwell Nye, Augustus Odena, Arushi Somani, Sağnak Taşırlar
We’re open-sourcing Fuyu-8B - a small version of the multimodal model that powers our product.
The Adventure of the Errant Hardware
September 19, 2023 — Erich Elsen, Curtis Hawthorne, Arushi Somani
A tale of mystery, intrigue and derring-do. We recount our investigation into curious errors occuring during our large training runs–clues found, causes deciphered and solutions implemented.
Releasing Persimmon-8B
September 7, 2023 — Erich Elsen, Augustus Odena, Maxwell Nye, Sağnak Taşırlar, Tri Dao, Curtis Hawthorne, Deepak Moparthi, Arushi Somani
We’re open-sourcing Persimmon-8B, the most powerful fully permissively-licensed language model with <10 billion parameters.
FlashAttention: Fast Transformer training with long sequences
January 17, 2023 — Tri Dao
Transformers have grown deeper and wider, but training them on long sequences remains difficult. The attention layer at their heart is the compute and memory bottleneck: doubling the sequence length would quadruple the runtime and memory requirements.