our first mini LLM
Levy 1 is a compact language model developed by Toothed AI: a lightweight, conversational AI that runs on accessible hardware.
It is pretrained from scratch on a single GPU, then refined with Supervised Fine-Tuning (SFT) to follow instructions, hold natural conversations and keep a consistent identity.
Levy 1's conversational capabilities are developed using a combination of carefully selected datasets covering:
SFT focuses learning on the assistant's responses, so Levy 1 produces appropriate answers instead of mirroring the input.
During testing, Levy 1 has already demonstrated basic conversational behavior and recognition of its own identity. The current development phase is focused on improving response quality, reducing unnecessary or unrelated generations, and making the model more consistent across different types of questions and conversations.
A major part of this process involves improving the quality and diversity of Levy 1's conversational and identity data rather than relying solely on larger training runs.
The goal of Levy 1 is to create a capable and efficient language model that demonstrates how much can be achieved with a relatively compact architecture and carefully designed training.
Levy 1 is an ongoing research project from Toothed AI, with future development centered around stronger reasoning, better instruction following, improved conversational consistency, and overall model quality.
How Levy 1 compares with other models of a similar size (around 125 million parameters) on standard zero-shot benchmarks.
One essential cookie keeps you signed in on the chat page. We use no advertising, analytics or tracking cookies. Read the Cookie Policy.