Overview
I built InfiniteDante to see how far a small, domain-focused language model could go with a narrow literary corpus and a browser as its only runtime. It generates Dante-style Italian text with a decoder-only Transformer exported to ONNX and executed through ONNX Runtime Web.
The project deliberately works at a very different scale from contemporary language models. Its value is not broad knowledge, but the fact that the entire path—from corpus preparation and tokenization to training, export, and interactive generation—can be inspected in one compact repository.
Model and training
The model has 15 Transformer layers, an embedding size of 256, eight attention heads, a context length of 256 tokens, and approximately 12.1 million parameters. I trained a custom byte-pair encoding tokenizer with a vocabulary of 512 tokens so the browser can use the same encoding produced during data preparation.
Training follows two stages. The first uses a broader collection of Italian literature, including medieval and classical sources. The second fine-tunes the model on works by Dante and closely related material. The training script tracks validation loss, applies early stopping, saves the best checkpoint, and exports the resulting model for the web interface.
This two-stage setup gives the model a wider Italian-language starting point before concentrating on the vocabulary and patterns of the target corpus. It also makes the effect of the Dante-focused phase visible in the recorded loss curves.
Browser experiment
The demo exposes temperature, top-k, top-p, repetition penalty, and generation speed. Text can be paused and edited before generation continues, turning the model into a small interactive writing experiment rather than a fixed sample page.
Running in the browser keeps the deployment simple: after loading the model, generation does not depend on an application server. The generated text remains exploratory—meter, historical language, and coherence vary with the prompt and sampling settings—but that variability is part of what the interface is designed to expose.
The live demo is the most direct way to try the model, while the repository contains the corpus pipeline, training code, ONNX export, loss history, and browser implementation.