hholt

TimeCapsuleLLM (Pre-1700)

Train a language model using only sources dated 1700 or earlier.

Core Idea

  • Train from scratch on time-limited data
  • Keep outputs historically grounded
  • Reduce modern leakage

Data Scope

  • Include sources dated <= 1700
  • Exclude modern commentary editions
  • Use public-domain primary texts

Planned Steps

  1. Collect and date-verify sources
  2. Clean OCR and metadata noise
  3. Train tokenizer
  4. Train base model
  5. Evaluate style and leakage
  6. Fine-tune a chat version

Why This Project

Test whether strict pre-1700 filtering leads to different behavior than modern-pretrained or later-period historical models.