Independent research lab · Oxford, UK
Understanding what happens to a model while it is being trained.
Ochs Labs studies the training dynamics of modern generative models and large language models — the optimisation geometry that decides whether training is stable, and the training data that installs a behaviour long before that behaviour becomes visible to an evaluation.
What we work on
Optimisation geometry
Why some training objectives are robust to hyperparameters and others are not. We analyse the local Jacobian of the training dynamics near equilibrium and derive what the spectrum predicts about learning-rate sensitivity, oscillation, and collapse.
Training data attribution
Which documents in a fine-tuning corpus caused a given behaviour — and, just as importantly, at which point in the training run that causal signal is strongest. Attribution is almost always measured at the final checkpoint; we think that is the wrong place to look.
Emergent misalignment
Narrow fine-tunes that generalise into broadly misaligned behaviour. The mechanistic transition that installs the behaviour precedes its appearance in evaluations by hundreds of optimiser steps — a window in which the problem is, in principle, detectable in advance.
Research
2 projectsWhen should you measure influence? Attribution timing and emergent misalignment
Training data attribution is normally computed once, at the end of a fine-tuning run. But when a narrow fine-tune installs a broadly misaligned behaviour, the mechanistic transition that installs it happens well before the behaviour is measurable by evaluations. We hold the attribution method fixed and vary only the checkpoint at which it is applied, asking whether the documents responsible for a behaviour are recovered more reliably at the transition than at convergence. If they are, every final-checkpoint attribution result is a lower bound, and attribution becomes a leading indicator of misalignment rather than a post-mortem.
Llama-3.1-8B · rank-1 LoRA organisms · per-step checkpointing · influence functions
Why adversarial diffusion trains more stably than GANs: a local Jacobian view
Diffusion models train far more reliably than GANs. Does that reliability come from the scalar ELBO objective, or from the step-wise denoising structure itself? We answer locally, by comparing the Jacobian of the training dynamics near equilibrium for one-step GANs, adversarial diffusion, and ELBO diffusion. Min–max training rotates around the equilibrium; step-wise denoising averages that rotation down by a factor of order log T / T; and a scalar objective removes it entirely.
Local stability · Jacobian spectra · DDGAN · learning-rate robustness
Publications
About
Ochs Labs is an independent research lab founded and run by Florian Ochs, an Oxford MSc Mathematical Sciences graduate (Distinction) working on the theoretical foundations of statistics and machine learning. Previous research experience spans probability metrics and generative-model training dynamics, through positions at Luiss Guido Carli University in Rome and Allianz in Munich.
The through-line across the work is a preference for claims that can be measured. Where the theory side derives what the local geometry of an objective predicts, the empirical side asks whether the corresponding diagnostic actually moves in a real training run — on toy mixtures where the Jacobian can be computed exactly, and on open-weight language models where it cannot.
The current agenda is safety-motivated: if a fine-tune installs an unwanted behaviour, we want to know which data did it and how early that is detectable. Work is shared openly, with code and checkpoints released alongside results wherever licensing permits.
Experiments run on single-node A100 / H100 instances. The attribution work in particular requires per-step checkpointing and per-document gradient caches across several fine-tuning seeds, which makes compute, rather than method, the usual binding constraint.
Oxford, United Kingdom · Curriculum vitae (PDF)
Contact
Enquiries about the work, collaborations, or access to code and checkpoints: florian.ochs@ochslabs.xyz