Research interests

I don't have publications. I have six years of experience building machine learning systems, most recently nearly two years at Unify building AI agents, ending with work on agents that improve from their own experience. This page is about the questions that work left me with, and the areas I want to work in next.

Post-training open models

Much of what makes a model useful for real work is added after pre-training. I'm most interested in reinforcement learning with rewards that can be checked, such as whether an answer is correct or a test passes, because the signal comes from the task rather than from a learned judge. That makes it easier to reason about, though not impossible to game. I want to understand when it teaches a model something new and when it only sharpens what the base model could already do, and what makes an environment worth training in: tasks that can be verified, whose difficulty can rise as the model improves, and that look like the work the model will actually do.

Continual learning

A model that keeps training on new work tends to lose some of what it already knew. I want to understand how much of that forgetting is avoidable, through replay, staying close to the original model, or training on the model's own outputs, and how to measure it honestly, so that a gain on new work is not paid for with a quiet loss elsewhere.

Distillation is where I think much of this gets practical. Fine-tuning a model on a stronger teacher's answers is the usual way to pass on a skill, but the student only ever sees the teacher's paths, not its own mistakes. Training on the model's own attempts, with a teacher or the model itself judging them token by token, as on-policy self-distillation does, seems to teach new behaviour while disturbing less of the old. I'd like to find out when that holds, and how it combines with reinforcement learning on the same tasks.

Learning from a company's own data

I expect companies that run agents to want them to keep improving from their own data and processes, and to do it privately, by post-training open models rather than sending that data to someone else's API. That makes the data flywheel the practical problem: deciding which of an agent's experiences are worth training on, checking them, and training on them without leaking anything or losing what the model already does well.

Agent harnesses

The harness around a model still matters. At Unify I led the redesign of an agent harness and its memory, which was built both to reuse work the agent had done before and to transfer what it learned to new work. Most of what memory gave turned out to be reuse rather than transfer. What I took from it is that much of what an agent learns on the job belongs in its weights, and that a harness is most useful when it gathers clean, verified experience to train on. How a harness and a training loop should divide that work is a question I want to keep working on.

I'm working through these from the basics: reading, reimplementing the core methods myself, and running small experiments of my own.