Language models and efficient fine-tuning
AutoGate-QLoRA: URL-only ad generation
Master’s research on generating short advertising headlines and descriptions from a landing-page URL while training only a small part of a language model.
Master’s research · Private research code
The question
Can a language model generate a short advertising headline and description from a landing-page URL while training only a small number of parameters? My master’s project explores this task using example ads as training labels. The input is the URL text itself; the model does not retrieve the page.
What I built
The pipeline prepares and validates examples, trains several adapter methods, generates structured headlines and descriptions, and evaluates the outputs. Adapters are small trainable additions to a largely frozen language model. QLoRA combines them with a quantized base model.
Examples from the same URL stay together when the data are split. Generated text is checked for format, completeness, and duplication, with results summarised by output field and URL group.
A fair comparison
The current experiment asks whether selecting particular adapter sites helps more than choosing sites at random. Both methods begin with the same checkpoint, retain exactly the same adapter cost and number of sites, and receive the same further training. These controls separate the effect of site selection from differences in the starting model or training budget.
Findings and next steps
The exploratory study found closer matches to the example ads than training with shuffled targets. Output completeness was assessed separately from text-matching scores.
The newer adapter-site selection protocol is implemented and has passed CPU tests. Its full GPU comparison is the next step in testing whether learned selection adds value beyond equally sized random selections.
The research code is private. I can discuss the project’s methods and current direction through the contact page.