Generate fresh DPO pairs by re-querying the model with
existing prompts and keeping your original "chosen" rewrites.
Installation:
Download the autodpo.py file and place it in a folder titled plugins in
the same folder from which you launch chathaiku_dev.py.
Workflow:
For each record in an input JSONL file with prompt and chosen fields,
the plugin clears conversation, sends the prompt to the current model
endpoint, captures the response as a new "rejected", and writes a fresh
pair to the output JSONL.
This is useful when:
- Your model has been updated since you collected the original pairs, and
you want fresh rejections reflecting current failure modes - You want to expand a small "chosen" corpus into a fresh DPO dataset
without re-doing the hand-rating work - You want to verify which prompts the model has actually improved on
(identical chosen and new-rejected = the model now does the right thing)
Usage in chat:
/autodpo -> show status and usage
/autodpo run -> run generation
/autodpo run --limit 20 -> only process first 20 records
/autodpo run --resume -> skip prompts already in output
/autodpo stop -> noop placeholder (Ctrl-C during run works too)
Input format (one JSON object per line):
{"prompt": "...", "chosen": "...", ...other fields ignored...}
Output format (one JSON object per line):
{"prompt": "...", "chosen": "", "rejected": "",
"history": [], "source": "autodpo", "model": "...", "ts": "..."}
Quality filters reject pairs where:
- The model's response is identical to chosen (no preference signal)
- The model's response is empty or unparsable
- Either field is missing or empty in the input