TinyLlama-1.1B — SFT + DPO Fine-Tuned

Two-stage LoRA fine-tuning pipeline: SFT on OpenOrca → DPO on UltraFeedback. Best SFT: sft_5 (BLEU 0.0451, BERTScore F1 0.8811) · Best DPO: dpo_4 (BLEU 0.0492, BERTScore F1 0.8757)

64 512
0.1 1.5
0.5 1
1 1.5

Model: ogx786/tinyllama-sft-dpo-finetuned · Base: TinyLlama-1.1B-Chat-v1.0 ·