Auto-Rubric as Reward — explicit criteria beat scalar labels
arXiv paper proposes ARR (Auto-Rubric as Reward), a framework that replaces opaque scalar reward signals with explicit, decomposed criteria for aligning multimodal generative models.
Standard RLHF collapses nuanced human preferences into pairwise labels, creating reward-hacking vulnerabilities. ARR recovers compositional structure by generating rubrics that are reliable, scalable, and data-efficient.