Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning

Hyundong Justin Cho1, Karishma Sharma2, Nicolaas Jedema2, Leonardo F.R. Ribeiro2, Alessandro Moschitti2, Ravi Krishnan2, Jonathan May1

1        2 
arXiv Slides Poster

Challenge: Personalizing Language Models for Text Generation in Realistic Settings

Language models are aligned to the collective voice of many, resulting in generic outputs that do not align with specific users' styles. Yet end users often have specific needs that call for personalized text generation, such as writing an email. Therefore, adapting LLMs for personalized text generation has drawn a growing interest, but prior work on personalized text generation either rely on large amounts of personal or parameter updates that result in cumbersome per-user model parameters. Can we instead personalize language models for text generation tasks without any parameter updates and with only a few examples?

Trial-Error-Explain In-Context Learning (TICL)

TICL Overview

To address this challenge, we propose Trial-Error-Explain In-Context Learning (TICL), an inference-only approach that front-loads inference compute to create a user-specific in-context learning prompt that does not require extra generation steps at test time. The key idea of TICL is to iteratively expand a few-shot in-context learning prompt via a trial-error-explain process, adding model-generated negative samples and explanations that provide fine-grained guidance towards a specific user's style.

TICL Teaser

We find that a providing model with its own "failures" as negative samples and corresponding explanations enables the model to learn stylistic context more effectively and overcome the bias towards structural and formal phrases observed in their zero-shot and few-shot outputs. TICL achieves favorable win rates on pairwise comparisons with LLM-as-a-judge up to 91.5% against the previous state-of-the-art and outperforms competitive tuning-free baselines for personalized alignment tasks of writing emails, essays and news articles.

BibTeX

@inproceedings{cho-etal-2025-tuning,
        title = "Tuning-Free Personalized Alignment via Trial-Error-Explain In-Context Learning",
        author = "Cho, Hyundong Justin  and
          Sharma, Karishma  and
          Jedema, Nicolaas Paul  and
          Ribeiro, Leonardo F. R.  and
          May, Jonathan  and
          Moschitti, Alessandro",
        editor = "Chiruzzo, Luis  and
          Ritter, Alan  and
          Wang, Lu",
        booktitle = "Findings of the Association for Computational Linguistics: NAACL 2025",
        month = apr,
        year = "2025",
        address = "Albuquerque, New Mexico",
        publisher = "Association for Computational Linguistics",
        url = "https://aclanthology.org/2025.findings-naacl.326/",
        pages = "5864--5885",
        ISBN = "979-8-89176-195-7"    
    }