AI & Machine Learning

LLM Evaluation

LLM evaluation assesses the quality, accuracy, safety and reliability of large language models and the applications built on them, using benchmarks, task-specific test sets, automated scoring and human review.

Public benchmarks such as MMLU measure general knowledge and reasoning, but application-level evaluation matters more: a curated set of realistic inputs with expected outputs or grading criteria. Scoring methods include exact match for structured outputs, similarity metrics, LLM-as-a-judge, where another model grades responses against a rubric, and expert human review. RAG systems are evaluated for retrieval relevance, faithfulness to sources and answer correctness, and agents for task completion.

Evaluation is run when choosing a model, after changing prompts, retrieval settings or model versions, and continuously in production through sampling and user feedback. It prevents regressions and provides evidence for governance and audits.

Benchmarks can be contaminated when test questions appear in training data, and LLM judges show biases such as favouring longer answers. Safety evaluation includes red teaming for harmful outputs, prompt injection and data leakage. Results should be tracked over time, with test sets refreshed to reflect real usage.

Key points

• Combines benchmarks, application test sets, automated scoring and human review
• LLM-as-a-judge uses a model to grade responses against a rubric
• RAG systems are evaluated for retrieval relevance and faithfulness
• Benchmark contamination and judge bias limit reliability

Related terms

• Model EvaluationAI & Machine Learning
• AI HallucinationAI & Machine Learning
• Grounding (AI)AI & Machine Learning
• AI GuardrailsAI & Machine Learning
• Large Language Model (LLM)AI & Machine Learning
• Retrieval-Augmented Generation (RAG)AI & Machine Learning

Terms that refer to LLM Evaluation

• Prompt EngineeringAI & Machine Learning
• Reinforcement Learning from Human Feedback (RLHF)AI & Machine Learning