langchain.evaluation.criteria.eval_chain.resolve_criteria¶

langchain.evaluation.criteria.eval_chain.resolve_criteria(criteria: Optional[Union[Mapping[str, str], Criteria, ConstitutionalPrinciple, str]]) → Dict[str, str][source]¶

Resolve the criteria to evaluate.

Parameters

criteria (CRITERIA_TYPE) –

The criteria to evaluate the runs against. It can be:

  • a mapping of a criterion name to its description

  • a single criterion name present in one of the default criteria

  • a single ConstitutionalPrinciple instance

Returns

A dictionary mapping criterion names to descriptions.

Return type

Dict[str, str]

Examples

>>> criterion = "relevance"
>>> CriteriaEvalChain.resolve_criteria(criteria)
{'relevance': 'Is the submission referring to a real quote from the text?'}