ComponentsBundles
Cleanlab
Cleanlab components available in ottom8: Cleanlab Evaluator, Cleanlab RAG Evaluator, Cleanlab Remediator.
Cleanlab Evaluator
Evaluates any LLM response using Cleanlab and outputs trust score and explanation.
| Input | Type | Required |
|---|---|---|
| Cleanlab API Key | str | Yes |
| Prompt | str | Yes |
| Response | str | Yes |
| System Message | str | No |
Advanced options: Cleanlab Evaluation Model, Quality Preset
| Output | Type |
|---|---|
| Response | Message |
| Trust Score | number, float |
| Explanation | Message |
Cleanlab RAG Evaluator
Evaluates context, query, and response from a RAG pipeline using Cleanlab and outputs trust metrics.
| Input | Type | Required |
|---|---|---|
| Cleanlab API Key | str | Yes |
| Context | str | Yes |
| Query | str | Yes |
| Response | str | Yes |
Advanced options: Cleanlab Evaluation Model, Quality Preset, Run Context Sufficiency, Run Query Ease, Run Response Groundedness, Run Response Helpfulness
| Output | Type |
|---|---|
| Response | Message |
| Trust Score | number, float |
| Explanation | Message |
| Other Evals | Data, dict |
| Evaluation Summary | Message |
Cleanlab Remediator
Remediates an untrustworthy response based on trust score from the Cleanlab Evaluator, score threshold, and message handling settings.
| Input | Type | Required |
|---|---|---|
| Trust Score | other | Yes |
| Explanation | str | No |
| Fallback Answer | prompt | No |
| Response | str | Yes |
| Show Untrustworthy Response | bool | No |
| Threshold | float | Yes |
| Warning for Untrustworthy Response | prompt | No |
| Output | Type |
|---|---|
| Remediated Message | Message |