ComponentsBundles
vLLM
vLLM components available in ottom8: vLLM, vLLM Embeddings.
vLLM
Generates text using vLLM models via OpenAI-compatible API.
| Input | Type | Required |
|---|---|---|
| vLLM API Base | str | No |
| API Key | str | No |
| Input | str | No |
| Model Name | str | No |
| System Message | str | No |
| Temperature | slider | No |
Advanced options: JSON Mode, Max Retries, Max Tokens, Model Kwargs, Seed, Stream, Timeout
| Output | Type |
|---|---|
| Model Response | Message |
| Language Model | LanguageModel |
vLLM Embeddings
Generate embeddings using vLLM models via OpenAI-compatible API.
| Input | Type | Required |
|---|---|---|
| vLLM API Base | str | No |
| API Key | str | No |
| Model Name | str | No |
Advanced options: Chunk Size, Default Headers, Default Query, Dimensions, Max Retries, Model Kwargs, Request Timeout, Show Progress Bar, Skip Empty
| Output | Type |
|---|---|
| Embedding Model | Embeddings |