Recommended model parameters
Recommended serving parameters, environment variables, and hardware for supported Managed Inference models.
CosmicAC recommends these serving parameters, environment variables, and hardware for the following models. The Job configuration reference defines each parameter. For how to apply them, see Create a Managed Inference Job.
Tensor parallel equals GPUs per replica for every model on this page. The web interface applies both from the model master, so these values matter most when you create a job from the CLI.
Qwen3-VL-235B-A22B-Thinking-FP8
Model ID Qwen/Qwen3-VL-235B-A22B-Thinking-FP8.
Serving parameters
| Parameter | Value |
|---|---|
| Runtime image | vllm/vllm-openai:v0.15.1 |
| Data type | Auto |
| Quantisation | None |
| Tensor parallel | 8 |
| GPUs per replica | 8 |
| GPU memory utilization | 0.9 |
| Max model length | 131072 |
| Max concurrent sequences | 64 |
| Reasoning parser | deepseek_r1 |
| Video & image input | Yes |
| Root disk size | 500 GB |
Environment variables
TRUST_REMOTE_CODE=true
ENABLE_EXPERT_PARALLEL=true
ENFORCE_EAGER=trueHardware
| Resource | Value |
|---|---|
| GPUs | 8 H100 80 GB |
| CPU cores per GPU | 16 |
| RAM per GPU | 150 GB |
Qwen3.5-122B-A10B
Model ID Qwen/Qwen3.5-122B-A10B.
Serving parameters
| Parameter | Value |
|---|---|
| Runtime image | vllm/vllm-openai:v0.17.1 |
| Data type | Auto |
| Quantisation | FP8 |
| Tensor parallel | 8 |
| GPUs per replica | 8 |
| GPU memory utilization | 0.9 |
| Max model length | 32768 |
| Max concurrent sequences | 32 |
| Reasoning parser | deepseek_r1 |
| Video & image input | Yes |
| Root disk size | 500 GB |
Environment variables
TRUST_REMOTE_CODE=true
SWAP_SPACE=0
ENABLE_EXPERT_PARALLEL=true
ENFORCE_EAGER=trueMiniMax M2.5
Model ID MiniMaxAI/MiniMax-M2.5.
Serving parameters
| Parameter | Value |
|---|---|
| Runtime image | vllm/vllm-openai:v0.15.1 |
| Data type | Auto |
| Quantisation | None |
| Tensor parallel | 4 |
| GPUs per replica | 4 |
| GPU memory utilization | 0.85 |
| Max model length | 27000 |
| Max concurrent sequences | 256 |
| Reasoning parser | deepseek_r1 |
| Video & image input | Yes |
| Root disk size | 500 GB |
Environment variables
TRUST_REMOTE_CODE=true
SWAP_SPACE=0
ENABLE_EXPERT_PARALLEL=true
ENFORCE_EAGER=trueHardware
| Resource | Value |
|---|---|
| GPUs | 4 H100 80 GB |
| CPU cores per GPU | 16 |
| RAM per GPU | 150 GB |
Qwen2-VL-2B-Instruct
Model ID Qwen/Qwen2-VL-2B-Instruct.
Serving parameters
| Parameter | Value |
|---|---|
| Runtime image | vllm/vllm-openai:v0.15.1 |
| Data type | Auto |
| Quantisation | None |
| Tensor parallel | 1 |
| GPUs per replica | 1 |
| GPU memory utilization | 0.9 |
| Max model length | 32768 |
| Max concurrent sequences | 64 |
| Reasoning parser | default |
| Video & image input | Yes |
| Root disk size | 150 GB |
Environment variables
TRUST_REMOTE_CODE=true
ENFORCE_EAGER=true
SWAP_SPACE=0