ansible-roles-vllm

Deploys vLLM OpenAI-compatible inference server as a rootless Podman container.

• https://docs.vllm.ai/en/stable/deployment/docker.html
• https://hub.docker.com/r/vllm/vllm-openai/tags

Task Configuration

- name: Setup vLLM
  hosts: somehost
  become: true
  roles:
    - role: vllm
      vllm_model: meta-llama/Llama-3.2-1B
      vllm_hf_token: "{{ vault_vllm_hf_token }}"

Deployment and Removal

systemctl --user stop container-vllm.service

Deploy

./run.sh deploy actual --tags vllm --limit somehost

Remove

./run.sh deploy actual --tags vllm --extra-vars "deployment_state=absent" --limit somehost