Skills : Python, TGI, Triton, vLLM,
The MLOps discipline adapted for large language models — owns the infrastructure for deploying, monitoring, versioning, and scaling LLM-based systems in production, including cost management and reliability.
Job Description:
The MLOps discipline adapted for large language models — owns the infrastructure for deploying, monitoring, versioning, and scaling LLM-based systems in production, including cost management and reliability.
Key Skills:
- Model serving infrastructure (vLLM, TGI, Triton, or managed API orchestration)
- Observability/monitoring for LLMs (latency, token cost, drift, hallucination rate)
- CI/CD for ML systems, model versioning
- Kubernetes, Docker, cloud infra (AWS/GCP/Azure)
- Cost optimization (caching, batching, model routing)
Experience Required:
- 3–7 years in MLOps, DevOps, or infrastructure engineering
- Direct experience running LLM workloads at production scale is a strong differentiator