Senior Applied AI Software Engineer, Distributed Inference Systems
$148k/y-$287.5k/y
a day ago
Highlight
OnsiteFull-timeSenior
Summary
NVIDIA is seeking a Senior Applied AI Software Engineer for their Dynamo Project in Santa Clara, CA. The role focuses on building scalable inference systems for large language models in distributed GPU environments.
About the Role
As a Senior Applied AI Software Engineer, you will tackle high-impact challenges in distributed inference, including building the Kubernetes deployment for Dynamo, developing robust workload management systems, optimizing disaggregated serving, and enhancing intelligent routing and KV cache management. You will collaborate on design and development, contribute to open-source repositories, and write documentation.
About You
Required:
- BS/MS or higher in computer engineering, computer science, or related engineering (or equivalent experience).
- 5+ years of proven experience in related field.
- Strong proficiency in systems programming (Rust and/or C++), with experience in Python for workflow and API development.
- Deep understanding of distributed systems, parallel computing, and GPU architectures.
- Experience with cloud-native deployment and container orchestration (Kubernetes, Docker).
- Experience with large-scale inference serving, LLMs, or similar high-performance AI workloads.
- Familiarity with open-source development workflows (GitHub, continuous integration and continuous deployment).
- Excellent problem-solving and communication skills.
Preferred:
- Prior contributions to open-source AI inference frameworks (e.g., vLLM, TensorRT-LLM, SGLang).
- Experience with GPU resource scheduling, cache management, or high-performance networking.
- Understanding of LLM-specific inference challenges, such as context window scaling and multi-model agentic workflows.
Benefits
- Highly competitive salaries.
- Comprehensive benefits package including equity.
NVIDIA
Since its founding in 1993, NVIDIA (NASDAQ: NVDA) has been a pioneer in accelerated computing. The company’s invention of the GPU in 1999 sparked the growth of the PC gaming market, redefined computer graphics, ignited the era of modern AI and is fueling the creation of the metaverse. NVIDIA is now a full-stack computing company with data-center-scale offerings that are reshaping industry.
Santa Clara, CA