LLM Inference Frameworks and Optimization Engineer
$160k/y-$230k/y
2 days ago
Highlight
OnsiteFull-timeMid
Summary
Together AI is seeking an LLM Inference Frameworks and Optimization Engineer to join their team in San Francisco, CA. The role involves designing, developing, and optimizing distributed inference engines for large language models (LLMs) and multimodal models.
About the Role
As an LLM Inference Frameworks and Optimization Engineer, you will be responsible for developing fault-tolerant, high-concurrency distributed inference engines, implementing distributed inference strategies, and collaborating with hardware teams to optimize performance for GPUs and TPUs. You will also work closely with AI researchers to enhance model execution plans and optimize end-to-end model serving pipelines.
About You
Required:
- 3+ years of experience in deep learning inference frameworks, distributed systems, or high-performance computing.
- Familiarity with at least one LLM inference framework (e.g., TensorRT-LLM, vLLM, SGLang, TGI).
- Proficient in Python and C++/CUDA for high-performance deep learning inference.
- Deep understanding of Transformer architectures and LLM/VLM/Diffusion model optimization.
Preferred:
- Experience in developing software systems for large-scale data center networks with RDMA/RoCE.
- Familiarity with distributed filesystems (e.g., 3FS, HDFS, Ceph).
- Familiarity with open source distributed scheduling/orchestration frameworks, such as Kubernetes.
- Contributions to open-source deep learning inference projects.
Benefits
- Competitive compensation with a salary range of $160k–$230k plus equity.
- Health insurance and other competitive benefits.
Together AI
Together AI is a research-driven AI cloud infrastructure provider. Our purpose-built GPU cloud platform empowers AI engineers and researchers to train, fine-tune, and run frontier class AI models. Our customers include leading SaaS companies such as Salesforce, Zoom, and Zomato, as well as pioneering AI startups like ElevenLabs, Hedra, and Cartesia. We advocate for open source AI and believe that transparent AI systems will drive innovation and create the best outcomes for society.
San Francisco, CA