Peilin Li
Core Developer of KTransformers
Peilin Li is a Ph.D. candidate at Tsinghua University, under the supervision of Associate Professor Mingxing Zhang. He is involved in the development of the Qujing Technology engine, with research interests in GPU/CPU heterogeneous systems for inference and fine-tuning, as well as inference optimization for large-scale heterogeneous and homogeneous clusters. He is a core contributor to KTransformers, a framework designed for cost-efficient inference and fine-tuning of large-scale models, and currently leads the development of fine-tuning capabilities and the maintenance of its open-source ecosystem. The project has earned more than 19,000 stars on GitHub and has collaborated extensively with leading open-source large-model training and inference frameworks, including LlamaFactory and SGLang.
Topic
KTransformers: A Cost-Efficient Training and Inference Platform for Algorithm–System Co-Design of Large-Scale Models
As large-scale Mixture-of-Experts (MoE) models continue to grow, algorithm research is increasingly constrained by GPU memory and cluster costs. KTransformers is an open-source heterogeneous training and inference platform designed for large-scale models. It leverages CPU/GPU collaboration to host sparse Experts, connects LlamaFactory with SGLang, and provides extensible placement, backend, and training/inference interfaces, creating an environment for jointly validating algorithmic ideas and system-level optimizations. Drawing on real-world community practices, this talk will demonstrate a complete workflow spanning fine-tuning, inference, and evaluation, along with case studies of domain-specific fine-tuning. The goal is to help academic labs and small-to-medium-sized teams conduct reproducible and research-driven research on large-scale models under constrained computing resources. Outline Training and Inference Bottlenecks of Large-Scale MoE Models on a Single Machine Exploring resource constraints and key bottlenecks in memory, computation, communication, and scheduling. KTransformers’ CPU/GPU Heterogeneous Architecture and Key Optimizations An overview of how KTransformers leverages CPU/GPU heterogeneity to enable cost-efficient local training and inference. An End-to-End Training, Inference, and Evaluation Workflow From fine-tuning with LlamaFactory to inference with SGLang, covering the complete iteration loop. Community Collaboration: Case Studies in Domain-Specific Fine-Tuning Sharing practical research cases and community-driven exploration in specialized domains. Key Takeaways Understand the core bottlenecks of large-scale MoE models in memory, computation, communication, and scheduling, and learn how KTransformers leverages CPU/GPU heterogeneity to enable localized, cost-efficient training and inference. Learn a low-cost workflow based on KTransformers, LlamaFactory, and SGLang, and apply it to your own hardware for iterative fine-tuning, deployment, and evaluation. For enterprises: Learn from real-world cases how to translate domain-specific requirements into reproducible experiments. For academic institutions: Gain practical approaches for conducting algorithm–system co-design research on large-scale models with limited computing resources.
