Topic

Optimisation and Practice of an Efficient Reasoning Framework for Domestic AI Chips

INTRODUCTION: With the rapid development of AI technology, big models have become the core technology for all kinds of applications, bringing new structural opportunities for various industries. However, large models require high arithmetic power to perform efficient reasoning to meet the needs of business scenarios. In the face of the huge arithmetic gap under NVIDIA's limitation, the rise of domestic chips provides us with brand new opportunities and challenges. As a company focusing on providing model inference acceleration and arithmetic optimisation, we are committed to filling the gap of domestic optimisation, and we have built MLGuider - a large model inference acceleration tool library for domestic arithmetic, to fully release the potential of domestic hardware, and to accelerate the efficient landing of large models and related applications. This technical sharing will discuss domestic chip adaptation and optimisation, introduce our methodology and practical experience, and share specific cases and results. It promotes the close integration of domestic arithmetic and big models, and promotes the rapid landing and application of AI technology. Outline: 1. the development status and trend of domestic chips 2.Efficient inference framework MLGuider across chips 3.Application and practice case sharing