Requirements: basic understanding of deep learning platform, Serverless Computing, eg. Pytorch, Tensorflow
Key Paper: ServerlessLLM: Locality-Enhanced Serverless Inference for Large Language Models (OSDI 24)
Yao Fu and Leyang Xue and Yeqi Huang and Andrei-Octavian Brabete and Dmitrii Ustiugov, et.al.
Source Code:
https://github.com/ServerlessLLM/ServerlessLLM
Reference:
FlexGen: High-Throughput Generative Inference of Large Language Models with a Single GPU;
Ying Sheng, Lianmin Zheng, Binhang Yuan, Zhuohan Li, et al. (https://arxiv.org/abs/2303.06865)
Fast Distributed Inference Serving for Large Language Models; ; Bingyang Wu∗ Yinmin Zhong∗ Zili Zhang∗ Gang Huang Xuanzhe Liu Xin Jin.
Tabi: An Efficient Multi-Level Inference System for Large Language Models;
*Yiding Wang, Kai Chen, Haisheng Tan, and Kun Guo. 2023. Tabi: An Efficient Multi-Level Inference System for Large Language Models. In Proceedings of the Eighteenth European Conference on Computer Systems (EuroSys '23). Association for Computing Machinery, New York, NY, USA, 233–248. https://doi.org/10.1145/3552326.3587438*
ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning; *Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, Shaden Smith, and Yuxiong He. 2021. ZeRO-infinity: breaking the GPU memory wall for extreme scale deep learning. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (SC '21). Association for Computing Machinery, New York, NY, USA, Article 59, 1–14. https://doi.org/10.1145/3458817.3476205*
Requirements: basic understanding of deep learning platform, Serverless Computing, eg. Pytorch, Tensorflow
Key Paper: ElasticFlow: An Elastic Serverless Training Platform for Distributed Deep Learning
Gu, Diandian, et al. [ASPLOS 2023]