Marcin Copik, Grzegorz Kwasniewski, Maciej Besta, Michal Podstawski, Torsten Hoefler:
SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing
(In Proceedings of the 22nd International Middleware Conference, presented in Qu\'{e}bec city, Canada, ACM, ISBN: 9781450385343, Dec. 2021)
Abstract
Function-as-a-Service (FaaS) is one of the most promising directions for the future of cloud services, and serverless functions have immediately become a new middleware for building scalable and cost-efficient microservices and applications. However, the quickly moving technology hinders reproducibility, and the lack of a standardized benchmarking suite leads to ad-hoc solutions and microbenchmarks being used in serverless research, further complicating metaanalysis and comparison of research solutions. To address this challenge, we propose the Serverless Benchmark Suite: the first benchmark for FaaS computing that systematically covers a wide spectrum of cloud resources and applications. Our benchmark consists of the specification of representative workloads, the accompanying implementation and evaluation infrastructure, and the evaluation methodology that facilitates reproducibility and enables interpretability. We demonstrate that the abstract model of a FaaS execution environment ensures the applicability of our benchmark to multiple commercial providers such as AWS, Azure, and Google Cloud. Our work facilities experimental evaluation of serverless systems, and delivers a standardized, reliable and evolving evaluation methodology of performance, efficiency, scalability and reliability of middleware FaaS platforms.
Documents
download article: download slides:
Recorded talk (best effort)
BibTeX
@inproceedings{, author={Marcin Copik and Grzegorz Kwasniewski and Maciej Besta and Michal Podstawski and Torsten Hoefler}, title={{SeBS: A Serverless Benchmark Suite for Function-as-a-Service Computing}}, year={2021}, month={Dec.}, booktitle={Proceedings of the 22nd International Middleware Conference}, location={Qu\'{e}bec city, Canada}, publisher={ACM}, isbn={9781450385343}, source={http://www.unixer.de/~htor/publications/}, }
