To demonstrate that a compiler optimization, a memory management algorithm, or a synchronization technique is useful, a VM researcher needs benchmarks that demonstrate the desired behavior and, at the same time, capture representative aspects of real-world applications. During the last decade, multiple new programming paradigms have appeared on the JVM, including functional programming, big-data processing, parallel and concurrent programming, message passing, stream processing, and machine learning. The JVM has evolved as a platform, too: new features, such as e.g. method handles, variable handles, the invokedynamic instruction, lambdas, atomic and relaxed memory operations, present new challenges for JIT compilers and runtime environments. Existing benchmark suites do not capture these new features, because they were made in a time when such workloads did not exist.

To enable the pending Renaissance for VM and compiler research, new benchmarks are required. The Renaissance Suite is an ongoing, open-source effort to collect representative real-world workloads and advance the research and development of VMs. In this talk, we compare Renaissance against existing benchmark suites and we show how our benchmarks have helped us identify new optimizations for the Graal JIT compiler. We conclude by summarizing the future goals of the Renaissance Suite.