salsa-rs / salsa

A generic framework for on-demand, incrementalized computation. Inspired by adapton, glimmer, and rustc's query system.
https://salsa-rs.netlify.app/
Apache License 2.0
2.1k stars 143 forks source link

internal: add some initial input benchmarks #525

Closed davidbarsky closed 1 month ago

davidbarsky commented 1 month ago

Pulling from https://github.com/davidbarsky/salsa-benchmarks/; these should probably live in here.

netlify[bot] commented 1 month ago

Deploy Preview for salsa-rs canceled.

Name Link
Latest commit 0557605cb86b7de497bc3f7b915b18d49cd4de02
Latest deploy log https://app.netlify.com/sites/salsa-rs/deploys/66bb6ff920f9dd000838e444
MichaReiser commented 1 month ago

Patch to integrate it with codspeed / new db design

Subject: [PATCH] Use codspeed
---
Index: Cargo.toml
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/Cargo.toml b/Cargo.toml
--- a/Cargo.toml    (revision 892862325c0f58fe61b138413be2907367c7481c)
+++ b/Cargo.toml    (revision 171e116fcc50b0d2c1937fa84ab1e02d49fd9151)
@@ -34,7 +34,6 @@
 rustversion = "1.0"
 test-log = "0.2.11"
 trybuild = "1.0"
-criterion = "0.5.1"

 [[bench]]
 name = "compare"
Index: benches/compare.rs
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/benches/compare.rs b/benches/compare.rs
--- a/benches/compare.rs    (revision 892862325c0f58fe61b138413be2907367c7481c)
+++ b/benches/compare.rs    (revision 171e116fcc50b0d2c1937fa84ab1e02d49fd9151)
@@ -1,28 +1,13 @@
-use criterion::{criterion_group, criterion_main, BenchmarkId, Criterion};
+use codspeed_criterion_compat::{criterion_group, criterion_main, BenchmarkId, Criterion};
 use salsa::Setter;

-#[salsa::db]
-pub trait Db: salsa::Database {}
-
-#[salsa::db]
-#[derive(Default)]
-pub struct Database {
-    storage: salsa::Storage<Self>,
-}
-
-#[salsa::db]
-impl salsa::Database for Database {}
-
-#[salsa::db]
-impl Db for Database {}
-
 #[salsa::input]
 pub struct Input {
     pub text: String,
 }

 #[salsa::tracked]
-pub fn length(db: &dyn Db, input: Input) -> usize {
+pub fn length(db: &dyn salsa::Database, input: Input) -> usize {
     input.text(db).len()
 }

@@ -32,15 +17,16 @@
 }

 #[salsa::tracked]
-pub fn interned_length<'db>(db: &'db dyn Db, input: InternedInput<'db>) -> usize {
+pub fn interned_length<'db>(db: &'db dyn salsa::Database, input: InternedInput<'db>) -> usize {
     input.text(db).len()
 }

 fn mutating_inputs(c: &mut Criterion) {
-    let mut group: criterion::BenchmarkGroup<criterion::measurement::WallTime> =
-        c.benchmark_group("Mutating Inputs");
+    let mut group: codspeed_criterion_compat::BenchmarkGroup<
+        codspeed_criterion_compat::measurement::WallTime,
+    > = c.benchmark_group("Mutating Inputs");

-    let mut db = Database::default();
+    let mut db = salsa::DatabaseImpl::default();

     for n in &[10, 20, 30] {
         let base_string = "hello, world!".to_owned();
@@ -68,9 +54,11 @@
 }

 fn inputs(c: &mut Criterion) {
-    let mut group: criterion::BenchmarkGroup<criterion::measurement::WallTime> =
-        c.benchmark_group("Inputs");
-    let db = Database::default();
+    let mut group: codspeed_criterion_compat::BenchmarkGroup<
+        codspeed_criterion_compat::measurement::WallTime,
+    > = c.benchmark_group("Mutating Inputs");
+
+    let db = salsa::DatabaseImpl::default();

     group.bench_function(BenchmarkId::new("new", "InternedInput"), |b| {
         b.iter(|| {
codspeed-hq[bot] commented 1 month ago

CodSpeed Performance Report

Merging #525 will not alter performance

Comparing davidbarsky:master (0557605) with master (4657ac3)

Summary

โœ… 1 untouched benchmarks

๐Ÿ†• 7 new benchmarks

Benchmarks breakdown

Benchmark master davidbarsky:master Change
๐Ÿ†• amortized[Input] N/A 3.9 ยตs N/A
๐Ÿ†• amortized[InternedInput] N/A 3.3 ยตs N/A
๐Ÿ†• new[Input] N/A 14 ยตs N/A
๐Ÿ†• new[InternedInput] N/A 5.8 ยตs N/A
๐Ÿ†• mutating[10] N/A 19.8 ยตs N/A
๐Ÿ†• mutating[20] N/A 19 ยตs N/A
๐Ÿ†• mutating[30] N/A 20.3 ยตs N/A
davidbarsky commented 1 month ago

The test failure is due to https://github.com/rust-lang/rust/issues/129031; I'll add an allow in the example.