Internal error when clicking on the timeline button

[irevoire]

Hey!

I finally got everything set up correctly, wrote a small client to interact with rebenchDB that seems to work, and took a bunch of old benchmark runs I had (~900 commits and about 20 datapoints per commit, I would say).

Everything was accepted correctly from what I see, but in the end the view is really strange, I can't compare two commits, and the timeline button doesn't work and return an internal error.

I think the internal error is generated because of this message in the logs:

rebenchdb-app-1  | 
rebenchdb-app-1  |   TypeError: ejs:50
rebenchdb-app-1  |       48| <div class="container-fluid"><div class="row flex-xl-nowrap">
rebenchdb-app-1  |       49|   <nav class="compare">
rebenchdb-app-1  |    >> 50|   {% for (const b of it.benchmarks) {
rebenchdb-app-1  |       51|        for (const e of b.exec) { %}
rebenchdb-app-1  |       52|       <nav><span>{%= e.execName %}</span>
rebenchdb-app-1  |       53|       <a href="#se-{%= b.suiteId %}-{%= e.execId %}">{%= b.suiteName %}</a>
rebenchdb-app-1  |   
rebenchdb-app-1  |   it.benchmarks is not iterable
rebenchdb-app-1  |       at eval (eval at compile (/project/node_modules/ejs/lib/ejs.js:673:12), <anonymous>:36:27)
rebenchdb-app-1  |       at anonymous (/project/node_modules/ejs/lib/ejs.js:703:17)
rebenchdb-app-1  |       at renderTimeline (/project/src/backend/timeline/timeline.ts:52:16)
rebenchdb-app-1  |       at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
rebenchdb-app-1  | 

---

Here's the view I get right before clicking on the timeline button:

As you can see I can't compare two commits since I only have one column of commits.

---

And if it can help, here is a subset of what I sent to rebench:

Toggle me!

```json5 { // ... { "d": [ { "in": 1, "it": 10, "m": [ { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 }, { "c": 0, "v": 37195962.712608315 } ] } ], "runId": { "benchmark": { "name": "smol-songs.csv: words/the black saint and the sinner lady and the good doggo ", "suite": { "name": "search_songs_main_70c906d4/smol-songs.csv: words/the black saint and the sinner lady and the good doggo ", "desc": null, "executor": { "name": "Bench", "desc": null } }, "runDetails": { "maxInvocationTime": 0, "minIterationTime": 0, "warmup": null }, "desc": null }, "cmdline": "cargo bench --bench search_songs -- smol-songs.csv: words/the black saint and the sinner lady and the good doggo ", "location": "smol-songs.csv_ words/the black saint and the sinner lady and the good doggo ", "varValue": "the black saint and the sinner lady and the good doggo ", "cores": null, "inputSize": null, "extraArgs": null } }, { "d": [ { "in": 1, "it": 10, "m": [ { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 }, { "c": 0, "v": 55297151.1253019 } ] } ], "runId": { "benchmark": { "name": "smol-songs.csv: words/whathavenotnsuchforth and a good amount of words to pop to match the first one ", "suite": { "name": "search_songs_main_70c906d4/smol-songs.csv: words/whathavenotnsuchforth and a good amount of words to pop to match the first one ", "desc": null, "executor": { "name": "Bench", "desc": null } }, "runDetails": { "maxInvocationTime": 0, "minIterationTime": 0, "warmup": null }, "desc": null }, "cmdline": "cargo bench --bench search_songs -- smol-songs.csv: words/whathavenotnsuchforth and a good amount of words to pop to match the first one ", "location": "smol-songs.csv_ words/whathavenotnsuchforth and a good amount of words to pop to match", "varValue": "whathavenotnsuchforth and a good amount of words to pop to match the first one ", "cores": null, "inputSize": null, "extraArgs": null } } ], "criteria": [ { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" }, { "i": 0, "c": "total", "u": "ms" } ], "env": { "hostname": null, "cpu": "Bench", "clockSpeed": 0, "memory": 4194304, "osType": "Linux", "software": [], "userName": "Bench", "manualRun": false }, "source": { "repoUrl": "http://github.com/meilisearch/meilisearch", "branchOrTag": "main", "commitId": "70c906d4", "commitMsg": "Merge #3576\n\n3576: Add boolean support for csv documents r=irevoire a=irevoire\n\nFixes https://github.com/meilisearch/meilisearch/issues/3572\r\n\r\n## What does this PR do?\r\nAdd support for the boolean types in csv documents.\r\nThe type definition is `boolean` and the possible values are\r\n- `true` for true\r\n- `false` for false\r\n- ` ` for null\r\n\r\nHere is an example:\r\n```csv\r\n#id,cute:boolean\r\n0,true\r\n1,false\r\n2,\r\n```\n\nCo-authored-by: Tamo \n", "authorName": "bors[bot]", "authorEmail": "26634292+bors[bot]@users.noreply.github.com", "committerName": "GitHub", "committerEmail": "noreply@github.com" }, "experimentName": "search_songs", "experimentDescription": null, "startTime": "2023-03-14T12:28:12Z", "endTime": null, "projectName": "Milli's benchmark" } ```

Which, even if it's super ugly and with a lot of repetitions, was accepted into rebenchDB;

[/home/irevoire/rebenchdb-client/src/lib.rs:18:9] ret = Response[status: 201, status_text: Created, url: http://localhost:33333/rebenchdb/results]

[smarr]

Sorry, examining a PhD today. I'll try to look at it late this week. Could you possibly give me a SQL dump of the database? This way, I'd be sure to have the same data in the database for testing.

[irevoire]

Sorry, examining a PhD today. I'll try to look at it late this week.

No problem; hope they did well :grin:

Could you possibly give me a SQL dump of the database? This way, I'd be sure to have the same data in the database for testing.

Sorry, I struggled to get one and, in the end, didn't know how to do it. I tried a bunch of stuff with docker and volumes, but nothing worked, and I still can't run rebenchDB locally :pensive:

I wrote a reproducer, though; if you're ok with installing rust, maybe that should put you in the same situation as I am;

% curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh # install rust, C/C from https://rustup.rs/
% git clone https://github.com/irevoire/meilisearch_upload_milli_benchmark_to_rebenchdb
% cd meilisearch_upload_milli_benchmark_to_rebenchdb
% cargo run --release

That should do it if rebenchDB can be accessed on localhost:33333. If you need to change the addr you can specify it at the end: cargo run --release -- http://addr.to.rebench.

Or, if you prefer, I also made a Linux binary that you should be able to execute as-is and should reproduce the issue as well: reproducer.gz

I had to compress it in order to send it through GitHub thus, you'll need to decompress it first:

% wget https://github.com/smarr/ReBenchDB/files/14037721/reproducer.gz
% gunzip reproducer.gz
% chmod +x reproducer
% ./reproducer [addr to rebenchDB if needed]

I can also send you a binary for macOS aarch64 if you prefer.

[irevoire]

Hey, we tried a way simpler use case, and we still don't know how to make it work:pensive: It should be easier to reproduce on your case because it's composed of only two jons files instead of a thousands. rebench.json rebench-other-commit.json

[smarr]

Sorry... Unfortunately this is really not my day job. So took a long while to get to look at this, but I believe I fixed this with this commit: https://github.com/smarr/ReBenchDB/pull/186/commits/93929f36256465268576f30596226a41a21fd237

There might be another underlying reason why no benchmarks were found for the project though. Not sure.

One of the reasons for the delay was that I finally wanted to get #100 out of the way before delving into the various other issues. This is a major change of the database schema and has a bit of a bug tail... And it also changed the API. To keep my own pain as minimal as possible, I didn't add backwards compatibility though. (the code for it is there, since I use it to convert the test data, it's just not hooked up) Details for the new API are on the expanded PR summary of #100.

[irevoire]

Hey, thanks for the update!

We had to finish this subject a few weeks ago and ended up making our own frontend in the meantime. I’m still interested in rebenchDB, though, and I’ll let you know if I end up revisiting the project one day.

Thanks

[smarr]

Ah, nice. Yeah, sometimes a quickly hacked together custom solution is really much better suited for the needs :)