ocurrent / current-bench

Experimental benchmarking infrastructure using OCurrent pipelines
Apache License 2.0
33 stars 17 forks source link

merlin benchmark #457

Closed 3Rafal closed 1 year ago

3Rafal commented 1 year ago

Hello :) I have created a PR that introduces merlin benchmark: https://github.com/ocaml/merlin/pull/1627

It seems to me that it executes correctly, but for some reason I get message No data for selected interval. and no graphs/results:

image

Job: https://autumn.ocamllabs.io/ocaml/merlin/pull/1627/base/master?worker=autumn&image=bench.Dockerfile Logs: http://autumn.ocamllabs.io:8081/job/2023-06-15/124937-ocluster-build-3cc6b4

I've checked my output with cb-check and it seems correct:

Correctly parsed some benchmarks:
{
  "name": "Merlin benchmark",
  "results": [
    {
      "name": "buffer-typed",
      "metrics": [
        {
          "name": "case-analysis",
          "description": "",
          "value": [
            2.0, 2.0, 6.0, 2.0, 3.0, 1.0, 41.0, 37.0, 35.0, 34.0, 34.0, 34.0,
            34.0, 79.0, 64.0, 48.0, 56.0, 38.0, 38.0, 40.0, 39.0, 40.0, 49.0,
            37.0, 39.0, 38.0, 45.0, 37.0, 38.0, 0.0, 0.0, 0.0, 0.0, 0.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "complete-prefix",
          "description": "",
          "value": [
            2.0, 8.0, 12.0, 6.0, 11.0, 44.0, 40.0, 40.0, 45.0, 40.0, 43.0,
            40.0, 45.0, 82.0, 71.0, 54.0, 54.0, 44.0, 44.0, 49.0, 67.0, 48.0,
            43.0, 54.0, 37.0, 40.0, 43.0, 39.0, 39.0, 72.0, 102.0, 1.0, 1.0,
            1.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "errors",
          "description": "",
          "value": [
            2.0, 1.0, 1.0, 35.0, 38.0, 34.0, 124.0, 34.0, 34.0, 62.0, 49.0,
            42.0, 38.0, 128.0, 39.0, 101.0, 37.0, 37.0, 41.0, 143.0, 34.0,
            37.0, 0.0, 0.0, 0.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "expand-prefix",
          "description": "",
          "value": [
            9.0, 10.0, 10.0, 7.0, 9.0, 40.0, 41.0, 42.0, 47.0, 47.0, 40.0,
            39.0, 40.0, 82.0, 71.0, 68.0, 54.0, 45.0, 44.0, 44.0, 48.0, 48.0,
            44.0, 47.0, 43.0, 44.0, 44.0, 46.0, 40.0, 49.0, 56.0, 1.0, 1.0,
            1.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "locate",
          "description": "",
          "value": [
            2.0, 78.0, 3.0, 24.0, 7.0, 18.0, 34.0, 138.0, 37.0, 232.0, 36.0,
            116.0, 34.0, 110.0, 38.0, 38.0, 65.0, 261.0, 50.0, 213.0, 39.0,
            117.0, 40.0, 368.0, 39.0, 179.0, 38.0, 150.0, 51.0, 195.0, 37.0,
            62.0, 54.0, 156.0, 2.0, 0.0, 2.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "occurrences",
          "description": "",
          "value": [
            2.0, 2.0, 5.0, 2.0, 1.0, 1.0, 34.0, 34.0, 41.0, 37.0, 34.0, 34.0,
            35.0, 34.0, 34.0, 34.0, 75.0, 80.0, 48.0, 53.0, 41.0, 39.0, 40.0,
            39.0, 39.0, 40.0, 40.0, 37.0, 38.0, 38.0, 34.0, 34.0, 39.0, 37.0,
            0.0, 0.0, 0.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        },
        {
          "name": "type-enclosing",
          "description": "",
          "value": [
            2.0, 5.0, 4.0, 2.0, 2.0, 12.0, 39.0, 42.0, 71.0, 34.0, 42.0,
            34.0, 34.0, 34.0, 78.0, 133.0, 51.0, 51.0, 40.0, 48.0, 62.0,
            60.0, 42.0, 79.0, 37.0, 40.0, 39.0, 41.0, 34.0, 36.0, 44.0,
            166.0, 0.0, 0.0, 0.0, 0.0, 0.0
          ],
          "units": "ms",
          "trend": "",
          "lines": []
        }
      ]
    }
  ]
}

Can you help me with identifying the reason for this behavior?

punchagan commented 1 year ago

The benchmark has a name and you need to click on the Merlin benchmark link in the sidebar to view the results. See #423. But, there was also a bug with the GraphQL query when the name has a space, which make the sidebar link to not work well too. This was fixed in #462 and the benchmarks are now visible. See this link for instance.

You could also consider getting rid of the name in the benchmark to work around the current-bench UX issue in #423.

3Rafal commented 1 year ago

Thanks a lot @punchagan ! :)