search

JuliaGraphs / Graphs.jl

An optimized graphs package for the Julia programming language
http://juliagraphs.org/Graphs.jl/
Other
451 stars 87 forks source link

Test bench CI PR #352

Open filchristou opened 4 months ago

filchristou commented 4 months ago

Not to merge

codecov[bot] commented 4 months ago

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 97.29%. Comparing base (de9cac7) to head (62e9733). Report is 6 commits behind head on master.

:exclamation: Current head 62e9733 differs from pull request most recent head bd5e39a. Consider uploading reports for the commit bd5e39a to get more accurate results

Additional details and impacted files ```diff @@ Coverage Diff @@ ## master #352 +/- ## ======================================= Coverage 97.29% 97.29% ======================================= Files 117 117 Lines 6814 6814 ======================================= Hits 6630 6630 Misses 184 184 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

filchristou commented 4 months ago

Unfortunately, the PostBenchmarks action from https://github.com/JuliaGraphs/Graphs.jl/blob/master/.github/workflows/postbench.yml was not triggered.

gdalle commented 4 months ago

Any idea why?

filchristou commented 4 months ago

Not sure at the moment.. But it also doesn't work in my fork so I i'll first fix it there and then make a PR here.

filchristou commented 3 months ago

I got the workflow chain to work, but BenchmarkCI.jl is failing. The problem is that BenchmarkCI.jl is engineered such that it uses the current github workflow event to receive the information about where to post. Since the new workflow is not triggered with a Pull-Request event the code fails here. I know I've been supporting BenchmarkCI.jl because it "just works", but it seems that in our case it just doesn't. The options are either to fork it, or make a customized thing out of PkgBenchmark.jl or use something else.

gdalle commented 2 months ago

Sorry that you went through all of this trouble to conclude that BenchmarkCI is not the thing for us. Let's put this on pause for now and think about what other options we have for benchmarking infrastructure.

@willow-ahrens is taking over maintenance of BenchmarkTools, and one of her goals is to figure out the CI conundrum (see the announcement on Discourse). I have explained to her the limits that @filchristou and I have discovered, and she said she'd ping when she has stuff to review in this area.

willow-ahrens commented 2 months ago

Is there a simple fix to benchmarkCI that would make it work, or do we need to rewrite it?