Progress bar

[lorenzwalthert]

I think it would be nice to use a progress bar so if you go to see the running GHA, you can see how much is left instead of / in addition to this generic status update.

[assignUser]

Yeah I thought about that too (e.g. {progressr}) but I am not sure if that is possible with the way gha logs work so I never tried it :D

[assignUser]

I just tested it with {progress} and the log just pauses until the progressbar is done, I don't think this is possible. But I will have another look at the grouping that doesn't really seem to work at the moment. Then each benchmark would be grouped, that would give an overall indivation at what point in script.R the process is and make it not as messy to look at xD

[assignUser]

I think this is much better than before, what do you think @lorenzwalthert: https://github.com/assignUser/simstudy/runs/4840074147?check_suite_focus=true#step:2:2501

[lorenzwalthert]

Much better already, I agree. A few more ideas:

• Also put the outputs above the benchmark itself in a log group, in particular those under Run touchstone::run_script("touchstone/script.R"):

• Benchmark: dist_binomial as a log group is already great, but we could indicate progress in that name of the group also, i.e. change it to Benchmark: dist_binomial [33% completed, 2 out of 3 remaining]. Then, I'd see something like

Benchmark: dist_binomial  [ 0% completed,  3 out of 100 remaining]
Benchmark: dist_norm      [33% completed,  1 out of 100 remaining]
Benchmark: dist_geometric [67% completed, 99 out of 100 remaining]
                         ^ ^             ^       
                           needs padding in multiple places for alignment 

Just realizing that this is probably not feasible easily because when running benchmark dist_binomial, we don't know how many benchmark_run() are ahead and detecting it with static code analysis is probably brittle and not worth the effort because quite hard. Only solution would be to wrap all benchmark_run() calls in one big function, which I don't think we should do, because even more wokr.

bm1 <- touchstone::benchmark_define(
  random_test = yourpkg::fun(data_clean),
  n = 2
)

bm2 <- touchstone::benchmark_define(
  non_random_test = yourpkg::fun(data_clean),
  n = 44
)

touchstone::benchmark_run(bm1, bm2)

And then if they have different n it's also not really a progress bar so... forget my last suggestion, way too much new complexity and work for little benefit.

[assignUser]

For the stuff above the benchmarks: I think the easiest way here is to group the output from branch_install and let the rest (more or less just asset pinning) stand on its own as doing else is not really worth the effort imho.

I agree your second idea would look very nice but is not possible as I don't think you can change group names once they are set + your other concerns :D

[assignUser]

https://github.com/assignUser/simstudy/runs/4840898017?check_suite_focus=true#step:2:2055