search

dask / distributed

A distributed task scheduler for Dask
https://distributed.dask.org
BSD 3-Clause "New" or "Revised" License
1.58k stars 720 forks source link

Refactor report generation logic into a separate repository #6079

Open sjperkins opened 2 years ago

sjperkins commented 2 years ago

The following file contains report generation logic that could potentially be utilised by multiple testing and benchmarking efforts

https://github.com/dask/distributed/blob/main/continuous_integration/scripts/test_report.py

It may be beneficial to split report generation logic into a separate repository:

/cc @fjetter @jrbourbeau @ncclementi @crusaderky @jcrist @bryanwweber @ian-r-rose @phobson

fjetter commented 2 years ago

I'm not sure if this is worth setting up another repository. Even if it is in an extra repository, how would you reuse the code? git submodules? Would you actually package it to be installable?

sjperkins commented 2 years ago

I'm not sure if this is worth setting up another repository. Even if it is in an extra repository, how would you reuse the code? git submodules? Would you actually package it to be installable?

Perhaps it would be useful if it lived in the dask/dask repository? Then it could be available to all benchmarks that use dask.

fjetter commented 2 years ago

I don't have a strong opinion on this. My concern is that I don't want anybody worry about "public API" when changing CI tooling and scripts like these. It's just a few hundred lines of code. I don't think vendoring/copying is a big deal.

sjperkins commented 2 years ago

It's just a few hundred lines of code. I don't think vendoring/copying is a big deal.

Yes, maybe its not worth identifying common elements at this point.