ssl-hep / ServiceX_frontend

Client access library for ServiceX
https://servicex-frontend.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
5 stars 11 forks source link

The marching ants do not deal with evolving total numbers of files #464

Closed gordonwatts closed 2 months ago

gordonwatts commented 2 months ago

Running on a vary large dataset means that it takes servicex a while to determine the complete list of files. Foe example, data18_13TeV:data18_13TeV.periodAllYear.physics_Main.PhysCont.DAOD_PHYSLITE. This has some 64K files in it. While the dashboard seems to report things (mostly) correctly:

image

The marching ants seems to take the first number back from SX and report that for the complete run (rather than updating):

image

It can take quite a while to determine what the list of files is - so this means the marching ants are incorrect!