search

iris-hep / idap-200gbps-atlas

benchmarking throughput with PHYSLITE
6 stars 1 forks source link

Recursion errors at scale #37

Closed alexander-held closed 5 months ago

alexander-held commented 5 months ago

When running with sufficiently many files, we hit RecursionError during preprocessing. The solution to that should be https://github.com/CoffeaTeam/coffea/pull/1079. Once that is fixed we'll need a new AB-dev image. cc @matthewfeickert: I assume we don't need a coffea tag to build an image in principle? It's not critical enough to warrant building the image before this is reviewed and merged though.

alexander-held commented 5 months ago

resolved with latest coffea https://github.com/CoffeaTeam/coffea/releases/tag/v2024.4.1 and propagated to the AB-dev image

matthewfeickert commented 4 months ago

I assume we don't need a coffea tag to build an image in principle? It's not critical enough to warrant building the image before this is reviewed and merged though.

Correct. We can always build anything from a Git repo commit hash as well.