HEPData / hepdata

Repository for main HEPData web application
https://hepdata.net
GNU General Public License v2.0
40 stars 11 forks source link

converter: large records timeout after 220 seconds #788

Open GraemeWatt opened 2 months ago

GraemeWatt commented 2 months ago

I'm creating this issue to keep track of which records timeout after 220 seconds due to the conversion taking longer than this limit.

  1. https://www.hepdata.net/record/ins1740909?version=2
  2. https://www.hepdata.net/record/ins1798511?version=1

The hepdata-converter package should be made more efficient (for example, by resolving HEPData/hepdata-converter#50, HEPData/hepdata-converter#51, HEPData/hepdata-converter#52, HEPData/hepdata#563). A workaround is for a user to run the conversion offline using the provided Docker image hepdata/hepdata-converter-ws where the hepdata-converter package (docs) is provided with all dependencies. For example, after clicking "Download All" then "YAML with resource files" to download the original directory HEPData-ins1740909-v2, conversion to the yoda format can be performed with:

docker run -v $PWD:$PWD -w $PWD --rm -it hepdata/hepdata-converter-ws hepdata-converter -o yoda -r ATLAS_2019_I1740909 -d 10.17182/hepdata.89321.v2 HEPData-ins1740909-v2 HEPData-ins1740909-v2-yoda.yoda

The -o option can be one of yoda, yoda1, csv, root, yaml. The -r option should be omitted unless converting to the yoda or yoda1 format.