opentargets / issues

Issue tracker for Open Targets Platform and Open Targets Genetics Portal
https://platform.opentargets.org https://genetics.opentargets.org
Apache License 2.0
12 stars 2 forks source link

POS fails to run create_disk_data_tarballs for elastic search on genetics data #3542

Open jdhayhurst opened 1 month ago

jdhayhurst commented 1 month ago

Describe the bug As part of POS, for make image, the create_disk_data_tarballs step throws an error which breaks the image creation step. The code is as follows:

tar czf - -C ${path_source} . | gsutil cp - ${path_destination}

Note that removing the z flag from tar makes this work without error.

Observed behaviour Error is as follows:

google_metadata_script_runner[1831]: error while communicating with "startup-script" script: bufio.Scanner: token too long
google_metadata_script_runner[1831]: Script "startup-script" failed with error: signal: broken pipe

Expected behaviour When we run make image it completes without errors

To Reproduce Steps to reproduce the behaviour:

  1. using the full genetics platform data
  2. run make image