prefetch heartbeat/progress does not seem to work

[jlongo62]

I am using the SRAToolkit container ncbi/sra-tools ID=SRR9046178

My prefetch is: prefetch $ID --resume yes -H 1 -p --output-directory /aci/mnt or prefetch $ID --resume yes --progress -heartbeat 1 --output-directory /aci/mnt

My expectation (from the help file) is the output will update the progress every 1 minute, hopefully in a log fashion (not a gui type display as current). Instead, I get a running continuous progress update as if the --heartbeat is not recognized. This is unusable for a log file. (I plan to use the log to tell me when a prefetch is hung so I can restart the container)

[klymenko]

You are right, both --progress and --heartbeat are designed for users, not for log file.

[jlongo62]

as far as I can tell heartbeat does nothing. progress creates a changing % status line to stdout heartbeat seems to do nothing, by itself no data appears in log or stdout

[klymenko]

--heartbeat was created to manage aspera output - it's not used anymore.

[jlongo62]

how am I supposed to know when prefetch hangs ? I have a number of items where prefetch just hangs with no output.

[jlongo62]

My suggestion is heartbeat will output a %complete every n minutes. ie: --heartbeat 5 returns: 04:03:15 36% 04:09:15 42% etc...

Then we can check the log tail, and if it does not changes, assume a hang and restart. Right now there is no way to know if it prefetch has hung or just busy.

[klymenko]

It is keeping retrying. We did not have any reports that prefetch hangs.

[klymenko]

Progress reports progress. No progress (nothing was downloaded) does not mean that in hangs. It means that it cannot download anything now and retries.

[klymenko]

You can run prefetch with -vvv to see that it reads from server. However it will be very chatty.

[jlongo62]

I see nothing happening for hours. Looks like a hang. process should be killed and restarted. but no way to know when to this.

[klymenko]

Run it with -p.