Closed mdsumner closed 2 months ago
wait
will probably help, but at a guess it'll still happen. It looks like it might be intermittent network flakiness that eventually manifests itself because there are so many files. Maybe need to add a retries
parm ...
ok that's helpful, I actually ran the whole read every file from URL, and only a handful of fails - but not sure if they were my out of date list (with preliminary ones) or the network.
the cool thing with slurm + parallel mcmapply is it automatically has a try/catch in it so you get most of the results
is there any problem with bowerbird just giving up on a file rather than failing ?
it's still going with wait = 10, seems ok!!
I'm having this bomb out:
I tried it again, to see if it was similar but it bombed earlier:
trying again with 'wait = 10'