Closed sebastianrosenzweig closed 3 years ago
Now the pipeline is fixed again. The problem was that requests.get()
by default apparently tries to download the whole websource. When a direct download link to a (large) is provided in the yaml, this leeds to timeouts (because it downloads the whole dataset). Adding the stream=True
keyword helps to avoid this behavior.
Thanks, @sebastianrosenzweig!
Please add the MTD and the SWD dataset to the list.