Open Peter9192 opened 1 year ago
cheers @Peter9192 :beer: Was your internet connection working when you restarted it? ie was the script trying to "download" local files in an equivalent mode to offline=True if the internet was not working? Also - it could be that when the internet stopped working a file that was half-downloaded still appeared as a valid local file, whereas it's just a shell and it's corrupted
LocalFile is part of the public API and can be imported from esmvalcore.local.LocalFile
, so that should help with filtering. However, I do agree that it would be nicer if they were automatically skipped.
esmvalcore.esgf.LocalFile
is actually a subclass of pathlib.Path
, so you could also filter on that. Or filter on esmvalcore.esgf.ESGFFile
, as that is the expected input data type for esmvalcore.esgf.download
.
First attempt at this worked but halfway through the download my internet connection was interrupted. Executing again resulted in
I can workaround it by excluding the localfiles, but that requires importing the type from a hidden module
esmvalcore.esgf._download
. It would be nicer if ESMValCore could skip these automatically.