o2r-project / geoextent

Python library for extrating geospatial extent of files and directories with multiple data formats
https://o2r.info/geoextent/
MIT License
1 stars 4 forks source link

Folder / zip files extraction bug #93

Closed SbastianGarzon closed 3 years ago

SbastianGarzon commented 3 years ago

Folder/ zip files extraction fails if a file of a supported format is invalid. Geoextent should ignore the erroneous file. Instead of an error, geoextent should display a warning (?). Final bbox should merge all of the bboxes of supported (and valid) files even if an erroneous file is in the folder.

Version : geoextent 0.4.0

geoextent -b tests/testdata/geojson/

Traceback (most recent call last): File geoextent/geoextent/__main__.py", line 153, in main output = extent.fromDirectory(files, bbox = args['bounding_box'], tbox = args['time_box']) File "geoextent/geoextent/lib/extent.py", line 74, in fromDirectory metadata_file = fromFile(os.path.join(path, filename), bbox, tbox) File "garzon/geoextent/geoextent/lib/extent.py", line 139, in fromFile raise Exception(os.getcwd() + " The file {} is not valid (e.g. empty):\n{}".format(filePath, str(e))) Exception: garzon/geoextent The file tests/testdata/geojson/empty.geojson is not valid (e.g. empty): GeoJSON file tests/testdata/geojson/empty.geojson is not valid: Expecting value: line 1 column 1 (char 0)

nuest commented 3 years ago

Yes, there should be a warning if the extent of a directory/ZIP archive is requested. If there are only files with warnings, then the bbox should be empty / NA.

What do you think about an option -w / --warnings-are-errors to throw an error in such a case?

What do you think about adjusting the behaviour for individual files, too?

nuest commented 3 years ago

Alternative and maybe more specific name: --fail-on-invalid, because warnings may also be cause by other things than invalid inputs.

We could also have --fail-on-missing to stop with an exception if one of the files has not bounding box.

nuest commented 3 years ago

--fail-on-unsupported might be a better name to distinguish the case that a file has no bounding box but could have.

nuest commented 3 years ago

If you can very quickly realise #40 and #49 when you look at this, go for it.

nuest commented 3 years ago

@SbastianGarzon Before you start, please document here which of the suggestions or what other better approach you'd like to implement here. I'm leaning towards my latest suggestion (--fail-on-unsupported) but might be missing something.

nuest commented 3 years ago

This bug is fixed, and there is a warning if one of many files in a folder/ZIP archive cannot be extracted.

For our current use cases, we don't need the option to fail in such a case, because other files could have been successfully extracted.