Open soxofaan opened 12 months ago
the solution/workaround here is to not specify an overlap resolver
Another use case and symptom : using same number of bands, but different bands:
bands1 = ["B02"]
cube1 = connection.load_collection(
...
bands=bands1,
)
bands2 = ["B03"]
cube2 = connection.load_collection(
...
bands=bands2
)
This fails with user facing error
YARN application status reports error diagnostics: User application exited with status 139
And I have to manually look into the YARN logs to find:
malloc_consolidate(): invalid chunk size
Fatal Python error: Aborted
...
Current thread 0x00007f71292ca100 (most recent call first):
File "/usr/lib64/python3.8/site-packages/osgeo/gdal.py", line 5896 in InfoInternal
File "/usr/lib64/python3.8/site-packages/osgeo/gdal.py", line 413 in Info
File "batch_job.py", line 647 in read_gdal_info
File "batch_job.py", line 831 in _process_gdalinfo_for_netcdf_subdatasets
File "batch_job.py", line 680 in parse_gdal_raster_metadata
File "batch_job.py", line 617 in read_gdal_raster_metadata
File "batch_job.py", line 519 in _extract_asset_raster_metadata
File "batch_job.py", line 387 in _extract_asset_metadata
File "batch_job.py", line 260 in _assemble_result_metadata
File "batch_job.py", line 1208 in run_job
File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/utils.py", line 52 in memory_logging_wrapper
File "batch_job.py", line 999 in run_driver
File "batch_job.py", line 1028 in main
File "batch_job.py", line 1291 in <module>
indeed, dropping the overlap resolver from cube1.merge_cubes(cube2)
makes both use cases above work.
there is however no workaround yet for use case with actual overlap:
bands1 = ["B02", "B03"]
cube1 = connection.load_collection(
...
bands=bands1,
)
bands2 = ["B03", "B04"]
cube2 = connection.load_collection(
...
bands=bands2
)
Without an overlap resolver specified, it fails complaining there should be one. With an overlap resolver, it fails without a usable user facing error message. In YARN logs I find:
corrupted size vs. prev_size
Fatal Python error: Aborted
...
Current thread 0x00007f3ab5d29100 (most recent call first):
File "/usr/lib64/python3.8/site-packages/osgeo/gdal.py", line 5896 in InfoInternal
File "/usr/lib64/python3.8/site-packages/osgeo/gdal.py", line 413 in Info
File "batch_job.py", line 647 in read_gdal_info
File "batch_job.py", line 831 in _process_gdalinfo_for_netcdf_subdatasets
File "batch_job.py", line 680 in parse_gdal_raster_metadata
File "batch_job.py", line 617 in read_gdal_raster_metadata
File "batch_job.py", line 519 in _extract_asset_raster_metadata
File "batch_job.py", line 387 in _extract_asset_metadata
File "batch_job.py", line 260 in _assemble_result_metadata
File "batch_job.py", line 1208 in run_job
File "/opt/venv/lib64/python3.8/site-packages/openeogeotrellis/utils.py", line 52 in memory_logging_wrapper
File "batch_job.py", line 999 in run_driver
that last error is related to metadata assembly. So either it is not the real error, or the workflow does succeed but it's simply the new metadata generation that fails.
Related to https://github.com/Open-EO/openeo-geopyspark-driver/issues/507 (and https://github.com/Open-EO/openeo-aggregator/issues/115):
merge_cubes
of two cubes with different bands is not handled correctly:process graph:
running this as a job fails with:
What should happen is creating a cube with three bands ["B02", "B03", "B04"], not trying to combine ["B02"] values with ["B03", "B04"] values