Upload GeoTiff Error You can't execute queries until the end of the 'atomic' block

jeff-bradley commented 1 year ago

Expected Behavior

Upload Geotiff file successfully

Actual Behavior

Error of "An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.. Request: None"

Steps to Reproduce the Problem

Download and unzip wildlife areas https://sitewindright.s3.amazonaws.com/KeyWildlifeAreas_Solar_GT_20210907.zip
Upload Geotiff to geonode
Upload Fails

Specifications

GeoNode version: 4.1
Installation type (vanilla, geonode-project): Vanilla
Installation method (manual, docker): docker
Platform: Windows WSL Ubuntu 22

Additional details: ExecutionRequest


{
        "exec_id": "5e0c56a8-98c7-40ab-b9f6-915426202776",
        "status": "failed",
        "func_name": "rollback",
        "created": "2023-09-01T14:54:11.136364Z",
        "finished": "2023-09-01T14:54:15.553520Z",
        "last_updated": "2023-09-01T14:54:15.553538Z",
        "input_params": {
            "files": {
                "tif_file": "/mnt/volumes/statics/uploaded/tmp8s7iro8f/KeyWildlifeAreas_Solar_GT_20210907.tif",
                "base_file": "/mnt/volumes/statics/uploaded/tmp8s7iro8f/KeyWildlifeAreas_Solar_GT_20210907.tif"
            },
            "total_layers": 1,
            "store_spatial_file": true,
            "handler_module_path": "importer.handlers.geotiff.handler.GeoTiffFileHandler",
            "skip_existing_layers": false,
            "overwrite_existing_layer": false
        },
        "output_params": {
            "errors": [
                "Task: importer.create_geonode_resource raised an error during actions for layer: import: An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.. Request: None"
            ],
            "failed_layers": [
                "import"
            ]
        },
        "step": "importer.rollback",
        "log": "An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block.. Request: None",
        "name": "KeyWildlifeAreas_Solar_GT_20210907.tif",
        "action": "import",
        "source": "upload",
        "user": 1000,
        "geonode_resource": null,
        "link": "http://localhost/api/v2/executionrequest/5e0c56a8-98c7-40ab-b9f6-915426202776"
    },

mattiagiupponi commented 1 year ago

Hi @jeff-bradley Did you try with a different GeoTiff or always happen? Just to understand if is something related to the GeoTiff or to the import flow

jeff-bradley commented 1 year ago

Hello @mattiagiupponi I was able to successfully load a very small test geotiff, but I received the error on both the wind and solar from these downloads https://sitewindright.s3.amazonaws.com/KeyWildlifeAreas_Wind_GT_20211113.zip https://sitewindright.s3.amazonaws.com/KeyWildlifeAreas_Solar_GT_20210907.zip

mattiagiupponi commented 1 year ago

Ok is not happening on all GeoTiff (which is a good point) I try in local and looks like that the importer is extracting as CRS EPSG:5070 which is not one of the default accepted format. @giohappy do you have any hints on this? how should we proceed?

jeff-bradley commented 1 year ago

Thanks @mattiagiupponi I was able to load the GeoTiff after I transformed it to EPSG:3857

mattiagiupponi commented 1 year ago

Thanks @mattiagiupponi I was able to load the GeoTiff after I transformed it to EPSG:3857

Great news, I'll keep this open waiting an input from @giohappy. Then we can close the issue

giohappy commented 1 year ago

I've reopened and tranfered the issue inside the importer repo.

@mattiagiupponi in case of unsupported CRS a more informative error should be returned by the importer. "An error occurred in the current transaction. You can't execute queries until the end of the 'atomic' block. Request: None" is very cryptic, to say the least :)

ridoo commented 1 year ago

May I ask if adding additional CRSs in Geoserver would also solve this problem? As described here: https://docs.geoserver.org/2.23.x/en/user/configuration/crshandling/customcrs.html

But I agree, that a better error message is key here ;)

giohappy commented 1 year ago

@jeff-bradley after some troubleshooting we found that the problem is with the CRS assigned to the Geotiff. It's not EPSG:5070 but ESRI:102039. Geoserver doesn't parese this CRS correctly (to be investigated why) but it creates the layer: Everything is fine if we reproject the original tiff to EPSG:5070.

We are going to open an issue for the importer to trigger a rollback if Geoserver fails at returning a valid CRS before pubslighing the dataset in GeoNode.

mattiagiupponi commented 1 year ago

I just merge a PR which will raise a clear error if the Srid is not recognied correctly. At least should be more clear the issue instead of the "atomic block" exception

mattiagiupponi commented 1 year ago

This was resolved via https://github.com/GeoNode/geonode-importer/issues/198

GeoNode / geonode-importer