search

actinia-org / actinia-module-plugin

Contains actinia module self-description and process-chain-template management and processing
Apache License 2.0
2 stars 3 forks source link

Support all processing types #47

Open ninsbl opened 1 year ago

ninsbl commented 1 year ago

Currently, it seems the module plugin only supports persistent processing. THis is the error I get when I try to execute a module template as processing_async_export:

"accept_datetime": "2023-05-15 12:59:01.894293",
"accept_timestamp": 1684155541.8942907,
"api_info": {
"endpoint": "asyncephemeralexportresource",
"method": "POST",
"path": "/api/v3/locations/ETRS_33N/processing_async_export",
"request_url": "http://my.actinia.io/api/v3/locations/ETRS_33N/processing_async_export"
},
"datetime": "2023-05-15 12:59:02.531210",
"exception": {
"message": "GrassInitError:  GRASS module timesreies_zones_univar not found in ['/usr/local/grass/bin/timesreies_zones_univar', '/usr/local/grass/scripts/timesreies_zones_univar', '/root/.grass8/addons/bin/timesreies_zones_univar', '/root/.grass8/addons/scripts/timesreies_zones_univar']",
"traceback": [
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1944, in run\n    self._execute()\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral/ephemeral_processing_with_export.py\", line 546, in _execute\n    EphemeralProcessing._execute(self)\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1703, in _execute\n    self._execute_process_list(process_list=process_list)\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1888, in _execute_process_list\n    self._run_module(process)\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1488, in _run_module\n    return self._run_executable(process, poll_time)\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1540, in _run_executable\n    proc = self.ginit.run_module(\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/core/grass_init.py\", line 572, in run_module\n    return self.runner.run_module(\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/core/grass_init.py\", line 451, in run_module\n    grass_module_path = self._create_grass_module_path(grass_module)\n",
"  File \"/usr/lib/python3.9/site-packages/actinia_core/core/grass_init.py\", line 402, in _create_grass_module_path\n    raise GrassInitError(\n"
],
"type": "<class 'actinia_core.core.grass_init.GrassInitError'>"
},
"http_code": 400,
"message": "GrassInitError:  GRASS module timesreies_zones_univar not found in ['/usr/local/grass/bin/timesreies_zones_univar', '/usr/local/grass/scripts/timesreies_zones_univar', '/root/.grass8/addons/bin/timesreies_zones_univar', '/root/.grass8/addons/scripts/timesreies_zones_univar']",

The same call works fine with just processing.

Idally, the module-plugin should support all processing types in actinia. Alternatively, limitations should be documented more prominently (I may just have overlooked)...

anikaweinmann commented 1 year ago

@ninsbl Please try the endpoint /api/v3/locations/ETRS_33N/processing_async for ephemeral processing.

metzm commented 1 year ago

Is there a simple typo somewhere? timesreies_zones_univar -> timeseries_zones_univar, see above "message"

ninsbl commented 1 year ago

Is there a simple typo somewhere? timesreies_zones_univar -> timeseries_zones_univar, see above "message"

Yes, well spotted. The typo should not affect the issue at hand, however...

ninsbl commented 1 year ago

I now tried the /api/v3/locations/ETRS_33N/processing_async endpoint, and in order to make the issue more easy to reproduce I used the example template:

{
  "id": "user_default_value",
  "description": "test default value in actinia-module-plugin",
  "template": {
    "version": "1",
    "list": [
      {
        "module": "r.mapcalc",
        "id": "r.mapcalc_test",
        "inputs": [
          {
            "param": "expression",
            "comment": "output = r.mapcalc result, string; value = raster value (default=0.428), float",
            "value": "{{ output }} = {{ value|default(0.428) }}"
          }
        ]
      },
      {
        "id": "r_info",
        "module": "r.info",
        "inputs": [
          {
            "param": "map",
            "value": "{{ output }}"
          }
        ]
      }
    ]
  }
}

And then the following JSON payload:

{
    "version": "1",
    "list": [
      {
        "module": "user_default_value",
        "id": "r.mapcalc_test",
        "inputs": [
          {
            "param": "output",
            "value": "mod_test2"
          }
        ]
      }
    ]
  }

With that /api/v3/locations/ETRS_33N/processing_async endpoint throws an error:

{
    "accept_datetime": "2023-08-31 13:03:46.721964",
    "accept_timestamp": 1693487026.7219613,
    "api_info": {
        "endpoint": "asyncephemeralresource",
        "method": "POST",
        "path": "/api/v3/locations/ETRS_33N/processing_async",
        "request_url": "http://my-actinia/api/v3/locations/ETRS_33N/processing_async"
    },
    "datetime": "2023-08-31 13:03:47.233112",
    "exception": {
        "message": "GrassInitError:  GRASS module user_default_value not found in ['/usr/local/grass/bin/user_default_value', '/usr/local/grass/scripts/user_default_value', '/root/.grass8/addons/bin/user_default_value', '/root/.grass8/addons/scripts/user_default_value']",
        "traceback": [
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1943, in run\n    self._execute()\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1702, in _execute\n    self._execute_process_list(process_list=process_list)\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1887, in _execute_process_list\n    self._run_module(process)\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1487, in _run_module\n    return self._run_executable(process, poll_time)\n           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/processing/actinia_processing/ephemeral_processing.py\", line 1539, in _run_executable\n    proc = self.ginit.run_module(\n           ^^^^^^^^^^^^^^^^^^^^^^\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/core/grass_init.py\", line 566, in run_module\n    return self.runner.run_module(\n           ^^^^^^^^^^^^^^^^^^^^^^^\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/core/grass_init.py\", line 445, in run_module\n    grass_module_path = self._create_grass_module_path(grass_module)\n                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n",
            "  File \"/usr/lib/python3.11/site-packages/actinia_core/core/grass_init.py\", line 396, in _create_grass_module_path\n    raise GrassInitError(\n"
        ],
        "type": "<class 'actinia_core.core.grass_init.GrassInitError'>"
    },
    "http_code": 400,
    "message": "GrassInitError:  GRASS module user_default_value not found in ['/usr/local/grass/bin/user_default_value', '/usr/local/grass/scripts/user_default_value', '/root/.grass8/addons/bin/user_default_value', '/root/.grass8/addons/scripts/user_default_value']",

While the /api/v3/locations/ETRS_33N/mapsets/tmp_mapset/processing endpoint succeeds...