With "native" provider, a Python error occured : pyqgiswps.exceptions.ProcessException: Failed to initialize Qgis processes

[cBevilaqua]

Maybe this is not an issue but I'm writing here just to get your help. When I start the server, I get this warning:

WARNING No published processes !

And then in lizmap, the wps process select is always empty. I also the empty when calling /ows/?request=GetCapabilities&service=WPS. Do I need to enable something in the Qgis server? (I'm using the official Qgis server and not the py-qgis-server, should I used it instead)

Below are the logs when I start py-qgis-wps:

2023-07-03 14:02:02,262 [963]   INFO    Proxy configuration enabled
2023-07-03 14:02:02,262 [963]   INFO    WPS Server ready
2023-07-03 14:02:02,263 [963]   INFO    Running WPS server on port 0.0.0.0:8081
2023-07-03 14:02:02,264 [963]   WARNING No WPS plugin found in /usr/share/qgis_wps/processing
2023-07-03 14:02:02,266 [963]   WARNING No WPS plugin found in .
2023-07-03 14:02:02,641 [966]   INFO    Starting Qgis application: 3.30.3-'s-Hertogenbosch
2023-07-03 14:02:02,642 [966]   INFO    Setting offscreen mode
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2023-07-03 14:02:03,511 [966]   INFO    [qgis:966] Installing cleanup hook
2023-07-03 14:02:03,536 [966]   INFO    [qgis:966] Qgis application initialized......
2023-07-03 14:02:05,565 [966]   ERROR   [qgis:966] Processing: Problem with OTB installation: OTB folder is not set.
2023-07-03 14:02:05,582 [966]   INFO    [qgis:966] QGis processing initialized
2023-07-03 14:02:05,677 [966]   WARNING No published processes !
2023-07-03 14:02:05,682 [972]   INFO    Starting worker pool
2023-07-03 14:02:05,717 [963]   INFO    Starting supervisor
2023-07-03 14:02:05,718 [963]   INFO    WPS Server ready
2023-07-03 14:02:06,120 [973]   INFO    Starting Qgis application: 3.30.3-'s-Hertogenbosch
2023-07-03 14:02:06,121 [973]   INFO    Setting offscreen mode
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2023-07-03 14:02:06,856 [973]   INFO    [qgis:973] Installing cleanup hook
2023-07-03 14:02:06,857 [973]   INFO    [qgis:973] Qgis application initialized......
2023-07-03 14:02:07,286 [973]   ERROR   [qgis:973] Processing: Problem with OTB installation: OTB folder is not set.
2023-07-03 14:02:07,291 [973]   INFO    [qgis:973] QGis processing initialized

I didn't published any scripts by my self. Shouldn't the default qgis geoprocessing functions like "intersects", "join", etc, appear by default?

Thanks.

[Gustry]

I'm very not familiar with this part of the project, but for me the warning looks normal. I think, for security purpose, it's look normal for me than nothing is published by default from the server. I didn't check.

[cBevilaqua]

Thanks, anyway, for the default qgis processes to appear (link in the attached image), I need to do some extra configuration?

[cBevilaqua]

I was able to make the default qgis algorithms appear by settings the environment variable export QGSWPS_PROCESSING_EXPOSED_PROVIDERS="qgis,gdal", but some algorithms like "Intersection" or "Union" are now showing, do you have some ideia?

[Gustry]

I guess these are in the native provider.

[cBevilaqua]

Thanks! I tried to put native in QGSWPS_PROCESSING_EXPOSED_PROVIDERS, but now I got the following error:

2023-07-04 12:07:16,661 [4440]  ERROR   native:categorizeusingstyle: unsupported param <class 'qgis._core.QgsProcessingParameterExpression'>:'expression'
2023-07-04 12:07:16,674 [4440]  ERROR   Traceback (most recent call last):
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 79, in task
    processes = factory._create_qgis_processes()
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 228, in _create_qgis_processes
    processes.extend(QgsProcess(alg) for alg in provider.algorithms() if not _is_hidden(alg))
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 228, in <genexpr>
    processes.extend(QgsProcess(alg) for alg in provider.algorithms() if not _is_hidden(alg))
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processingprocess.py", line 309, in __init__
    inputs = list(parse_input_definitions(alg, context=context))
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processingio.py", line 217, in parse_input_definitions
    yield parse_input_definition(param, alg, context=context)
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processingio.py", line 199, in parse_input_definition
    or layersio.parse_input_definition(param, kwargs, context) \
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/io/layersio.py", line 272, in parse_input_definition
    datatypes = get_layers_type(param, kwargs)
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/io/layersio.py", line 137, in get_layers_type
    kwargs['metadata'].append(Metadata('processing:dataTypes', ','.join(SourceTypes[dtyp] for dtyp in datatypes)))
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/io/layersio.py", line 137, in <genexpr>
    kwargs['metadata'].append(Metadata('processing:dataTypes', ','.join(SourceTypes[dtyp] for dtyp in datatypes)))
KeyError: 10

2023-07-04 12:07:16,678 [4438]  INFO    Terminating factory delegate
Traceback (most recent call last):
  File "/usr/local/bin/wpsserver", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/wpsserver.py", line 103, in main
    run_server(port=args.port, address=args.interfaces, user=args.setuid)
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/runtime.py", line 260, in run_server
    asyncio.run(_main())
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 646, in run_until_complete
    return future.result()
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/runtime.py", line 234, in _main
    processes = pr_factory.initialize(True)
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 158, in initialize
    processes = self.create_qgis_processes()
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 255, in create_qgis_processes
    processes = self._delegate.create_qgis_processes()
  File "/usr/local/lib/python3.10/dist-packages/pyqgiswps/executors/processfactory.py", line 66, in create_qgis_processes
    raise ProcessException("Failed to initialize Qgis processes")
pyqgiswps.exceptions.ProcessException: Failed to initialize Qgis processes

[Gustry]

Ok, this is a valid bug.

But are you sure you want to publish all algorithms on the public side ?

Even if all algorithms are not publicly published, there are still available in the backend, to let you create your own QGIS processing model.

At least, with a model, you can control more labels which will be displayed for inputs, default values etc.

[cBevilaqua]

Thanks! My plan is not to publish all the algorithms, but I was testing with all in a first moment, thanks.

Is there a way to expose only a specific algorithm like intersection through the QGSWPS_PROCESSING_EXPOSED_PROVIDERS or do I need to do it by python code? If you have some example that you can share I'll be thankful.

[Gustry]

I was also expecting an "exposed list" of algorithms like :

native:intersection
qgis:joinbylocation
...

If you have just a few, I think the best is to create your own model, which wrap your single algorithm. So at least you can create your own labels for inputs/outputs etc.

[dmarteau]

Please, see https://github.com/3liz/py-qgis-wps/issues/36 for a discussion about exposing native provider,

[Gustry]

Thanks @dmarteau so exactly the same conclusion, better to make a small model wrapping your algorithm.

I will try to add it in the doc.