Feature/tsunami wrapper

[arnevogt]

add tsunami wrapper

[MichaelLangbein]

I'm getting an error when running a job on that container.

2023-04-25 10:55:08,596 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 10:55:09,719 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 8)
2023-04-25 10:55:09,720 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 10:55:10,833 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 9)
2023-04-25 10:55:10,834 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 10:55:12,012 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 10)
2023-04-25 10:55:12,013 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 10:55:12,167 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Something went wrong when storing the links on the file storage or running the wps itself.
2023-04-25 10:55:12,168 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Could not decode Inputstream.
org.n52.geoprocessing.wps.client.WPSClientException: Could not decode Inputstream.
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:620)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveResponseOrExceptionReportInpustream(WPSClientSession.java:559)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveCapsViaGET(WPSClientSession.java:536)
        at org.n52.geoprocessing.wps.client.WPSClientSession.connect(WPSClientSession.java:170)
        at org.n.riesgos.asyncwrapper.process.wps.WPSClientService.establishWPSConnection(WPSClientService.kt:18)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:349)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.utils.RetryUtilsKt.retry(RetryUtils.kt:22)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.runOneJob(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.fillConstraintsAndRun(AbstractWrapper.kt:275)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.run(AbstractWrapper.kt:65)
        at org.n.riesgos.asyncwrapper.events.OrderMessageHandler.handleMessage(OrderMessageHandler.kt:23)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.receiveMessages(PulsarConsumer.kt:22)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.run(PulsarConsumer.kt:13)
        at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[17,33]
Message: unexpected tag: {http://www.opengis.net/ows/2.0}individualname
        at org.n52.iceland.util.XmlFactories.unexpectedTag(XmlFactories.java:142)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readServiceContact(GetCapabilitiesResponseDecoder.java:241)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readServiceProvider(GetCapabilitiesResponseDecoder.java:206)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readCapabilities(GetCapabilitiesResponseDecoder.java:71)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readCapabilitiesResponse20(WPSResponseReader.java:105)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readElement(WPSResponseReader.java:54)
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:618)
        ... 14 common frames omitted
2023-04-25 10:55:12,168 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Job is skipped
acknowledge message

The input was lat: -33.1, lon: -71.6, mag: 8.0 That combination is not in the list of pre-approved parameters, I think, but I think the wrapper did not notify the db of a failure, because my frontend keeps showing the state running

[nbrinckm]

@MichaelLangbein what you have here looks like an error when parsing the capabilities of the wps server. In general I was in the opinion that this was something that we somehow solved with the retry decorator.

It depends a little bit if this is something that happens all the time - then it may be an issue with the answer of the pywps (or this wps dialect thing that we had). If it doesn't happen all the time, then the retry decorator should catch that normally (as long as it is configured properly).

The wrapper itself looks good for me.

[MichaelLangbein]

Ah, indeed, this happens repeatedly.

I tried again, this time with one of the valid sets of inputs (-71.6564, -32.7423, 8.0).

Result:

023-04-25 11:01:38,467 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Got the process id 1
2023-04-25 11:01:38,471 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Created one job
2023-04-25 11:01:38,474 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for mag: 8.0
2023-04-25 11:01:38,476 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for lon: -71.6564
2023-04-25 11:01:38,477 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for lat: -32.7423
2023-04-25 11:01:38,478 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added job to order
2023-04-25 11:01:38,479 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Start mapping to wps inputs
2023-04-25 11:01:38,479 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 0)
2023-04-25 11:01:38,481 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:39,613 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 1)
2023-04-25 11:01:39,614 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:40,734 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 2)
2023-04-25 11:01:40,735 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:41,855 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 3)
2023-04-25 11:01:41,856 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:42,973 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 4)
2023-04-25 11:01:42,974 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:44,094 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 5)
2023-04-25 11:01:44,095 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:45,222 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 6)
2023-04-25 11:01:45,223 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:46,342 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 7)
2023-04-25 11:01:46,342 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:47,455 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 8)
2023-04-25 11:01:47,456 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:48,579 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 9)
2023-04-25 11:01:48,580 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:49,700 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 10)
2023-04-25 11:01:49,701 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-25 11:01:49,824 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Something went wrong when storing the links on the file storage or running the wps itself.
2023-04-25 11:01:49,824 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Could not decode Inputstream.
org.n52.geoprocessing.wps.client.WPSClientException: Could not decode Inputstream.
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:620)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveResponseOrExceptionReportInpustream(WPSClientSession.java:559)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveCapsViaGET(WPSClientSession.java:536)
        at org.n52.geoprocessing.wps.client.WPSClientSession.connect(WPSClientSession.java:170)
        at org.n.riesgos.asyncwrapper.process.wps.WPSClientService.establishWPSConnection(WPSClientService.kt:18)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:349)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.utils.RetryUtilsKt.retry(RetryUtils.kt:22)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.runOneJob(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.fillConstraintsAndRun(AbstractWrapper.kt:275)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.run(AbstractWrapper.kt:65)
        at org.n.riesgos.asyncwrapper.events.OrderMessageHandler.handleMessage(OrderMessageHandler.kt:23)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.receiveMessages(PulsarConsumer.kt:22)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.run(PulsarConsumer.kt:13)
        at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[17,33]
Message: unexpected tag: {http://www.opengis.net/ows/2.0}individualname
        at org.n52.iceland.util.XmlFactories.unexpectedTag(XmlFactories.java:142)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readServiceContact(GetCapabilitiesResponseDecoder.java:241)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readServiceProvider(GetCapabilitiesResponseDecoder.java:206)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilitiesResponseDecoder.readCapabilities(GetCapabilitiesResponseDecoder.java:71)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readCapabilitiesResponse20(WPSResponseReader.java:105)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readElement(WPSResponseReader.java:54)
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:618)
        ... 14 common frames omitted
2023-04-25 11:01:49,824 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Job is skipped
acknowledge message

[MichaelLangbein]

It prints out 10 retries. So the error does persist through multiple attempts to parse the capabilities-response. That would be an issue with the WPSClientService, right?

[arnevogt]

Even when explicitly request WPS version 2.0.0, the response seems to be version 1.0.0

https://riesgos.52north.org/wps?request=GetCapabilities&service=WPS&version=2.0.0

Might this be the issue?

[MichaelLangbein]

Even when explicitly request WPS version 2.0.0, the response seems to be version 1.0.0 https://riesgos.52north.org/wps?request=GetCapabilities&service=WPS&version=2.0.0 Might this be the issue?

@bpross-52n : the wps serving the ts-simulations is a pywps, right? I think pywps only supports wps version 1.0.0 (https://pywps.readthedocs.io/en/latest/ : support for 2.0 is presently being planned)

[bpross-52n]

When requesting capabilities, there is needs to be a different query parameter: https://riesgos.52north.org/wps?request=GetCapabilities&service=WPS&acceptVersions=2.0.0 The service supports both versions.

[arnevogt]

Are xml tags actually case sensitive?

Because this seems to be about the IndividualName tag. In the version 1.0.0 capabilities document (and also for example for the GFZ WPS) it is in camel case (IndividualName). In the version 2.0.0 capabilities document it is lower case (individualname)

compare: https://riesgos.52north.org/wps?request=GetCapabilities&service=WPS&acceptVersions=1.0.0 https://riesgos.52north.org/wps?request=GetCapabilities&service=WPS&acceptVersions=2.0.0

(Same for the PositionName tag)

[nbrinckm]

@arnevogt I think so that they are case sensitive.

@all Maybe I'm a ltitle bit outdated here, but I have in mind that WPS 2.0.0 support in pyWPS is somehow limited. (At least it was the case when Matthias Rüster refactored the tridec cloud). So maybe it makes sense to query & work with it with the WPS 1.0.0.

It may take some additional effort, but I think it is a (at least half-way) clean way to proceed. (And I hope due to the wps client it should not be super complicated).

[MichaelLangbein]

Hmm, getting a new error now ... so, progress :)

2023-04-26 08:33:31,266 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ClientCnx: [id: 0x54f424a0, L:/172.25.0.7:56582 - R:queue/172.25.0.2:6650] Connected through proxy to target broker at localhost:6650
2023-04-26 08:33:31,267 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl: [new-order][tsunami-asyncwrapper_new-order_subscription] Subscribing to topic on cnx [id: 0x54f424a0, L:/172.25.0.7:56582 - R:queue/172.25.0.2:6650], consumerId 0
2023-04-26 08:33:31,283 [pulsar-client-io-1-1] INFO  org.apache.pulsar.client.impl.ConsumerImpl: [new-order][tsunami-asyncwrapper_new-order_subscription] Subscribed to topic on queue/172.25.0.2:6650 -- consumer: 0
2023-04-26 08:34:37,026 [pulsar-client-io-1-1] WARN  com.scurrilous.circe.checksum.Crc32cIntChecksum: Failed to load Circe JNI library. Falling back to Java based CRC32c provider
order message handler receiver: {"orderId":1}
2023-04-26 08:34:37,041 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Got new order to handle with wrapper 1
2023-04-26 08:34:37,059 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input constraint for tsunami mag: 8.0
2023-04-26 08:34:37,059 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input constraint for tsunami lon: -71.6
2023-04-26 08:34:37,059 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input constraint for tsunami lat: -33.1
2023-04-26 08:34:37,060 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Parsed constraints
2023-04-26 08:34:37,060 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Define the literal constraints for the jobs
2023-04-26 08:34:37,060 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Define the complex constraints for the jobs
2023-04-26 08:34:37,060 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Define the bbox constraints for the jobs
2023-04-26 08:34:37,060 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Process job
2023-04-26 08:34:37,064 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Lookup the process id
2023-04-26 08:34:37,069 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Got the process id 4
2023-04-26 08:34:37,073 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Created one job
2023-04-26 08:34:37,074 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for mag: 8.0
2023-04-26 08:34:37,075 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for lon: -71.6
2023-04-26 08:34:37,076 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added literal input for lat: -33.1
2023-04-26 08:34:37,077 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Added job to order
2023-04-26 08:34:37,078 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: Start mapping to wps inputs
2023-04-26 08:34:37,080 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 0)
2023-04-26 08:34:37,282 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:38,916 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 1)
2023-04-26 08:34:38,917 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:40,357 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 2)
2023-04-26 08:34:40,358 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:41,843 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 3)
2023-04-26 08:34:41,844 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:43,443 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 4)
2023-04-26 08:34:43,444 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:44,904 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 5)
2023-04-26 08:34:44,906 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:46,383 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 6)
2023-04-26 08:34:46,384 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:47,727 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 7)
2023-04-26 08:34:47,728 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:49,095 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 8)
2023-04-26 08:34:49,096 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:50,454 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 9)
2023-04-26 08:34:50,455 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:51,847 [tsunami-asyncwrapper_new-order_subscription] INFO  AbstractWrapper: retrieve getCapabilities document from  https://riesgos.52north.org/wps (retries: 10)
2023-04-26 08:34:51,848 [tsunami-asyncwrapper_new-order_subscription] INFO  org.n52.geoprocessing.wps.client.WPSClientSession: CONNECT
2023-04-26 08:34:52,190 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Something went wrong when storing the links on the file storage or running the wps itself.
2023-04-26 08:34:52,191 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Could not decode Inputstream.
org.n52.geoprocessing.wps.client.WPSClientException: Could not decode Inputstream.
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:620)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveResponseOrExceptionReportInpustream(WPSClientSession.java:559)
        at org.n52.geoprocessing.wps.client.WPSClientSession.retrieveCapsViaGET(WPSClientSession.java:536)
        at org.n52.geoprocessing.wps.client.WPSClientSession.connect(WPSClientSession.java:170)
        at org.n.riesgos.asyncwrapper.process.wps.WPSClientService.establishWPSConnection(WPSClientService.kt:18)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:349)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper$runOneJob$wpsProcess$2.invoke(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.utils.RetryUtilsKt.retry(RetryUtils.kt:22)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.runOneJob(AbstractWrapper.kt:346)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.fillConstraintsAndRun(AbstractWrapper.kt:275)
        at org.n.riesgos.asyncwrapper.dummy.AbstractWrapper.run(AbstractWrapper.kt:65)
        at org.n.riesgos.asyncwrapper.events.OrderMessageHandler.handleMessage(OrderMessageHandler.kt:23)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.receiveMessages(PulsarConsumer.kt:22)
        at org.n.riesgos.asyncwrapper.pulsar.PulsarConsumer.run(PulsarConsumer.kt:13)
        at java.base/java.lang.Thread.run(Thread.java:831)
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[12,50]
Message: unexpected tag: {http://www.opengis.net/ows/1.1}Type
        at org.n52.iceland.util.XmlFactories.unexpectedTag(XmlFactories.java:142)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilities100ResponseDecoder.readKeywords(GetCapabilities100ResponseDecoder.java:422)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilities100ResponseDecoder.readServiceIdentification(GetCapabilities100ResponseDecoder.java:379)
        at org.n52.geoprocessing.wps.client.decoder.stream.GetCapabilities100ResponseDecoder.readCapabilities(GetCapabilities100ResponseDecoder.java:70)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readCapabilitiesResponse100(WPSResponseReader.java:95)
        at org.n52.geoprocessing.wps.client.xml.WPSResponseReader.readElement(WPSResponseReader.java:56)
        at org.n52.geoprocessing.wps.client.WPSClientSession.parseInputStreamToString(WPSClientSession.java:618)
        ... 14 common frames omitted
2023-04-26 08:34:52,191 [tsunami-asyncwrapper_new-order_subscription] ERROR AbstractWrapper: Job is skipped
acknowledge message
2023-04-26 08:35:31,261 [pulsar-timer-5-1] INFO  org.apache.pulsar.client.impl.ConsumerStatsRecorderImpl: [new-order] [tsunami-asyncwrapper_new-order_subscription] [86470] Prefetched messages: 0 --- Consume throughput received: 0.02 msgs/s --- 0.00 Mbit/s --- Ack sent rate: 0.02 ack/s --- Failed messages: 0 --- batch messages: 0 ---Failed acks: 0

[nbrinckm]

@MichaelLangbein Still seems to be an issue with reading the capabilities.

[MichaelLangbein]

@MichaelLangbein Still seems to be an issue with reading the capabilities.

Indeed, only now the error is thrown over another location in the response-xml. @arnevogt : Forgot to mention, to reproduce that error, exeute ./run.sh to build all containers including the frontend, and from the frontend just send the pre-configured default-request from the order-form. The lon/lat/mag values in that default request are valid values for the ts-service.

[arnevogt]

The issue is about this line in the capabilities documenr: <ows:Type codeSpace="ISOTC211/19115">theme</ows:Type> It is only there in version 1.0.0.

I looked into the code of the WPSClient and apparently it is not aware of the Type tag:

    private List<String> readKeywords(StartElement start,
            XMLEventReader reader) throws XMLStreamException {
        List<String> keywords = new ArrayList<String>();

        while (reader.hasNext()) {
            XMLEvent event = reader.nextEvent();
            if (event.isStartElement()) {
                StartElement elem = event.asStartElement();
                if (elem.getName().equals(OWS11Constants.Elem.QN_KEYWORD)) {
                    keywords.add(reader.getElementText());
                } else {
                    throw unexpectedTag(elem);
                }
            } else if (event.isEndElement()) {
                return keywords;
            }
        }
        throw eof();
    }

https://github.com/52North/wps-client-lib/blob/b4e3dcfb4e0b2cdf0976ece366f31e0e3a18753a/src/main/java/org/n52/geoprocessing/wps/client/decoder/stream/GetCapabilities100ResponseDecoder.java#L411

The questions is if that is an issue with the wps client lib (because it is not aware of the Type tag) or with the capabilities document (because it does not comply to the schema).

Though, the more important question is: Is there a workaround?

[MichaelLangbein]

If this were an issue with the WPSClient, that would be part of https://github.com/52North/wps-client-lib , right? So we could either change wps-client-lib, or maybe monkeypatch the pywps-server ... @bpross-52n : you've worked a lot on wps-client-lib, right? Any ideas?

[arnevogt]

@MichaelLangbein Yes, that is correct.

Actually it seems to be an issues with the client lib. https://schemas.opengis.net/ows/1.0.0/ows19115subset.xsd

I think the issue would occur as well with version 2.0.0 if the Type tag was set.

[arnevogt]

I raised the wps client lib version to 1.1.2 that fixes the issue with the Type tag.