riesgos / async

0 stars 0 forks source link

Extend interface for WPSProcess call #24

Closed nbrinckm closed 1 year ago

nbrinckm commented 2 years ago

Currently the WPSProcess missed the configruability of what outputs are requested (it currently only queries for the "literalOutput" element).

We need to extend the interface of this class (or the InputMapper) to also hold information about the outputs that we want to have.

Overall we need the following outputs:

Process Output Format
Quakeledger selectedRows text/xml http://quakeml.org/xmlns/quakeml/1.2/QuakeML-1.2.xsd and application/vnd.geo+json
Shakyground shakeMapFile text/xml http://earthquake.usgs.gov/eqcenter/shakemap and application/WMS
Assetmaster selectedRowsGeoJson application/json
Modelprop selectedRows application/json
Deus merged_output application/json
Deus shapefile_summary application/WMS
Deus meta_summary application/json

More inforamtion can be found explored https://rz-vm140.gfz-potsdam.de/wps-js-client

nbrinckm commented 2 years ago

If there is also the wish to test the current state:

I inserted an order in the database:

$ docker exec -ti async_db_1 psql -U postgres $ insert into users(email) values ('abc@gfz-potsdam.de'); $ insert into orders (user_id, order_constraints) values (1, '');

Instead of I used this:

    "assetmaster": {
        "literal_inputs": {
            "lonmin": ["-88"],
            "lonmax": ["-66"],
            "latmin": ["-21"],
            "latmax": ["-0"],
            "schema": ["SARA_v1.0"],
            "assettype": ["res"],
            "querymode": ["intersects"],
            "model": ["LimaCVT1_PD30_TI70_5000", "LimaCVT4_PD40_TI60_5000"]
        }
    },
    "eq-modelprop": {
        "literal_inputs": {
            "schema": ["SARA_v1.0"],
            "assetcategory": ["buildings"],
            "losscategory": ["structural"],
            "taxonomies": [""]
        }
    },
    "shakyground": {
        "literal_inputs": {
            "gmpe": ["MontalvaEtAl2016SInter", "GhofraniAtkinson2014", "AbrahamsonEtAl2015SInter", "YoungsEtAl1997SInterNSHMP2008"],
            "vsgrid": ["USGSSlopeBasedTopographyProxy", "FromSeismogeotechnicsMicrozonation"]
        },
        "complex_inputs": {
            "quakeMLFile": [
                {
                    "input_value": "[{\"type\": \"FeatureCollection\",\"features\": [{\"type\": \"Feature\",\"geometry\": {\"type\": \"Point\",\"coordinates\": [-77.9318,-12.1989]},\"properties\": {\"publicID\": \"quakeml:quakeledger/peru_70000017\",\"preferredOriginID\": \"quakeml:quakeledger/peru_70000017\",\"preferredMagnitudeID\": \"quakeml:quakeledger/peru_70000017\",\"type\": \"earthquake\",\"description.text\": \"observed\",\"origin.publicID\": \"quakeml:quakeledger/peru_70000017\",\"origin.time.value\": \"1746-10-28T00:00:00.000000Z\",\"origin.depth.value\": \"8.0\",\"origin.creationInfo.value\": \"GFZ\",\"magnitude.publicID\": \"quakeml:quakeledger/peru_70000017\",\"magnitude.mag.value\": \"8.5\",\"magnitude.type\": \"MW\",\"magnitude.creationInfo.value\": \"GFZ\",\"focalMechanism.publicID\": \"quakeml:quakeledger/peru_70000017\",\"focalMechanism.nodalPlanes.nodalPlane1.strike.value\": \"329.0\",\"focalMechanism.nodalPlanes.nodalPlane1.dip.value\": \"20.0\",\"focalMechanism.nodalPlanes.nodalPlane1.rake.value\": \"90.0\",\"focalMechanism.nodalPlanes.preferredPlane\": \"nodalPlane1\",\"selected\": true},\"id\": \"quakeml:quakeledger/peru_70000017\"}]}]",
                    "mime_type": "application / vnd.geo + json",
                    "xmlschema": "",
                    "encoding": "UTF-8"
                }
            ]
        }
    }   
}

Please note that the order contraint value must be quoted in single quotes in postgres.

With that in place, I changed the Publisher.kt class in the tools/pulsar_pub_sub project:

import org.apache.pulsar.client.api.*
import java.io.ByteArrayOutputStream
import java.nio.charset.StandardCharsets
import java.util.stream.IntStream

fun main(args: Array<String>) {
    //create pulsar client
    val client = PulsarClient.builder()
        .serviceUrl("pulsar://localhost:6650")
        .build()

    //create producer
    val producer: Producer<ByteArray> = client.newProducer()
        .topic("new-order")
        .compressionType(CompressionType.LZ4)
        .create()

    //create and send message messages
    val content: String = "{\"orderId\": 1}"
    val msg: TypedMessageBuilder<ByteArray> = producer.newMessage();
    msg.value(content.toByteArray())
    //send message
    msg.send()
    System.out.println("sent message")
}

I can run this with mvn exec:java -Dexec.mainClass="PublisherKt" (after I updated the mainClass attribute in the exec-maven-plugin to PublisherKt - but not sure if that is still needed).

nbrinckm commented 2 years ago

I guess if we have this WPSProcess class:

class WPSProcess(private val wpsClient : WPSClientSession, private val url: String, private val processID: String, private val wpsVersion: String) : Process {

    override fun runProcess(input: ProcessInput): ProcessOutput {

        // take a look at the process description
        val processDescription = wpsClient.getProcessDescription(url, processID, wpsVersion)

        // create the request, add literal input
        val executeBuilder = ExecuteRequestBuilder(processDescription)
        processDescription.inputs.forEach {
            // ...
        }
         // |-----------------------------------------------------------------------------------------------------
         // here is the place where we need to define our own outputs that we query for.
        val parameterOut = "literalOutput"
        executeBuilder.setResponseDocument(parameterOut, null, null, "text/xml")
        // some more stuff
nbrinckm commented 2 years ago

Found a mistake in the order constraints.

"mime_type": "application / vnd.geo + json",

should be

"mime_type": "application/vnd.geo+json",
nbrinckm commented 2 years ago

@arnevogt did some of the stuff already (and this is in the main branch already). In https://github.com/riesgos/async/pull/25 I put it into the wrapper instances (as I find it easier to tackle in kotlin code for the moment). Also in this pull request, there are some extensions on the input mapper & the wps process class. I wanted it to be working for the deus call & that is what is done in the pull request.

MichaelLangbein commented 1 year ago

Closed as code has been merged in #25