USACE / cwms-data-api

Corps Water Management System RESTful Data Service
MIT License
9 stars 13 forks source link

CWMS-DATA/RATINGS defaults to JSON - Gives ORA-31011 XML Parse Failed #699

Open krowvin opened 2 weeks ago

krowvin commented 2 weeks ago

Change cwms-data/ratings
to default to XML for now. I know the pending fixes for the data types might fix this but thought I would mention it.

If you do this query,

curl -X 'GET' \
  'https://cwms-data.usace.army.mil/cwms-data/ratings?name=Chester-Mississippi.Stage%3BFlow.USGS.Production&office=MVS' \
  -H 'accept: application/json'

You get this

{
  "ratings": {
    "query-info": {
      "time-of-query": "2024-06-14T13:51:40Z",
      "process-query": "PT7.774S",
      "format-output": "PT0S",
      "requested-format": "JSON",
      "requested-office": "MVS",
      "requested-start-time": "2024-06-13T13:51:40Z",
      "requested-end-time": "2024-06-14T13:51:40Z",
      "requested-items": [
        {
          "name": "Chester-Mississippi.Stage;Flow.USGS.Production",
          "unit": "NATIVE",
          "datum": "NATIVE"
        }
      ],
      "templates-retrieved": 1,
      "total-specifications-retrieved": 1,
      "unique-specifications-retrieved": 1,
      "total-ratings-retrieved": 1,
      "unique-ratings-retrieved": 1
    },
    "error": "[Chester-Mississippi.Stage;Flow.USGS.Production] ORA-31011: XML parsing failed"
  }
}

Knowing the DB returns XML directly to CDA it might be worth returning the types completely from the swagger docs at least.

Setting the format to xml returns the expected results

MikeNeilson commented 2 weeks ago

@RyanM-RMA @rma-rripken Charles has a point here. For this particular end point XML should be the default as it is the native format of our ratings and thus the least risk of error by default for retrieval.

rma-rripken commented 2 weeks ago

There is already an issue for this. https://github.com/USACE/cwms-data-api/issues/639

That url goes to RatingController.getAll. Looking at that method - it only pays attention to the format arg - the Controller does nothing with the accept header.

We can easily make it default to xml but...have you looked at the xml ? Its built by pl/sql and the element is a wall of text. I think the reason we see those "ORA-31011: XML parsing failed" errors are because the output format is not "standard".

For a different Rating related thing I got exports of a bunch of different ratings to test against. Not a single one of those files has this style of element. All the files I have (that do parse) have a series of \<point> under \<rating-points>.

This output has: \<rating-points>-5.44 38000 -5.43 38045.23 -5.42 38090.48 -5.41 38135.78 -5.4 38181.1 -5.39 38226.46 -5.38 38271.84 -5.37 38317.27 -5.36 38362.72

RyanM-RMA commented 2 weeks ago

@MikeNeilson @rma-rripken, Peter had me put these tickets on hold for a bit, but I was working on #527 just beforehand. I have it updated to read the accept header as well as the query param and default goes to JSON. I can easily update that to XML once I'm back on it.

krowvin commented 2 days ago

There is already an issue for this.

In my defense - I submitted the first as a result of testing from CWMSjs

This ticket was made because a user was actively trying to use this and ran into it

Duplicate issue none the less. Wondering if I should make a duplicate tag