qjhart
commented
5 days ago I have added some initial information about what I think the current state of DWR's API setup for CIMIS is, but I can't really propose a more complete solution until I have some questions answered:
- Has DWR already planned an update to the code at et.water.ca.gov?
- Has there been any changes to the planned data proccessing of station data for the CIMIS program?
- Are there existing plans to have breaking changes to the existing API?
- Have there been requests from users for updates to the API?
- Do we have metrics on what the usages are for the API? For example, the popularity of the XML output
- What are the expectations for the requests that the new Spatial CIMIS will serve?
In Version 1, the spatial CIMIS api (via et) has three main components. You can get station data, zipcode data, and point data from the system. The station and zipcode data are handled by an Oracale database, while the point queries are sent directly to the grass database. Version 2 will have an additional polygon tool as well.
One topic of discussion is how the docker containers might instead handle this processing. We need to look at how the backend might be changed for such a method.
For example, In Issue https://github.com/qjhart/dwr-spatial-cimis/issues/4 we discussed updates to the zipcode summary code. If the public API were handled by the docker, we would rethink this solution. In issue https://github.com/qjhart/dwr-spatial-cimis/issues/5 we showed how the grass database can handle dynamic polygon requests, but we didn't show how that would be connected to an API
Background
In the last review of the planned k8s implementation plan for the spatial_cimis program, I have been thinking of how the constellation of containers, (server gateway, database, grassdb) might be used to work together. If the implementation plan is to use the grassdb more directly for answering UI requests, and if the grassdb will answear all queries I think there are opportunities to improve the processing methodology.
In particular, both grass and k8s are really better supported with an open source postgres database. If we include a postgres database integrated with the existing grass database, then that would significantly modify the system could work, most specifically in how daily processing of components like the zipcode summaries would work.
In addition, containers like the grassdb could have http APIs developed that would simplify the overall architecture, and allow for better separation of the UI, the public facing server API, and the processing containers.
It might be beneficial to review the exising container architecture and discuss alternatives based on the above considerations.