SGCI / sgci-resource-inventory

This contains all the computational resource entities
https://sgci-resource-inventory.readthedocs.io/en/latest/introduction.html
Apache License 2.0
6 stars 2 forks source link

Proposed GitHub and XSEDE API synchronization design #28

Open jpnavarro opened 2 years ago

jpnavarro commented 2 years ago

XSEDE generates SGCI resource descriptions from RDR and IPF information that can be accessed at: https://info.xsede.org/wh1/warehouse-views/v1/resources-sgci/v1.0.0/

The SGCI resource description GitHub repository is at: https://github.com/SGCI/sgci-resource-inventory/tree/master/data

In both, the "host" attribute is the unique key.

Proposed design to synchronize from XSEDE to GitHub: daily compare API and GitHub descriptions, and generate a GitHub pull request for description where the API description doesn't exist in GitHub, where it is different than GitHub without considering whitespace, or has disappeared in the XSEDE API (deleted resource).

Proposed design for an enhanced XSEDE API that combines GitHub only descriptions and XSEDE generated descriptions:

https://info.xsede.org/wh1/warehouse-views/v1/resources-sgci/v1.0.0/?include=github Dynamically generates SGCI resource descriptions from RDR and IPF plus any GitHub resources that don't come from XSEDE. The GitHub cache used by this API is refreshed hourly.

Note that XSEDE's descriptions can take up to a day to make it to GitHub depending on how frequently pull requests are processed, and that GitHub descriptions can take up to an hour to show up in the enhanced XSEDE API.

If the team can agree on this design, XSEDE can implement it.