search

ExposuresProvider / cam-pipeline

Data loading pipeline for CAM database
https://exposuresprovider.github.io/cam-pipeline/
MIT License
2 stars 4 forks source link

Deploy Automat-CAM-KP to RENCI and ITRB #90

Closed gaurav closed 1 month ago

gaurav commented 1 year ago
balhoff commented 1 year ago

@gaurav, you have some existing CAM-KP API URLs up there (for dev and renci-prod). Were you thinking that Automat version would end up at those same URLs, or are those just examples? Asking because I don't really know how DNS is setup for Automat services.

gaurav commented 1 year ago

@yaphetKG @EvanDietzMorris Are there other steps we should be aware of when deploying Automat-CAM-KP to Sterling and ITRB?

gaurav commented 1 year ago

@gaurav, you have some existing CAM-KP API URLs up there (for dev and renci-prod). Were you thinking that Automat version would end up at those same URLs, or are those just examples? Asking because I don't really know how DNS is setup for Automat services.

Me neither! It would be great if we could just continue to use the same URLs (in which case we won't need to change the SmartAPI registration). If not, then the SmartAPI registrations would need to either be changed or deleted and recreated with the new URLs.

EvanDietzMorris commented 1 year ago

Without some redirections implemented the URLs will have to change, but the platers do generate their own openapi spec so it should be relatively easy to register for smartapi. @YaphetKG knows way more about this step than me though.

EvanDietzMorris commented 1 year ago

There is some complication here surrounding ITRB deployments.. currently all of the plater helm charts live here https://github.com/helxplatform/translator-devops and we make requests for pushing to ITRB test and prod through a different ITRB slack channel (devops-sri-ranking), typically many platers at a time (CI deployments happen automatically with jenkins when commits are merged to a certain branch in that devops repo). It shouldn't be a problem to push cam kp on it's own but it might be confusing to cross the request channels (at least from ITRBs perspective). Also - we don't always deploy a dev/prod version of every plater on Sterling, it looks like we call what you're calling renci-prod dev, but we could for you.

gaurav commented 1 year ago

There is some complication here surrounding ITRB deployments.. currently all of the plater helm charts live here https://github.com/helxplatform/translator-devops and we make requests for pushing to ITRB test and prod through a different ITRB slack channel (devops-sri-ranking), typically many platers at a time (CI deployments happen automatically with jenkins when commits are merged to a certain branch in that devops repo). It shouldn't be a problem to push cam kp on it's own but it might be confusing to cross the request channels (at least from ITRBs perspective).

I think it probably makes more sense to push CAM-KP along with all the other platers (in #devops-sri-ranking) instead of #devops-icees, as long as that doesn't create additional work for y'all!

Also - we don't always deploy a dev/prod version of every plater on Sterling, it looks like we call what you're calling renci-prod dev, but we could for you.

  1. For consistency, I've renamed those instances to RENCI-exp (https://cam-kp-api-dev.renci.org/1.3.0/docs/index.html?url=docs.yaml) and RENCI-dev (https://cam-kp-api.renci.org/1.3.0/docs/index.html?url=docs.yaml).
  2. I don't think anyone is using CAM-KP RENCI-dev, but it is registered on SmartAPI, so I prefer to test code on RENCI-exp first. But I don't think there's a problem with going straight to RENCI-dev if that's easier!

Thanks for pointing these out, Evan!

gaurav commented 1 year ago

Hi everybody! I've been playing around with Automat-CAM-KP in #91. The good news is that two of the test queries work fine, and two that produce an internal error both use deprecated Biolink 2 predicates. A few queries return . Queries with Biolink 3 predicates (with qualifiers) return zero results, which is not surprising. Two queries return results in CAM-KP-API but not in Automat-CAM-KP, which might be caused by modeling issues or maybe just missing data. Although CAM-KP-API is still better overall than Automat-CAM-KP, I think all the extra features we get -- including TRAPI 1.4 support eventually! -- means that we should go ahead and roll this out at least to ITRB CI while we try to get qualified predicates to work and while we try to figure out why those two queries don't work. Does that sound right?

I also tried some of the SRI testing data at random, and I was always able to retrieve the testing data using the query endpoint. And I've confirmed that https://automat-dev.apps.renci.org/cam-kp/1.3 could be used as an OTHER endpoint at https://arax.ncats.io/. So this is looking pretty good!

@EvanDietzMorris When are y'all planning to deploy to Automat on RENCI next? Could we include CAM-KP in that deployment?

EvanDietzMorris commented 1 year ago

Hey @gaurav, it's really easy to deploy these at RENCI and it doesn't have to be at the same time as the other platers, so just let me know anytime you want me to deploy it on Sterling (exp or dev). Deploying to CI is a bit different than the rest but it's easy as well, so just let me know when you want to go for that. Updating the smart api registrations is another process and we might want to involve Yaphet if you have specific things you want there or that may differ from other platers.

Re: the missing data it could be due to changes in node identifiers or predicates during normalization, we could hunt down that kind of thing easily in the metadata files. I'm happy to help with that.

gaurav commented 1 month ago

I think this is fully done! Closing.