Put the new GO API into production (and spot check)

geneontology / go-fastapi

https://api.geneontology.org/

5 stars 3 forks source link

Put the new GO API into production (and spot check) #55

Closed cmungall closed 1 year ago

cmungall commented 1 year ago

[@kltm]

After the completion and testing of documentation for #14 , we will need to proceed with the final switchover to the new combined API. This is broken into a few steps:

[x] activating the new API at api.geneontology.org (and checking that logging is working)
[x] doing a check that no services we run are affected by this change, including checking log errors

There is still the open need to make sure that GO-CAM calls are being used by current widgets so that we can turn off the GO-CAM API. (see https://github.com/geneontology/api-gorest-2021/issues/6, https://github.com/geneontology/go-site/issues/1928)

[Original @kltm to fill in]

kltm commented 1 year ago

This is a prerequisite for finishing https://github.com/orgs/geneontology/projects/129 , as we need the fixed and merged API to run this.

kltm commented 1 year ago

Necessary for geneontology/amigo#696

kltm commented 1 year ago

We went to production this afternoon. Waiting to see that logs propagate to S3.

kltm commented 1 year ago

[x] When checking logs, see how fast application logs are growing and whether application log overflow is possible.

kltm commented 1 year ago

I can confirm that logs are propagating to S3.

kltm commented 1 year ago

I would note that while the error logs do exist in S3, they are not super useful in trying to figure out if there are service misses.

kltm commented 1 year ago

~@sierra-moxon Looking at the running machine, I see the line:~ ~4 S root 5034 5033 0 80 0 - 8797 - 28324 1 Sep14 ? 00:00:44 /code/.venv/bin/python /code/.venv/bin/gunicorn app.main:app --workers 4 --timeout 120 --worker-class uvicorn.workers.UvicornWorker --bind 0.0.0.0:8080 --log-level info --access-logfile combined_access_error.log --error-logfile combined_access_error.log --capture-output~ ~Are the error and access logs flipped?~

Sorry, misread.

kltm commented 1 year ago

Current flux is something like (very very grossly) 7G/y. Given that the disk is over 100G, I think we're good for the moment. (Although, it would be better to rotate into /dev/null or something on two weeks, can revisit; do not want to block now.)

kltm commented 1 year ago

Okay, I think we are officially in a "stable production". Whether we stay like this TBD, but this is done.