Store longer history of logs for deployment

[jonrkarr]

The logs available in https://deployment.biosimulations.org/ (e.g., https://deployment.biosimulations.org/applications/biosimulations-prod?node=apps%2FDeployment%2Fprod%2Fapi%2F0&tab=logs) are helpful. However, they don't go back very far. It would be helpful to store these logs further back (e.g., 1 or a few months).

@bilalshaikh42 If these logs are already being stored, how I access them?

[bilalshaikh42]

@jonrkarr This is now available from https://monitoring.biosimulations.org/explore Select "Loki" as the datasource. The UI will then present multiple options for filters and selectors to explore the logs, allowing selection by app, namespace, pod, text search etc. The logs can also be grouped to remove repeats and similar lines. (see attached). I am not yet sure how much space these take up, but if we find that they are not stored long enough, it can very easily be expanded by attaching them to a google bucket.

For more advanced searching (eg specific app, simulation id, error code combinations ) you can build complex queries using https://grafana.com/docs/loki/latest/logql/

[jonrkarr]

Nice!

[jonrkarr]

I think this could be closed.