Promtail Pipelines

[elipe17]

Description: Now that we have an MVP of Loki and a logging dashboard in Grafana. We need to enhance the logging dashboard to allow querying based on the log message, log level, etc... We can do this by implementing Promtail pipelines which transform outgoing log messages to include metadata labels, i.e. log message, level, app name, etc...

Acceptance Criteria: Create a list of functional outcomes that must be achieved to complete this issue

• [x] Promtail pipeline(s) extract at least log level, message, and what app the log came from
• [ ] Grafana logging dashboard updated to query new metadata generated from Promtail pipeline
• [ ] Deployed PLG updated with new config and dashboard
• [ ] Testing Checklist has been run and all tests pass
• [ ] README is updated, if necessary

Tasks: Create a list of granular, specific work items that must be completed to deliver the desired outcomes of this issue

• [ ] Implement Promtail pipeline(s) to extract appropriate metadata
• [ ] Add new query variables to Logging dashboard
• [ ] Ensure new changes (if necessary) are applied to deployed configs/apps as well
• [ ] Run Testing Checklist and confirm all tests pass

Notes: Add additional useful information, such as related issues and functionality that isn't covered by this specific issue, and other considerations that will be helpful for anyone reading this

• Note 1
• Note 2
• Note 3

Supporting Documentation: Please include any relevant log snippets/files/screen shots

• Doc 1
• Doc 2

Open Questions: Please include any questions or decisions that must be made before beginning work or to confidently call this issue complete

• Open Question 1
• Open Question 2

[lhuxraft]

11/13/2024: Still working, no updates

[lhuxraft]

11/15: Issues with Docker yesterday (now fixed), added tags but are not holding (pairing w/ Eric)

[lhuxraft]

11/18: Unblocked, need about one more day to work on

[lhuxraft]

11/19: Eric and Mo resolved networking issues, @lhuxraft to confirm with Mo progress on the ticket

[lhuxraft]

11/20: Almost finished, planning to wrap up today