logging: Update the logging server endpoint

vunnyso commented 1 month ago

With updated server endpoint url, loki will be using https with basic auth. Access using insecure ip:port and http no longer possible.

Description of changes

Checklist for things done

[x] Summary of the proposed changes in the PR description
[x] More detailed description in the commit message(s)
[x] Commits are squashed into relevant entities - avoid a lot of minimal dev time commits in the PR
[x] Contribution guidelines followed
[ ] Ghaf documentation updated with the commit - https://tiiuae.github.io/ghaf/
[ ] PR linked to architecture documentation and requirement(s) (ticket id)
[x] Test procedure described (or includes tests). Select one or more:
- [x] Tested on Lenovo X1 x86_64
- [ ] Tested on Jetson Orin NX or AGX aarch64
- [ ] Tested on Polarfire riscv64
[ ] Author has run nix flake check --accept-flake-config and it passes
[x] All automatic Github Action checks pass - see actions
[x] Author has added reviewers and removed PR draft status

Testing

Prerequisite: Depends on https://github.com/tiiuae/ghaf-infra/pull/207 to deployed on server (Now its Merged and Deployed).

What can be checked? Need to identify if there is more frequent journal logs from system with this change and check if any other issues found.

vunnyso commented 1 month ago

ghaf-infra change already merged https://github.com/tiiuae/ghaf-infra/pull/207

brianmcgillion commented 1 month ago

the gap shows that the messages stopped when the new server hardening was deployed and now it is active again after these changes are applied to the device image

brianmcgillion commented 1 month ago

it seems that there is a lot more traffic being created in the net-vmafter the change. the left side of the image was the status with pr #701. the break is when the server hardening was deployed. the right side is with this #700 applied and starting to log.

vunnyso commented 1 month ago

it seems that there is a lot more traffic being created in the net-vmafter the change. the left side of the image was the status with pr #701. the break is when the server hardening was deployed. the right side is with this #700 applied and starting to log.

I have tried on my setup I have noticed only minor spikes in net-vm with #700 change.

leivos-unikie commented 1 month ago

Test results:

All apps launch
ci-test-automation pass

Regarding traffic caused by logging I tried iftop -t -s 300 > iftop.txt in admin-vm: iftop.txt

In addition:

Logging doesn't seem to have any effect on performance
However performance tests can be seen in the log volumes, e.g. as spikes for each vm. Especially during performance tests on VMs net-vm log volume drops down to ~40 (it has been normally quite steadily around 170 after boot).
If I do sudo systemctl restart microvm@net-vm.service net-vm log volume drops from ~170 to ~40 (until next boot).

tiiuae / ghaf