*: Data collection from FCOS machines

coreos / fedora-coreos-pinger

Telemetry service to be used in Fedora CoreOS (see https://github.com/coreos/fedora-coreos-tracker/issues/86)

Apache License 2.0

15 stars 7 forks source link

*: Data collection from FCOS machines #31

Open zonggen opened 4 years ago

zonggen commented 4 years ago

Source of truth for each part of data collected:

Minimal:
- platform: read_file /proc/cmdline
- current_os_version: read_output rpm-ostree status --json
- original_os_version: read_file /.coreos-aleph-version.json
- instance_type: read_file /run/metadata/afterburn
Full:
- hardware:
- disk: read_output lsblk --fs --json
- cpu: read_output lscpu --json
- memory: read_output lsmem --json
- network: read_output nmcli device show
- container_runtime:
  - podman:
    - check if running by pgrep podman
    - count running container by pgrep conmon | wc -l
  - docker:
    - check if running by pgrep dockerd
    - count running container by pgrep containerd-shim | wc -l
  - cri-o:
    - check if running by pgrep crio
    - count running container by pgrep crictl | wc -l
  - system-nspawn:
    - check if running and counting containers by pgrep systemd-nspawn

Signed-off-by: Allen Bai, abai@redhat.com

zonggen commented 4 years ago

Currently in main.rs the collected ignition platform id will be printed to the console, which will be updated later to be bundled with other data and sent to the remote endpoint.

zonggen commented 4 years ago

Also, this PR will be updated to resolve https://github.com/coreos/fedora-coreos-pinger/issues/30#issue-494138139

zonggen commented 4 years ago

UPDATE:

Added the module os-release.rs under identity to extract OS version from /etc/os-release

Note: Most portion of code is the same as platform.rs so in the future, towards the end of this PR should de-duplicate this part of code into a new utils helper module.

zonggen commented 4 years ago

STATUS: Currently Travis CI is failing since it is using Rust docker image which is based on Debian and it does not ship with lsmem, and lscpu does not contain an --json option. To test with more production-like environment, i.e. FCOS, it might be better to switch to UBI (https://www.redhat.com/en/blog/introducing-red-hat-universal-base-image). Will be updating with changes to see if it works.

EDIT:

Updated .travis.tml to use build a container using Dockerfile and test the code inside the UBI container

zonggen commented 4 years ago

STATUS:

Since collecting service is meant to running in the background and users should not expect visible error information, the program should not crash/return error when one of the commands returns non-zero and instead should suppress the error and collect whatever is available at the moment.
Could use a log file to dump the error message

zonggen commented 4 years ago

STATUS:

Client side data collection is done. though the send_data() function is not doing anything other than printing the collected data in Json format. This needs to be updated after we have an actual endpoint.
Persistent state so that pinger could send data periodically with some fixed interval will be added through another PR to reduce the size of this PR

Overall, ready for review!

zonggen commented 4 years ago

STATUS:

Built on top of https://github.com/coreos/fedora-coreos-pinger/pull/35, in the main function, to create a Daemon process, one process is forked and parent process is immediately terminated. Two threads are then spawned by child process to track the time for daily and monthly reports.
The two threads are put into sleep to avoid busy waiting, and only checks whether it needs to send new report every 12 hours / 15 days for daily and monthly reports correspondingly.

zonggen commented 4 years ago

cc @lucab @bgilbert

ashcrow commented 3 years ago

Is this something we want to continue pursuing?

bgilbert commented 3 years ago

Maybe not; xref https://github.com/coreos/fedora-coreos-tracker/issues/770.