coreos / fedora-coreos-tracker

Issue tracker for Fedora CoreOS
https://fedoraproject.org/coreos/
260 stars 60 forks source link

Add testing for kdump over network #1753

Closed jbtrystram closed 3 days ago

jbtrystram commented 1 week ago

We could host {SSH,NFS} server into a container like the tang test as well but the main issue i see is that in this scenario we need to :

So that requires access on two separate instances. We could nest them as we do in the iscsi tests, but that's very expensive as it requires pulling COSA and doing an installation each time. it's justifiable in the iSCSI case but i am not sure for kdump.

I haven't read too much into the tang test so I don't know if that's feasible. Quickly looking at the tang test it seems it's just a case of "create a container with some config"

jbtrystram commented 1 week ago

hmmm maybe a way to validate the test would be to simply crash the VM, wait for reboot, mount the NFS destination again and look for the expected files.

This way the source server would be just a "fire and forget" container like the tang test

mike-nguyen commented 1 week ago

Why can't we access both machines during the test?

travier commented 1 week ago

I vaguely remember that there are options for running multiple systems with kola.

Or we could have the first VM export an NFS mount, then start a VM inside it and point it to the host NFS, crash it and wait on the host for reboot to see if the crash dump is in the NFS mount?

jbtrystram commented 1 week ago

Or we could have the first VM export an NFS mount, then start a VM inside it and point it to the host NFS, crash it and wait on the host for reboot to see if the crash dump is in the NFS mount?

Starting a VM inside the NFS host is the same approach as the iscsi test, as I mentioned it's very expensive in term of bandwidth and make the test quite long..

mike-nguyen commented 1 week ago

I think you should be able to access both VMs in kola. If I recall correctly, the tang test sets up one machine with tang running as a container and another machine to use the first machine to decrypt with the tang fingerprint

jlebon commented 1 week ago

I think you should be able to access both VMs in kola.

Agreed, I think the same setup as Tang should work there with port forwarding. It'd be nice to try to do this with the iSCSI tests too, but I think it'd require some finagling to satisfy IP-related requirements (related: https://github.com/coreos/coreos-assembler/pull/3645).

jbtrystram commented 4 days ago

Reopening this as we should consider testing kdump over NFS as well

jlebon commented 4 days ago

Reopening this as we should consider testing kdump over NFS as well

Not against this, but also not really looking forward to have the team have to learn about NFS servers just for maintaining tests for this (whereas ssh is very familiar to most people).

How much value does this extra coverage bring vs the ssh one? Aren't they mostly on equal footing in the sense that our initrd by default doesn't include ssh?

(Anyway again, not blocking this.)

jbtrystram commented 3 days ago

How much value does this extra coverage bring vs the ssh one? Aren't they mostly on equal footing in the sense that our initrd by default doesn't include ssh?

I am not sure, I think NFS requires an extra kernel module to load but that's just an additional line in kdump config file. I you think it's not worth it i'll happily follow your advice and move on :)

jlebon commented 3 days ago

How much value does this extra coverage bring vs the ssh one? Aren't they mostly on equal footing in the sense that our initrd by default doesn't include ssh?

I am not sure, I think NFS requires an extra kernel module to load but that's just an additional line in kdump config file. I you think it's not worth it i'll happily follow your advice and move on :)

The intent is good. :) Just weighing the cost/benefit. We can always revisit if we think this would be very beneficial to cover.