search

tiiuae / ghaf

TII SSRC Secure Technologies: Ghaf Framework
https://tiiuae.github.io/ghaf/
Apache License 2.0
35 stars 56 forks source link

Storage (VM) phase 0.1 #705

Closed unbel13ver closed 2 weeks ago

unbel13ver commented 1 month ago

Description of changes

This is the initial implementation of the Storage solution. It uses impermanence framework along with virtiofs shares to make needed files/directories persistent.

Currently it allows to store WiFi configuration, user data from GUI-vm user home directory and common-purpose browser configuration and user sessions/data.

Each VM has an access to it's only data and does not know anything about other stored entities. The host system has an access to all the data stored.

Checklist for things done

Testing

  1. Boot the system. Connect to Wi-Fi network through GUI-VM.
  2. Open Chromium browser, go to any website and add it's address as a bookmark. Close browser window.
  3. Open Business-VM browser (MS365), log in to the Outlook. Close the browser window.
  4. Create any file in user's home directory in GUI-VM: touch ~/testfile
  5. Reboot the computer.
  6. Check that results of 1.-4. are survived the reboot. The Wi-Fi should connect automatically and all bookmarks/sessions/files are in place.

It is possible to restart a microvm without rebooting: systemctl restart microvm@<vm name>.service

vilvo commented 2 weeks ago

Did a bit of testing with net-vm. Would be good to draft the Testing-section of the PR description to align the expectations/known issues.

Anyway, here goes my findings after nixos-rebuild ... switch + reboot from this PR branch - for TWIMC:

Good:

Regression:

Notes:

vilvo commented 2 weeks ago 
* SSH signatures get regenerated on each reboot like before - to connect, one must either remove the ssh server fingerprints when connecting between `ghaf-host` <-> `-vm`s OR ignore the server verification. This is not critical as it concerns only `-debug`-builds and debug-scenarios but would be nice to persist the fingerprints over reboots.

@vunnyso - would you please test this PR and see if ^ affects the grafana upload between reboots?

vunnyso commented 2 weeks ago 
* SSH signatures get regenerated on each reboot like before - to connect, one must either remove the ssh server fingerprints when connecting between `ghaf-host` <-> `-vm`s OR ignore the server verification. This is not critical as it concerns only `-debug`-builds and debug-scenarios but would be nice to persist the fingerprints over reboots.

@vunnyso - would you please test this PR and see if ^ affects the grafana upload between reboots?

Hi @vilvo I have tried with this PR, somehow logs are not getting uploaded to grafana server. Note I have connected with wifi using nmcli and network is working fine in admin-vm

vilvo commented 2 weeks ago 
* SSH signatures get regenerated on each reboot like before - to connect, one must either remove the ssh server fingerprints when connecting between `ghaf-host` <-> `-vm`s OR ignore the server verification. This is not critical as it concerns only `-debug`-builds and debug-scenarios but would be nice to persist the fingerprints over reboots.

@vunnyso - would you please test this PR and see if ^ affects the grafana upload between reboots?

Hi @vilvo I have tried with this PR, somehow logs are not getting uploaded to grafana server. Note I have connected with wifi using nmcli and network is working fine in admin-vm

Thanks for the info. Please investigate more.

brianmcgillion commented 2 weeks ago 
* SSH signatures get regenerated on each reboot like before - to connect, one must either remove the ssh server fingerprints when connecting between `ghaf-host` <-> `-vm`s OR ignore the server verification. This is not critical as it concerns only `-debug`-builds and debug-scenarios but would be nice to persist the fingerprints over reboots.

@vunnyso - would you please test this PR and see if ^ affects the grafana upload between reboots?

Hi @vilvo I have tried with this PR, somehow logs are not getting uploaded to grafana server. Note I have connected with wifi using nmcli and network is working fine in admin-vm

Thanks for the info. Please investigate more.

Ultimatly I believe the problem is that all these connections are over ssh, and we are storing the .ssh/known_hosts but not the ephemeral keys that were used for the connection. so all the ssh related inter-vm will fail because the keys change on boot and the hosts will be untrusted. all the /run/waypipe-ssh/id_ed25519. And all the keys created by the packages/ssh-keys-helper . What if we move to more static keys now that we can store them. Eventually it will be just for debug and most uses will be replaced by the givc implementation so we can reduce the keys usage to only debug terminal based ssh <-> between vms.

vunnyso commented 2 weeks ago 
* SSH signatures get regenerated on each reboot like before - to connect, one must either remove the ssh server fingerprints when connecting between `ghaf-host` <-> `-vm`s OR ignore the server verification. This is not critical as it concerns only `-debug`-builds and debug-scenarios but would be nice to persist the fingerprints over reboots.

@vunnyso - would you please test this PR and see if ^ affects the grafana upload between reboots?

Hi @vilvo I have tried with this PR, somehow logs are not getting uploaded to grafana server. Note I have connected with wifi using nmcli and network is working fine in admin-vm

Thanks for the info. Please investigate more.

Ultimatly I believe the problem is that all these connections are over ssh, and we are storing the .ssh/known_hosts but not the ephemeral keys that were used for the connection. so all the ssh related inter-vm will fail because the keys change on boot and the hosts will be untrusted. all the /run/waypipe-ssh/id_ed25519. And all the keys created by the packages/ssh-keys-helper . What if we move to more static keys now that we can store them. Eventually it will be just for debug and most uses will be replaced by the givc implementation so we can reduce the keys usage to only debug terminal based ssh <-> between vms.

Yes even for logging hw-mac.service in admin-vm is failed to retrieve MACAddress from net-vm through ssh way.

vilvo commented 2 weeks ago

Yes even for logging hw-mac.service in admin-vm is failed to retrieve MACAddress from net-vm through ssh way.

Can you adapt this https://github.com/tiiuae/ghaf/pull/705/files#diff-477e661df628667d7a61761a39c61bfb59fa57fdba1e3b7e3e58468d2f5b396dR66-R70 to create a writable area with inpermanence for ssh server to generate the fingerprints once so that the fingerprints remain the same and won't cause errors from clients between reboots?

milva-unikie commented 2 weeks ago

Tested on Lenovo-X1 (lenovo-x1-carbon-gen11-debug on USB SSD)

The only issues I found have already been mentioned here:

Tested to be working:

I will test Lenovo-X1 installer with internal storage tomorrow.

Edit. Installer is working without issues!

unbel13ver commented 2 weeks ago

The issues with network applet and log collecting are caused by the PCI device enumeration. When persistent storage is enabled, there is a new VirtIO device appear in, for instance, Net-VM:

Before:
00:04.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:05.0 Network controller: Intel Corporation Raptor Lake PCH CNVi WiFi (rev 01)

After:
00:04.0 Mass storage controller: Red Hat, Inc. Virtio file system (rev 01)
00:05.0 Ethernet controller: Red Hat, Inc. Virtio network device
00:06.0 Network controller: Intel Corporation Raptor Lake PCH CNVi WiFi (rev 01)

which causes the Wi-Fi device name change:

Before:
wlp0s5f0

After:
wlp0s6f0
milva-unikie commented 2 weeks ago

Tested on Lenovo-X1 (lenovo-x1-carbon-gen11-debug on USB SSD)

Everything is working!