serial: Add --device virtio-serial,pty support

[cfergeau]

This new virtio-serial option allocates a pseudo-tty for the VM console. It can then be accessed using screen for example. This is a bit similar to the --device virtio-serial,stdio option, except that the console is not tied to the terminal running vfkit, it's possible to connect/disconnect from the pseudo-tty from any terminal.

This fixes https://github.com/crc-org/vfkit/issues/48

[openshift-ci[bot]]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Once this PR has been reviewed and has the lgtm label, please ask for approval from cfergeau. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files: - **[OWNERS](https://github.com/crc-org/vfkit/blob/main/OWNERS)** Approvers can indicate their approval by writing `/approve` in a comment Approvers can cancel approval by writing `/approve cancel` in a comment

[anjannath]

tested and working as expected..

% ./out/vfkit \
    --cpus 2 --memory 2048 \
    --bootloader efi,variable-store=efi-variable-store,create \
    --device virtio-blk,path=vfkit-test-image.raw --device virtio-serial,pty
INFO[0000] &{2 2048    {[efi variable-store=efi-variable-store create] true}  [virtio-blk,path=vfkit-test-image.raw virtio-serial,pty] none://  false}
INFO[0000] boot parameters: &{EFIVariableStorePath:efi-variable-store CreateVariableStore:true}
INFO[0000]
INFO[0000] virtual machine parameters:
INFO[0000]  vCPUs: 2
INFO[0000]  memory: 2048 MiB
INFO[0000]
INFO[0000] Adding virtio-blk device (imagePath: vfkit-test-image.raw)
INFO[0000] Using PTY (pty path: /dev/ttys002)
INFO[0000] virtual machine is running
INFO[0000] waiting for VM to stop

after this i was able to see the fedora login prompt by running % screen /dev/ttys002

[anjannath]

/lgtm

[praveenkumar]

I am testing it with crc bundle with the script part of contrib/script with following change but not able to get any login prompt

$ git diff
diff --git a/contrib/scripts/start-crc-bundle.sh b/contrib/scripts/start-crc-bundle.sh
old mode 100644
new mode 100755
index 34a28ee..965a497
--- a/contrib/scripts/start-crc-bundle.sh
+++ b/contrib/scripts/start-crc-bundle.sh
@@ -15,6 +15,6 @@ cp -c ${BUNDLE_PATH}/${DISKIMG} overlay.img
     --initrd "${BUNDLE_PATH}/${INITRD}" \
     --kernel-cmdline "${CMDLINE}" \
     --device virtio-blk,path=overlay.img \
-    --device virtio-serial,logFilePath=start-bundle.log \
+    --device virtio-serial,pty \ 
     --device virtio-net,nat,mac=72:20:43:d4:38:62 \
     --device virtio-rng

$ contrib/scripts/start-crc-bundle.sh /Users/prkumar/.crc/cache/crc_vfkit_4.13.14_arm64
[...]
INFO[0000]                                              
INFO[0000] virtual machine parameters:                  
INFO[0000]  vCPUs: 2                                    
INFO[0000]  memory: 2048 MiB                            
INFO[0000]                                              
INFO[0000] Adding virtio-blk device (imagePath: overlay.img) 
INFO[0000] Using PTY (pty path: /dev/ttys000)            <= tried to connect this with screen and nothing
INFO[0000] Adding virtio-net device (nat: true macAddress: [72:20:43:d4:38:62]) 
INFO[0000] Adding virtio-rng device                     
INFO[0000] virtual machine is running                   
INFO[0000] waiting for VM to stop  

I will try with fedora also same what @anjannath tested and put another comment.

[praveenkumar]

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

[cfergeau]

To me even steps mentioned by @anjannath didn't work after screen /dev/ttys000 I just see the blank without any prompt.

You need to make sure you have console=hvc0 on the kernel cmd line if you are booting with external kernel. If I connect late in the boot process, sometimes I cannot get output, not clear why.

[anjannath]

I tried again and it is working for me still, maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

https://github.com/crc-org/vfkit/assets/8885742/befe219b-cbc4-42ca-b8a5-3da6e50deafc

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

[praveenkumar]

maybe you have to attach to the pty quickly otherwise you miss the login prompt and it doesn't re-appear?

I also tried to attach pty quickly but still nothing, waiting more that 5 mins.

[praveenkumar]

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

[cfergeau]

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly. Really not familiar with pty/terminal input/output :-/

[praveenkumar]

but another thing i noticed is that the keyboard only works sometimes, a few times (2 out of 4 runs) during testing keys are not echoed in screen, which i assumed to be the keyboard not working

So after 5-6 retry finally I am able to get the login prompt for fedora and crc bundle but then on crc I am not able to use the keyboard but on fedora able to do.

Maybe virtio: Simplify setRawMode is causing issues. Or maybe it would be better to use https://github.com/pkg/term/blob/1a4a3b719465afccedae7788469c0b54a43417e0/termios/termios.go#L41-L50 , but I could not get this to work correctly. Really not familiar with pty/terminal input/output :-/

I am also not sure what causing the issue but as of now this functionality is quite buggy and we might not able to consume it when required like debugging an broken ssh connection. Do you think we can use tty along with log to a file because in tty I also don't see boot logs ?

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md). If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.