search

talex5 / wayland-proxy-virtwl

Allow guest VMs to open windows on the host
Apache License 2.0
114 stars 12 forks source link

virtio-gpu buffers not properly discarded in time #47

Open puckipedia opened 2 years ago

puckipedia commented 2 years ago

If I run wayland-proxy-virtwl in a memory-constrained VM (512MB), and resize foot a bunch, at some point it stops resizing properly, and instead throws the following exception:

Exception: Unix.Unix_error(Unix.ENOSPC, "DRM_IOCTL_VIRTGPU_RESOURCE_CREATE_BLOB", "")
Raised by primitive operation at Virtio_gpu__Dev.create_blob in file "virtio_gpu/dev.ml", line 74, characters 2-23
Called from Virtio_gpu__Dev.alloc in file "virtio_gpu/dev.ml", line 156, characters 21-1221
Called from Dune__exe__Relay.Shm.with_memory_fd in file "relay.ml", line 289, characters 16-51

However, if I persist and keep resizing, at some point it starts working, which seems to suggest to me that it's depending on a GC to discard the virtgpu resources?

talex5 commented 2 years ago

Yes, that's right. OCaml itself doesn't allow unmapping files directly (see the unmerged PR https://github.com/ocaml/ocaml/pull/389/files), but I included a copy of that ocaml_ba_unmap function in virtio_gpu_stubs.c, available via Utils:

https://github.com/talex5/wayland-proxy-virtwl/blob/5940346db2a4427f21c7b30a2593b179af36a935/virtio_gpu/utils.mli#L15-L18

It's currently only used to unmap the shared ring (see #31), but it could be used for the buffers too. The comment says it's only for things mapped with safe_map_file - I suspect because the default GC finalizer asserts that it isn't already freed - so might need to check that's the case where the buffers are mapped.