Custom pipelines and file error handling

eaglesemanation commented 1 month ago

I've been playing around with this widget for my small project to show usb camera feed on a small touchscreen display attached to Raspberry Pi 4 and I encountered 2 issues:

In current form, video stream would immediately freeze, but by adjusting pipeline to use video/x-raw I managed to get 640x480@30 with almost smooth playback except small stutters. So I propose adding new method to Video struct pub fn from_pipeline(pipeline: &str) -> Result<Video, Error> and add a clear error message in case input pipeline does not have an appsink element with name=iced_video or something like that. In the ideal scenario that would be a compile time error, but I'm not sure how that would look like.
At some point, I disconnected a camera while the process was still running, and it caused it to panic here. I think instead of panicking, this could emit an event that indicates that Video is no longer valid, and then it's up to the user to handle it. I'm still figuring out basics of Iced, so I'm not sure how that is usually done.

I'll probably submit PRs that address those, at least for the first one. But I wanted to share my thoughts before doing that.

jazzfool commented 1 month ago

Regarding the first point, what exactly did you change in the gstreamer pipeline that saw the improvement? I'm open to adding a Video::from_pipeline constructor but at the same time I'd like to address that issue more directly.

As for the panic, yes, I think logging the error and possibly emitting and publishing a Message would be better than just panicking here.

eaglesemanation commented 1 month ago

For pipeline I used multiple different ones with similar results, some with v4l2 hardware decoding acceleration, but ultimately I tried to skip any transcoding and landed on this:

v4l2src device=/dev/video0 ! video/x-raw,width=640,height=360,framerate=30/1,format=NV12 ! videoconvert ! videoscale ! appsink name=app_sink caps=video/x-raw,format=RGBA,pixel-aspect-ratio=1/1

Unfortunately RGBA is not exposed, so I still needed videoconvert, or at least I think that's why I needed it, I'm treading in extremely unfamiliar waters working with graphics

jazzfool commented 1 month ago

Ah, yeah that's a fairly bespoke pipeline. It makes sense to support custom pipelines for things like v4l2src, but yes we would just need to ensure name=app_sink exists etc. Also yes, you do still usually need videoconvert since video is typically encoded in YUV space (however if you search around you might be able to find a pipeline for v4l2 that does the conversion with hardware acceleration).

jazzfool commented 1 month ago

Resolved by #10

eaglesemanation commented 1 month ago

I see you've updated crate version, could you push to crates.io?

jazzfool / iced_video_player

Custom pipelines and file error handling #9