Capture still image with maximum camera resolution

[kalfasyan]

Hi, first of all thank you for the amazing work! I am trying to capture a still image with the maximum camera resolution. It's my first day trying Streamlit and streamlit-webrtc so apologies in advance for not understanding some basic principle.

I found this on your forums:

webrtc_streamer(key="example", 
            video_processor_factory=VideoProcessor, 
            media_stream_constraints= {"video": 
                                                {"width": {"ideal": 2500, "min": 1280}, 
                                                "height": {"ideal": 1000, "min": 720}}, })

but this is still about streaming video.. What I'm wondering is how will I just take one image with max camera-device resolution? st.camera_input doesn't seem to do the trick either and it saves an image based on the browser-displayed resolution..

[kalfasyan]

What I found is that Chrome/Firefox start the webcam stream with low resolution and it increases after 30seconds to maximize after ~45 seconds. Any idea why that is? On Brave it is stuck at a lower resolution (640x480).

Another thing I tried is a snapshot function to take still images:

class OpenCVVideoProcessor(VideoProcessorBase):
    frame_lock: threading.Lock

    def __init__(self) -> None:
        self.frame_lock = threading.Lock()
        self.img = None

    def recv(self, frame: av.VideoFrame) -> np.ndarray:
        img = frame.to_ndarray(format="bgr24")
        with self.frame_lock:
            self.img = img
        self.img = img
        return av.VideoFrame.from_ndarray(img, format="bgr24")

ctx = webrtc_streamer(key="snapshot", 
            video_processor_factory=OpenCVVideoProcessor, 
            mode=WebRtcMode.SENDRECV,
            async_processing=True,
            rtc_configuration={  # Add this config
                "iceServers": [{"urls": ["stun:stun.l.google.com:19302"]}]
            },
            media_stream_constraints= {"video": 
                                                {"width": {"ideal": 4056, "min": 1280}, 
                                                "height": {"ideal": 3040, "min": 720}},
                                        "audio": False,
                                        }
            )

if ctx.video_transformer:

    snap = st.button("Snapshot")
    if snap:
        with ctx.video_transformer.frame_lock:
            out_image = ctx.video_transformer.img

        if out_image is not None:
            data = out_image#.to_ndarray(format="bgr24")
            st.write("Output image:")
            st.image(data, channels="BGR")
            my_path = os.path.abspath(os.path.dirname(__file__))       
            cv2.imwrite(os.path.join(my_path, f"data/{fname}.png"), data)
            st.write(f"Image saved with filename: {fname}")
        else:
            st.warning("No frames available yet.")

Another problem is that when deployed on streamlit share, it doesn't work on a Xiaomi Poco 3 pro but it works fine on a Samsung A41..Still no idea why

[whitphx]

TBH I rather just used the Media API and the WebRTC API provided by the browsers, and have not investigated how the resolution is determined well...

Technically, there are these 2 APIs under the hood, Media API to access the WebCam and get image data, and WebRTC API to transport the data. It looks like the WebRTC API reduces the resolution automatically according to the resources and the environment such as CPU power or network bandwidth. (I saw the following pages) https://bloggeek.me/tweaking-webrtc-video-quality-unpacking-bitrate-resolution-and-frame-rates/ https://groups.google.com/g/discuss-webrtc/c/9g2eXWJwijI Yet I'm still not 100% confident and don't know the solution (or whether the solution exists or not).

BTW, as you mentioned, I think st.camera_input is the right way to capture still images rather than this extension. So it would be better for you to create an issue on the Streamlit official GitHub repo about changing resolution with st.camera_input.

[kalfasyan]

Thanks for the reply. Yeah the camera_input would be great if they add max resolution as a feature.

What I am doing now is using the st. file_uploader and that gives smartphone users the option to take a picture with their camera or browse files to upload. Using the camera is in max resolution.