OpenCV webcam feed running on Streamlit isn't showing output.

[Prana-S]

For the past few days, I have been trying to put this demo of Mediapipe on Streamlit Cloud. The demo shows up and works fine on my computer, but when I upload it to Streamlit Cloud I see the text, but the webcam feed isn't there. Here's a photo of what i see: If you have any suggestions or questions please feel free to tell me. Below is my code:

import cv2
import mediapipe as mp
import numpy as np
import streamlit as st

mp_drawing = mp.solutions.drawing_utils
mp_drawing_styles = mp.solutions.drawing_styles
mp_pose = mp.solutions.pose
stframe = st.empty()

IMAGE_FILES = []
BG_COLOR = (192, 192, 192)
with mp_pose.Pose(
    static_image_mode=True,
    model_complexity=2,
    enable_segmentation=True,
    min_detection_confidence=0.5) as pose:
  for idx, file in enumerate(IMAGE_FILES):
    image = cv2.imread(file)
    image_height, image_width, _ = image.shape
    results = pose.process(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))

    if not results.pose_landmarks:
      continue
    print(
        f'Nose coordinates: ('
        f'{results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].x * image_width}, '
        f'{results.pose_landmarks.landmark[mp_pose.PoseLandmark.NOSE].y * image_height})'
    )

    annotated_image = image.copy()
    condition = np.stack((results.segmentation_mask,) * 3, axis=-1) > 0.1
    bg_image = np.zeros(image.shape, dtype=np.uint8)
    bg_image[:] = BG_COLOR
    annotated_image = np.where(condition, annotated_image, bg_image)
    mp_drawing.draw_landmarks(
        annotated_image,
        results.pose_landmarks,
        mp_pose.POSE_CONNECTIONS,
        landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
    cv2.imwrite('/tmp/annotated_image' + str(idx) + '.png', annotated_image)
    mp_drawing.plot_landmarks(
        results.pose_world_landmarks, mp_pose.POSE_CONNECTIONS)
st.title("Mediapipe Feed")
cap = cv2.VideoCapture(0)
with mp_pose.Pose(
    min_detection_confidence=0.5,
    min_tracking_confidence=0.5) as pose:
  while cap.isOpened():
    success, image = cap.read()
    if not success:
      print("Ignoring empty camera frame.")
      continue

    image.flags.writeable = False
    image = cv2.cvtColor(image, cv2.COLOR_BGR2RGB)
    results = pose.process(image)

    image.flags.writeable = True
    image = cv2.cvtColor(image, cv2.COLOR_RGB2BGR)
    mp_drawing.draw_landmarks(
        image,
        results.pose_landmarks,
        mp_pose.POSE_CONNECTIONS,
        landmark_drawing_spec=mp_drawing_styles.get_default_pose_landmarks_style())
    stframe.image(image,  channels="BGR")

cap.release()
cap.destroyALLWindows()

[ayushgdev]

Hello @Prana-S Streamlit integration with MediaPipe has not been tested explicitly. Please allow us sometime to recreate this issue at our end.

[ayushgdev]

@Prana-S Are there any errors on the console?

[Prana-S]

@ayushgdev Sorry for the late response, yes there are logs:

[ayushgdev]

Hello @Prana-S The logs indicate that the OpenCV is not able to open the webcam. Camera index out of range is a common error which can come due to multiple reasons at the core, but all arise due to webcam being inaccessible by the opencv. Since multiple reasons can be the root cause, multiple things can be tried:

1. Make sure your webcam is connected if you are using external one
2. Make sure your webcam is internally accessible, detected by the OS and not disabled. Some systems do allow such capabilities. (like MSI allows Fn+F6 to disable detection of webcam altogether)
3. For accessing webcam over the web in streamlit cloud, you need to use streamlit-webrtc. Check this QnA. (This looks like the most probable cause) .
4. Make sure the permissions are allowed in the browser as well to use webcam

[Prana-S]

That seemed to work but now I have a different issue. The video feed isn't showing up, but the camera is turned on. I proved this with the activity light on the camera. Here is a screenshot of what I see after I click the start button. This function works on localhost, just not on Streamlit Cloud.

[ayushgdev]

@Prana-S From what could be inferred from the forums, cv2.VideoCapture would not work with streamlit-cloud. Please check this thread.

This is a streamlit specific issue since the webcam and MediaPipe solution works in the local setup. We would suggest opening a new thread in streamlit community forum.

[Prana-S]

@ayushgdev It isn't using Opencv as the webcam input. It is using Streamlit web-rtc.

[ayushgdev]

@Prana-S Did you open any issue in the Streamlit community forum? This is a streamlit specific issue MediaPipe solution works in the local setup.

[Prana-S]

No, but will do. Thanks for the help.

[google-ml-butler[bot]]

Are you satisfied with the resolution of your issue? Yes No