obsproject / obs-websocket

Remote-control of OBS Studio through WebSocket
GNU General Public License v2.0
3.91k stars 707 forks source link

Bug: GetSourceScreenshot returns image of preview and not program when both are the same scene but preview has changes #1257

Open nospam2k opened 1 month ago

nospam2k commented 1 month ago

Operating System Info

Windows 11

Other OS

No response

OBS Studio Version

Other

OBS Studio Version (Other)

30.2.3

obs-websocket Version

5.1.0

OBS Studio Log URL

No log needed

OBS Studio Crash Log URL

Not crashing

Expected Behavior

Make websocket request GetSourceScreenshot for currentProgramScene and get screen shot of what is visible in program view.

Current Behavior

Make websocket request GetSourceScreenshot for currentProgramScene and get screen shot of what is visible in preview view.

Steps to Reproduce

  1. Create Scene with a source.
  2. Set up websocket.
  3. In Studio Mode, Set Preview and Program to the same Scene.
  4. Toggle the source visibility.
  5. Make request for GetSourceScreenshot for the current scene.
  6. You will get back what you see in preview with the source visibility toggled.

Anything else we should know?

The reason this is a problem is apps like obs-web show the incorrect image of what is being sent to the stream if the preview scene has had sources toggled but not transitioned, when the scene is the same in program.

nospam2k commented 1 month ago

You just need to create one scene with one source and setup obs-websocket with no authentication. Set Preview and Program to the Scene, then toggle the source visibility so Preview is different than Program and click the web test button below (just need to change the url).

Here is the test code I'm using:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>OBS Program View Screenshot</title>
</head>
<body>
  <h1>OBS Program View Screenshot</h1>
  <button id="getScreenshot">Get Screenshot</button>
  <br><br>
  <img id="screenshot" alt="OBS Screenshot will appear here">

  <script>
    const obsWebSocketUrl = 'ws://192.168.0.189:4455';  // WebSocket URL for OBS
    let obsSocket;

    // Function to connect to OBS WebSocket
    function connectOBS() {
      obsSocket = new WebSocket(obsWebSocketUrl);

      obsSocket.onopen = function () {
        console.log('Connected to OBS WebSocket');
        identifyOBS();  // Send identify message upon connection
      };

      obsSocket.onmessage = function (event) {
        const message = JSON.parse(event.data);
        handleOBSResponse(message);
      };

      obsSocket.onerror = function (error) {
        console.error('WebSocket error:', error);
      };

      obsSocket.onclose = function (event) {
        console.log('OBS WebSocket connection closed');
        console.log('Close event code:', event.code);
        console.log('Close event reason:', event.reason);
        if (event.wasClean) {
          console.log('Connection closed cleanly');
        } else {
          console.error('Connection closed unexpectedly');
        }
      };
    }

    // Identify the connection with OBS
    function identifyOBS() {
      const identifyMessage = {
        "op": 1,  // Identify
        "d": {
          "rpcVersion": 1
        }
      };
      obsSocket.send(JSON.stringify(identifyMessage));
    }

    // Handle OBS responses
    function handleOBSResponse(message) {
      if (message.op === 7 && message.d.requestId === "getProgramScene") {
        // Handle the response to GetCurrentProgramScene
        const currentScene = message.d.responseData.currentProgramSceneName;
        console.log('Current Program Scene:', currentScene);
        requestScreenshot(currentScene);
      } else if (message.op === 7 && message.d.requestId === "getScreenshot") {
        // Handle the response to GetSourceScreenshot
        const { imageData } = message.d.responseData;

        // Log imageData for debugging
        console.log('Received imageData:', imageData);

        const imgElement = document.getElementById('screenshot');

        // Set the image source
        imgElement.src = imageData;
      } else if (message.op === 2) {
        console.log('Identified successfully');
      }
    }

    // Function to request the current program scene
    function requestProgramScene() {
      const programSceneRequest = {
        "op": 6,  // Request
        "d": {
          "requestType": "GetCurrentProgramScene",
          "requestId": "getProgramScene"
        }
      };
      obsSocket.send(JSON.stringify(programSceneRequest));
    }

    // Function to request a screenshot of the current program scene
    function requestScreenshot(sceneName) {
      const screenshotRequest = {
        "op": 6,  // Request
        "d": {
          "requestType": "GetSourceScreenshot",
          "requestId": "getScreenshot",
          "requestData": {
            "sourceName": sceneName,  // Request screenshot of the current program scene
            "imageFormat": "png",
            "imageWidth": 960,
            "imageHeight": 540
          }
        }
      };
      obsSocket.send(JSON.stringify(screenshotRequest));
    }

    // Event listener for button click
    document.getElementById('getScreenshot').addEventListener('click', function () {
      requestProgramScene();  // Request the current program scene each time the button is clicked
    });

    // Start the connection to OBS WebSocket
    connectOBS();
  </script>
</body>
</html>
nospam2k commented 1 month ago

ChatGPT suggested and alternative way of capturing the frame from Program:

#include <obs-module.h>

// Function to capture the current frame from the program view
void capture_program_view_frame() {
    obs_output_t *output = obs_get_output("obs-output-name"); // Replace with your output name
    if (!output) {
        blog(LOG_ERROR, "Failed to get output.");
        return;
    }

    obs_source_frame_t *frame = obs_output_get_video_frame(output);
    if (frame) {
        // Process the frame as needed
        blog(LOG_INFO, "Captured frame: %dx%d", frame->width, frame->height);

        // Don't forget to release the frame after processing
        obs_source_frame_release(frame);
    } else {
        blog(LOG_ERROR, "Failed to capture frame.");
    }
}
Important Functions
obs_get_output(): Gets the current output by name.
Make sure to replace "obs-output-name" with the actual name of your output (like "Output" or "Streaming").
obs_output_get_video_frame(): Retrieves the current video frame from the output.
obs_source_frame_release(): Frees the frame after you are done processing it.