search

eduramiba / webcam-capture-driver-native

Native driver for Webcam Capture API
26 stars 8 forks source link

Can't display camera stream on some resolutions on Windows #26

Open Rizen59 opened 4 months ago

Rizen59 commented 4 months ago

Hi @eduramiba,

I just tested the project and have some issues on Windows 10, not MacOS. Same on a colleage laptop. I ran TestDriver.java from the project. It does not display anything.

What I notice is it works with a resolution of 160x120, 320x180, 320x240 but not beyond.

Here are the logs when I switched to 160120, then the ones in 640 480

14:40:02.784 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Closing webcam Integrated Webcam
14:40:02.784 [atomic-processor-1] INFO com.github.sarxos.webcam.ds.cgt.WebcamCloseTask -- Closing Integrated Webcam
INFO_LEVEL: SessionProcessor: stopCapture - capture is stopped!!!  Error code: 0000000000, Description: Success with the result.
INFO_LEVEL: SessionProcessor: closeSession - session is closed!!!  Error code: 0000000000, Description: Success with the result.
14:40:03.073 [Thread-7] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (7, 1)
14:40:03.074 [Thread-8] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (9, 1)
14:40:03.079 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Webcam Integrated Webcam has been closed
14:40:03.079 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Setting new resolution 160x120
14:40:03.080 [atomic-processor-1] INFO com.github.sarxos.webcam.ds.cgt.WebcamOpenTask -- Opening webcam Integrated Webcam
14:40:03.080 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Media types for resolution 160x120 = [CaptureManagerMediaType{width=160, height=120, majorType=MFMediaType_Video, subType=MFVideoFormat_YUY2}]
14:40:03.080 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Media types for resolution 160x120 = [CaptureManagerMediaType{width=160, height=120, majorType=MFMediaType_Video, subType=MFVideoFormat_YUY2}]
14:40:03.081 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Using video media type: CaptureManagerMediaType{width=160, height=120, majorType=MFMediaType_Video, subType=MFVideoFormat_YUY2}
INFO_LEVEL: SessionProcessor: setTopology - topology is ready!!!  Error code: 0000000000, Description: Success with the result.
14:40:03.210 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- Successfully created CallSessionControl
INFO_LEVEL: SessionProcessor: startCapture - capture is started!!!  Error code: 0000000000, Description: Success with the result.
14:40:03.356 [Thread-9] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (5, 2)
14:40:03.355 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Webcam is now open Integrated Webcam
14:40:21.155 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Closing webcam Integrated Webcam
14:40:21.155 [atomic-processor-1] INFO com.github.sarxos.webcam.ds.cgt.WebcamCloseTask -- Closing Integrated Webcam
INFO_LEVEL: SessionProcessor: stopCapture - capture is stopped!!!  Error code: 0000000000, Description: Success with the result.
INFO_LEVEL: SessionProcessor: closeSession - session is closed!!!  Error code: 0000000000, Description: Success with the result.
14:40:21.444 [Thread-10] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (7, 2)
14:40:21.444 [Thread-11] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (9, 2)
14:40:21.446 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Webcam Integrated Webcam has been closed
14:40:21.446 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Setting new resolution 640x480
14:40:21.446 [atomic-processor-1] INFO com.github.sarxos.webcam.ds.cgt.WebcamOpenTask -- Opening webcam Integrated Webcam
14:40:21.446 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Media types for resolution 640x480 = [CaptureManagerMediaType{width=640, height=480, majorType=MFMediaType_Video, subType=MFVideoFormat_NV12}, CaptureManagerMediaType{width=640, height=480, majorType=MFMediaType_Video, subType=MFVideoFormat_MJPG}, CaptureManagerMediaType{width=640, height=480, majorType=MFMediaType_Video, subType=MFVideoFormat_YUY2}]
14:40:21.446 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Media types for resolution 640x480 = [CaptureManagerMediaType{width=640, height=480, majorType=MFMediaType_Video, subType=MFVideoFormat_NV12}]
14:40:21.446 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice -- Using video media type: CaptureManagerMediaType{width=640, height=480, majorType=MFMediaType_Video, subType=MFVideoFormat_NV12}
INFO_LEVEL: SessionProcessor: setTopology - topology is ready!!!  Error code: 0000000000, Description: Success with the result.
14:40:21.548 [atomic-processor-1] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- Successfully created CallSessionControl
INFO_LEVEL: SessionProcessor: startCapture - capture is started!!!  Error code: 0000000000, Description: Success with the result.
14:40:21.722 [JavaFX Application Thread] DEBUG com.github.sarxos.webcam.Webcam -- Webcam is now open Integrated Webcam
14:40:21.722 [Thread-12] INFO com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession -- invoke with (aCallEventCode, aSessionDescriptor) = (5, 3)

What should I check to have more information? Have you ever notice this problem?

Here is the code used to add a resolution choice box. Also notice that camera.close() is not enough to close the camera. It must be followed by dispose().

package com.github.eduramiba.webcamcapture;

import com.github.eduramiba.webcamcapture.drivers.NativeDriver;
import com.github.eduramiba.webcamcapture.drivers.WebcamDeviceWithBufferOperations;
import com.github.sarxos.webcam.Webcam;
import com.github.sarxos.webcam.WebcamDevice;

import java.awt.*;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

import com.github.sarxos.webcam.WebcamResolution;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.beans.value.ChangeListener;
import javafx.beans.value.ObservableValue;
import javafx.collections.FXCollections;
import javafx.scene.Scene;
import javafx.scene.control.Button;
import javafx.scene.control.ChoiceBox;
import javafx.scene.image.ImageView;
import javafx.scene.image.WritableImage;
import javafx.scene.layout.HBox;
import javafx.stage.Stage;
import javafx.util.StringConverter;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class TestDriver extends Application {

    private static final Logger LOG = LoggerFactory.getLogger(TestDriver.class);

    public static final ScheduledExecutorService EXECUTOR = Executors.newScheduledThreadPool(4);

    public static void main(String[] args) {
        Webcam.setDriver(new NativeDriver());

        launch(args);
    }

    public void changeResolution(Webcam camera, Dimension dimension) {
        camera.close();
        camera.getDevice().dispose();
        camera.setViewSize(dimension);
        camera.open();
    }

    @Override
    public void start(Stage stage) throws Exception {
        final ImageView imageView = new ImageView();
        final ChoiceBox<Dimension> resolutionChoiceBox = new ChoiceBox<>();
        resolutionChoiceBox.setConverter(new StringConverter<>() {

            @Override
            public String toString(Dimension dimension) {
                return dimension != null ? dimension.width + " x " + dimension.height : null;
            }

            @Override
            public Dimension fromString(String s) {
                return null;
            }
        });
        final HBox root = new HBox();
        root.getChildren().addAll(imageView, resolutionChoiceBox);

        Webcam.getWebcams().stream()
                .findFirst()
                .ifPresent((final Webcam camera) -> {
                    final WebcamDevice device = camera.getDevice();

                    resolutionChoiceBox.setItems(FXCollections.observableArrayList(device.getResolutions()));
                    camera.setViewSize(WebcamResolution.QVGA.getSize());
                    resolutionChoiceBox.getSelectionModel().select(WebcamResolution.QVGA.getSize());
                    resolutionChoiceBox.getSelectionModel().selectedItemProperty().addListener(new ChangeListener<Dimension>() {
                        @Override
                        public void changed(ObservableValue<? extends Dimension> observableValue, Dimension oldValue, Dimension newValue) {
                            changeResolution(camera, newValue);
                        }
                    });

                    LOG.info("Found camera: {}, device = {}", camera, device);

                    final int width = device.getResolution().width;
                    final int height = device.getResolution().height;
                    final WritableImage fxImage = new WritableImage(width, height);
                    Platform.runLater(() -> {
                        imageView.setImage(fxImage);
                        stage.setWidth(width + 100);
                        stage.setHeight(height + 100);
                        stage.centerOnScreen();
                    });

                    camera.getLock().disable();
                    camera.open();
                    if (device instanceof WebcamDeviceWithBufferOperations) {
                        final WebcamDeviceWithBufferOperations dev = ((WebcamDeviceWithBufferOperations) device);
                        EXECUTOR.scheduleAtFixedRate(new Runnable() {
                            private long lastFrameTimestamp = -1;

                            @Override
                            public void run() {
                                if (dev.updateFXIMage(fxImage, lastFrameTimestamp)) {
                                    lastFrameTimestamp = dev.getLastFrameTimestamp();
                                }

                            }
                        }, 0, 16, TimeUnit.MILLISECONDS);
                    }
                });

        stage.setOnCloseRequest(t -> {
            Platform.exit();
            System.exit(0);
        });

        // Create the Scene
        final Scene scene = new Scene(root);
        stage.setScene(scene);
        stage.setTitle("Webcam example");
        stage.show();
    }
}

Thanks for your feedback, and your great work ;)

eduramiba commented 4 months ago

Hi,

I have tested the library working with devices using even 4k resolution, both Windows and MacOS.

Are you using SDK capture manager 1.24.0 DLL? In that case, please make sure to use 1.23.0, as I have found it does not always work correctly with this approach and will probably need a new native binding to it that I still need to create.

Rizen59 commented 4 months ago

Thanks for your feedback,

I firstly tested with Capture Manager 1.21.0 which one is included in the project, then version 1.24.0. Both have the same behavior I mentionned above. Version 1.23.0 doesn't work at all which is more annoying ^^'

I also tested CaptureManagerSDK-JavaDemos which can be found at the https://www.codeproject.com/Articles/1017223/CaptureManager-SDK-Capturing-Recording-and-Streami

I continue my investigation but not sure what to check. I'm using the JDK 21 for information

eduramiba commented 4 months ago

That's quite strange. Could you let me know which camera is it? About java 21, I am using it to run a camera app without problems.

Rizen59 commented 3 months ago

It's an integrated Webcam of the Dell Vostro 7500 : https://www.dell.com/support/manuals/fr-fr/vostro-15-7500-laptop/mkb7500_setup_and_specs/camera?guid=guid-082a29ac-ef4a-4abf-8235-90348923eee6&lang=en-us

I read again my previous message and forgot to mention: the CaptureManagerSDK-JavaDemos I tested works well. But what is in output directory is based I think on the subproject CaptureManagerSDKJavaxDemo.

Is there any other logs I could check? Which version of JavaFX did u use?

Rizen59 commented 3 months ago

I have more information about the exception, a BufferOverFlowException when I choose a resolution more than 320x240.

java.nio.BufferOverflowException
    at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:1038)
    at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:250)
    at javafx.graphics@23-ea/com.sun.javafx.image.impl.BaseByteToByteConverter$ByteAnyToSameConverter.doConvert(BaseByteToByteConverter.java:199)
    at javafx.graphics@23-ea/com.sun.javafx.image.impl.BaseByteToByteConverter.convert(BaseByteToByteConverter.java:103)
    at javafx.graphics@23-ea/com.sun.javafx.image.impl.BaseByteToByteConverter$ByteAnyToSameConverter.convert(BaseByteToByteConverter.java:165)
    at javafx.graphics@23-ea/com.sun.prism.Image$BaseAccessor.setPixels(Image.java:1046)
    at javafx.graphics@23-ea/com.sun.prism.Image.setPixels(Image.java:725)
    at javafx.graphics@23-ea/javafx.scene.image.WritableImage$2.setPixels(WritableImage.java:231)
    at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession.updateFXIMage(CaptureManagerFrameGrabberSession.java:283)
    at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession.updateFXIMage(CaptureManagerFrameGrabberSession.java:273)
    at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice.updateFXIMage(CaptureManagerVideoDevice.java:259)
    at com.github.eduramiba.webcamcapture.TestDriver$3.run(TestDriver.java:104)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572)
    at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:358)
    at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java)
    at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
    at java.base/java.lang.Thread.run(Thread.java:1583)

image

eduramiba commented 3 months ago

Oh of course you should be careful to properly manage your image size so it's able to fit the new resolution. Please review UI code, image size must match

El lun, 24 jun 2024, 18:00, vianneyd @.***> escribió:

I have more information about the exception, a BufferOverFlowException when I choose a resolution more than 320x240.

java.nio.BufferOverflowException at java.base/java.nio.ByteBuffer.put(ByteBuffer.java:1038) at java.base/java.nio.HeapByteBuffer.put(HeapByteBuffer.java:250) at @./com.sun.javafx.image.impl.BaseByteToByteConverter$ByteAnyToSameConverter.doConvert(BaseByteToByteConverter.java:199) at @./com.sun.javafx.image.impl.BaseByteToByteConverter.convert(BaseByteToByteConverter.java:103) at @./com.sun.javafx.image.impl.BaseByteToByteConverter$ByteAnyToSameConverter.convert(BaseByteToByteConverter.java:165) at @./com.sun.prism.Image$BaseAccessor.setPixels(Image.java:1046) at @./com.sun.prism.Image.setPixels(Image.java:725) at @./javafx.scene.image.WritableImage$2.setPixels(WritableImage.java:231) at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession.updateFXIMage(CaptureManagerFrameGrabberSession.java:283) at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerFrameGrabberSession.updateFXIMage(CaptureManagerFrameGrabberSession.java:273) at com.github.eduramiba.webcamcapture.drivers.capturemanager.CaptureManagerVideoDevice.updateFXIMage(CaptureManagerVideoDevice.java:259) at com.github.eduramiba.webcamcapture.TestDriver$3.run(TestDriver.java:104) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:572) at java.base/java.util.concurrent.FutureTask.runAndReset$$$capture(FutureTask.java:358) at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java) at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) at java.base/java.lang.Thread.run(Thread.java:1583)

image.png (view on web) https://github.com/eduramiba/webcam-capture-driver-native/assets/321846/79ea1731-1f59-4d9a-91ad-091990090808

— Reply to this email directly, view it on GitHub https://github.com/eduramiba/webcam-capture-driver-native/issues/26#issuecomment-2186910102, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABQFJLMB7W7C6URGXBRXXTZJA7CRAVCNFSM6AAAAABJSB2X4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOBWHEYTAMJQGI . You are receiving this because you were mentioned.Message ID: @.***>

Rizen59 commented 3 months ago

You're probably rigth I did something wrong with the UI code.

But if I came back to the initial code in the project, initializing the resolution has a different behavior. And in that case not exception happen.

        Webcam.getWebcams().stream()
                .findFirst()
                .ifPresent((final Webcam camera) -> {
                    final WebcamDevice device = camera.getDevice();
                    device.setResolution(WebcamResolution.QVGA.getSize()); // Works well
                    LOG.info("Found camera: {}, device = {}", camera, device);

                    final int width = device.getResolution().width;
                    final int height = device.getResolution().height;
                    final WritableImage fxImage = new WritableImage(width, height);
        Webcam.getWebcams().stream()
                .findFirst()
                .ifPresent((final Webcam camera) -> {
                    final WebcamDevice device = camera.getDevice();
                    device.setResolution(WebcamResolution.HD.getSize()); // Does not work
                    LOG.info("Found camera: {}, device = {}", camera, device);

                    final int width = device.getResolution().width;
                    final int height = device.getResolution().height;
                    final WritableImage fxImage = new WritableImage(width, height);
eduramiba commented 3 months ago

Does that camera have exactly WebcamResolution.HD resolution?

I would make sure by listing available resolutions in the device and choosing one of those. When listing available resolutions from the device, which ones do you get?

Rizen59 commented 3 months ago

HD is in the list, like QVGA:

image