andreknieriem / photobooth

A photobooth Web-Application for raspberry pi with gphoto2
https://photobooth.andrerinas.de/
MIT License
295 stars 163 forks source link

gphoto2: preview at countdown from DSLR #242

Closed florianmueller closed 1 year ago

florianmueller commented 4 years ago

Is your feature request related to a problem? Please describe. First, thank you so much for this project, it is the best photo booth project out there, I tried a lot. I am missing a preview for people in front of the photo box before or while the image is taken. I know you can utilize device cams, but I would only have the DSLR live preview feed available.

Describe the solution you'd like A way to pipe out the cameras gphoto2 --capture-movie stream directly into the webinterface background.

Describe alternatives you've considered I tried setup a seperate http stream on port 8080 and use it in the user interface settings as background URL, but it does not work. It seems chromium has problems with disyplaing mjpeg streams directly as a background.

Additional context Thank you very much for any suggestion and help you could provide me on that topic.

couz74 commented 4 years ago

Hi, After multiples test I got this working:

In core.js

var pid;
    public.startVideo = function () {
          const data = {
            play: "true"
        };
        if (!navigator.mediaDevices) {
            return;
        }
         jQuery.post('api/takeVideo.php', data).done(function (result) {
               console.log('Start webcam',result);
               pid=result.pid;
                const getMedia = (navigator.mediaDevices.getUserMedia || 
 navigator.mediaDevices.webkitGetUserMedia || navigator.mediaDevices.mozGetUserMedia || false);

                if (!getMedia) {
                console.log('No user media');
                return;
                }`

                if (config.previewCamFlipHorizontal) {
                $('#video--view').addClass('flip-horizontal');
                }

                getMedia.call(navigator.mediaDevices, webcamConstraints)
                .then(function (stream) {
                    console.log('Success getting user media')
                    $('#video--view').show();
                    videoView.srcObject = stream;
                    public.stream = stream;
                })
                .catch(function (error) {
                console.log('Could not get user media: ', error)
            });
        }).fail(function (xhr, status, result) {
           console.log('Could not start webcam',result)
        });
    }
    public.stopVideoAndTakePic = function (data) {
        if (public.stream) {
             const dataVideo = {
            play: "false",
            pid: pid
            };

            jQuery.post('api/takeVideo.php', dataVideo).done(function (result) {
            console.log('Stop webcam',result)            
            const track = public.stream.getTracks()[0];
            track.stop();
            $('#video--view').hide();
            public.callTakePicApi(data);
            }).fail(function (xhr, status, result) {
           console.log('Could not stop webcam',result)
            });
        }
    }
// take Picture
    public.takePic = function (photoStyle) {
        if (config.dev) {
            console.log('Take Picture:' + photoStyle);
        }

        const data = {
            filter: imgFilter,
            style: photoStyle,
            canvasimg: videoSensor.toDataURL('image/jpeg')
        };  

        if (photoStyle === 'collage') {
            data.file = currentCollageFile;
            data.collageNumber = nextCollageNumber;
        }

        if (config.previewFromCam) {
            if (config.previewCamTakesPic && !config.dev) {
                videoSensor.width = videoView.videoWidth;
                videoSensor.height = videoView.videoHeight;
                videoSensor.getContext('2d').drawImage(videoView, 0, 0);
            }
            public.stopVideoAndTakePic(data);
        }else {
          public.callTakePicApi(data);
        }
     }

    public.callTakePicApi =function (data) {
        console.log(data);
      jQuery.post('api/takePic.php', data).done(function (result) {
            console.log('took picture', result);
            $('.cheese').empty();
            if (config.previewCamFlipHorizontal) {
                $('#video--view').removeClass('flip-horizontal');
            }

            // reset filter (selection) after picture was taken
            imgFilter = config.default_imagefilter;
            $('#mySidenav .activeSidenavBtn').removeClass('activeSidenavBtn');
            $('#' + imgFilter).addClass('activeSidenavBtn');

            if (result.error) {
                public.errorPic(result);
            } else if (result.success === 'collage' && (result.current + 1) < result.limit) {
                currentCollageFile = result.file;
                nextCollageNumber = result.current + 1;

                $('.spinner').hide();
                $('.loading').empty();
                $('#video--sensor').hide();

                if (config.continuous_collage) {
                    setTimeout(() => {
                        public.thrill('collage');
                    }, 1000);
                } else {
                    $('<a class="btn" href="#">' + L10N.nextPhoto + '</a>').appendTo('.loading').click((ev) => {
                        ev.preventDefault();

                        public.thrill('collage');
                    });
                    $('.loading').append($('<a class="btn" style="margin-left:2px" href="./">').text(L10N.abort));
                }
            } else {
                currentCollageFile = '';
                nextCollageNumber = 0;

                public.processPic(data.photoStyle, result);
            }

        }).fail(function (xhr, status, result) {
            public.errorPic(result);
        });
    }    

in takeVideo.php

<?php
header('Content-Type: application/json');

require_once('../lib/config.php');

function isRunning($pid){
    try{
        $result = shell_exec(sprintf("ps %d", $pid));
        if( count(preg_split("/\n/", $result)) > 2){
            return true;
        }
    }catch(Exception $e){}

    return false;
}

if ($_POST['play'] === "true" ) {
  $pid = exec('gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video0 > /dev/null 2>&1 & echo $!', $out);      
  sleep(3);  
  die(json_encode([
     'isRunning' => isRunning($pid),
      'pid' => $pid - 1
    ]));
}elseif($_POST['play'] === "false") { 
    exec('kill -15 '.$_POST['pid']);
    die(json_encode([
     'isRunning' => isRunning($_POST['pid']),
      'pid' => $_POST['pid']
    ]));
}

For it to work on startup I had to edit /etc/rc.local: modprobe v4l2loopback exclusive_caps=1 card_label="GPhoto2 Webcam" rmmod bcm2835-isp

rmmod bcm2835-isp was needed as chromium was taking this device instead of the V4l2 one.

How it work We use v4l2loopback to create a virtual device and gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video0 to send the photo preview to it. On "take pic" button we start the process and end to before the picture is taken to free gphoto2.

It is a ugly draft, I hope I will improve it and post the update here.

andi34 commented 4 years ago

@couz74 great! maybe you like to commit your changes and push them to GitHub?

couz74 commented 4 years ago

It's still not clean at all, I will try to do something nicer and push them

couz74 commented 4 years ago

I pushed the draft here https://github.com/couz74/photobooth. There is still two issue:

florianmueller commented 4 years ago

Thanks a lot for your solution! I used a similar very unstable solution outside your app, where I used a bash script as the "take picture" command that essential stoped the gphoto2 capture-movie, takes a picture and starts the capture video again. The issue was that running the --capture movie command as a service, crashes the live view quite quickly. So really appreciate your integrated approach into the application itself.

I tried running your solution, but having issues to make the livestream show up. Where do you enable it in the admin interface? Is it "Use device cam"? I don't see you using separate http service to stream the live view from /dev/video0 to a localhost port...like motion would do for example.

EDIT: it might be that on my pi the rmmod bcm2835-isp fails with rmmod: ERROR: Module bcm2835_isp is not currently loaded May that mean the pi is not recognizing the /dev/video0 live view as a webcam because the kernel module is not loaded for webcams?

couz74 commented 4 years ago

Hi, In the admin panel the setting "See preview by device cam" must indeed be set. Then there are three things:

As you can see there is still some issue with the solution XD

florianmueller commented 4 years ago

Great stuff! So theoretically that worked for me as well. Thanks! But somehow I only get a static and not up to date image as live view when taking a picture. Chrome mostly recognize the camera once access is allowed. Some strange observation I made with the live view performance: running the gphoto livefeed on a test website like webcamtests.com I get a super smooth framerate. Like something around 22 to 25 fps. But when using it on the application or on a motion server, the framerate drops to 9fps or lower. Probably the reason the preview as device cam looks like a single stand image thats not really updating.

I think for the interface, having the live view constantly running as a background image, like discussed in another issue here is the most user friendly way of taking your picture.

So as a workaround for now, I would use motion and enter the http stream as a background address in interfaces. But in order for it to work, would it be possible to change the script in core js to just start and stop the takeVideo.pho, but not utilizing it, so motion is free to access the feed from /dev/video0 everytime takeVideo.php starts/stops?

Regarding the bmc2835 blocking gphoto, I could not test it as I only have a DSLR connected and the camera port deactivated.

Thanks a lot for your support and great work with this photobooth application :)

andi34 commented 4 years ago

I think for the interface, having the live view constantly running as a background image, like discussed in another issue here is the most user friendly way of taking your picture.

https://github.com/andi34/photobooth/pull/58/files

That would be a way to use a stream as background. (Please note that my personal fork is quite ahead. some changes and commits might need to be adjusted to work here, e.g. for L10N translation library to work, but most I've added here too https://github.com/andi34/photobooth/tree/ipad2 )

couz74 commented 4 years ago

After rebasing on your repo @andi34, I also thought about letting the stream on the background. The only issue for me is that after a test my camera battery only last 1h in this situation. I will try to do so with another camera.

andi34 commented 4 years ago

Well, in such case you shouldn't use a battery but instead a power adapter.

Also preview by device cam should still be possible optional.

I'd so add an option for gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video0 in case user like to adjust the command via admin panel. Maybe:

$config['gphoto_preview'] = true; // true/false
$config['gphoto_preview']['cmd'] = "gphoto2 --stdout --capture-movie | ffmpeg -i - -vcodec rawvideo -pix_fmt yuv420p -threads 0 -f v4l2 /dev/video0";
andi34 commented 4 years ago

For a Quick test of the camera: https://www.aaronbenjamin.design/camera-app/part-2 (Source: https://github.com/abenjamin765/camera-app )

andi34 commented 4 years ago

@florianmueller @couz74 https://github.com/andi34/photobooth/issues/83 maybe you like to test that? Currently only have a cam only without the possibility to test gphoto video.

andi34 commented 1 year ago

Improved implementation in Photobooth v4

https://photoboothproject.github.io/Changelog