jim-easterbrook / python-gphoto2

Python interface to libgphoto2
GNU Lesser General Public License v3.0
359 stars 59 forks source link

waiting for variables to update, and wait_for_event causing picture to be taken? #75

Closed sdaau closed 5 years ago

sdaau commented 5 years ago

I'm not sure if this is an issue with python-gphoto2 per se, but it is first where I've noticed it, so I thought, it wouldn't harm asking here first.

I've been testing python-gphoto2 on Ubuntu 18.04 with a Canon S3 IS:

$ gphoto2 --version
gphoto2 2.5.15

Copyright (c) 2000-2017 Lutz Mueller and others

gphoto2 comes with NO WARRANTY, to the extent permitted by law. You may
redistribute copies of gphoto2 under the terms of the GNU General Public
License. For more information about these matters, see the files named COPYING.

This version of gphoto2 is using the following software versions and options:
gphoto2         2.5.15         gcc, popt(m), exif, cdk, aa, jpeg, readline
libgphoto2      2.5.16         all camlibs, gcc, ltdl, EXIF
libgphoto2_port 0.12.0         iolibs: disk ptpip serial usb1 usbdiskdirect usbscsi, gcc, ltdl, USB, serial without locking

$ pip show gphoto2
Name: gphoto2
Version: 1.9.0
Summary: Python interface to libgphoto2
Home-page: https://github.com/jim-easterbrook/python-gphoto2
Author: Jim Easterbrook
Author-email: jim@jim-easterbrook.me.uk
License: GNU GPL
Location: /usr/local/lib/python2.7/dist-packages
Requires: 

On this camera,

For other properties, I'm usually able to just set them to a different value, and then can read them back and confirm the new value is set. However, if I change shootingmode, I have to wait at least 2 seconds before I can see exposurecompensation (or rather, its read-only property) updated.

To work around this, I tried using wait_for_event, in the hopes that I'd get an event when exposurecompensation has been changed in response to the change of shootingmode, so I don't have to time.sleep for an arbitrary value. However, this didn't end up working either, because for some reason, camera.wait_for_event causes a picture to be taken - and if I had set up a exposure duration of say 8 seconds, it needs to wait for all those 8 seconds in order to complete!

So I have in essence 2 questions:

Here is the script, test-sm-ec.py:

#!/usr/bin/env python

import gphoto2 as gp
import time
SLEEPTIME=2 # at least 2 sec for correct response
ENDWITHAUTO=0 # 1 to end with Auto mode, 0 otherwise

camera = gp.Camera()
ctx = gp.Context()

camera.init(ctx)
camera_config = camera.get_config()

print("{} Start settings (sleeptime={}, endwithauto={}):".format(time.time(), SLEEPTIME, ENDWITHAUTO))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
if OK >= gp.GP_OK:
  print("  shootingmode = {}".format( gpprop.get_value() ))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "exposurecompensation" )
if OK >= gp.GP_OK:
  print("  exposurecompensation ro = {}".format( gpprop.get_readonly() ))

print("{} set shootingmode Auto".format(time.time()))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
if OK >= gp.GP_OK:
  gpprop.set_value( "Auto" )
  camera.set_config(camera_config) # must have this, else value is not effectuated!
time.sleep(SLEEPTIME)
camera_config = camera.get_config() # must have this, else effectuated values are not read back in
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "exposurecompensation" )
if OK >= gp.GP_OK:
  print("{} exposurecompensation ro = {}".format( time.time(), gpprop.get_readonly() ))

print("{} set shootingmode Manual".format(time.time()))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
if OK >= gp.GP_OK:
  gpprop.set_value( "Manual" )
  camera.set_config(camera_config) # must have this, else value is not effectuated!
time.sleep(SLEEPTIME)
camera_config = camera.get_config() # must have this, else effectuated values are not read back in
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "exposurecompensation" )
if OK >= gp.GP_OK:
  print("{} exposurecompensation ro = {}".format( time.time(), gpprop.get_readonly() ))

print("{} set shootingmode Auto (wait_for_event takes photo here?!)".format(time.time()))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
if OK >= gp.GP_OK:
  gpprop.set_value( "Auto" )
  camera.set_config(camera_config) # must have this, else value is not effectuated!
print(camera.wait_for_event(1000, ctx)) # shoots/takes a picture?! and may take longer than 1s to wait for shot to complete
camera_config = camera.get_config() # must have this, else effectuated values are not read back in
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "exposurecompensation" )
if OK >= gp.GP_OK:
  print("{} exposurecompensation ro = {}".format( time.time(), gpprop.get_readonly() ))

print("{} set shootingmode Manual (wait_for_event takes photo here?!)".format(time.time()))
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
if OK >= gp.GP_OK:
  gpprop.set_value( "Manual" )
  camera.set_config(camera_config) # must have this, else value is not effectuated!
print(camera.wait_for_event(1000, ctx)) # shoots/takes a picture?! and may take longer than 1s to wait for shot to complete
camera_config = camera.get_config() # must have this, else effectuated values are not read back in
OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "exposurecompensation" )
if OK >= gp.GP_OK:
  print("{} exposurecompensation ro = {}".format( time.time(), gpprop.get_readonly() ))

if ENDWITHAUTO == 1:
  print("{} set shootingmode Auto".format(time.time()))
  OK, gpprop = gp.gp_widget_get_child_by_name( camera_config, "shootingmode" )
  if OK >= gp.GP_OK:
    gpprop.set_value( "Auto" )
    camera.set_config(camera_config) # must have this, else value is not effectuated!

Here are the results of few invocations of the script - first, as posted:

$ python test-sm-ec.py 
1548334705.32 Start settings (sleeptime=2, endwithauto=0):
  shootingmode = Manual
  exposurecompensation ro = 1
1548334705.32 set shootingmode Auto
1548334707.95 exposurecompensation ro = 0
1548334707.95 set shootingmode Manual
1548334710.58 exposurecompensation ro = 1
1548334710.58 set shootingmode Auto (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fee3f9196c0>]
1548334716.97 exposurecompensation ro = 0
1548334716.97 set shootingmode Manual (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fee3f919650>]
1548334736.5 exposurecompensation ro = 1

As is, with SLEEPTIME = 1 - note the read-only property of the exposurecompensation property is not returned correctly after a sleep(1), but is correctly returned after wait_for_event, which however takes a picture (and may take even longer to wait out):

1548334942.83 Start settings (sleeptime=1, endwithauto=0):
  shootingmode = Manual
  exposurecompensation ro = 1
1548334942.83 set shootingmode Auto
1548334943.97 exposurecompensation ro = 1
1548334943.97 set shootingmode Manual
1548334945.59 exposurecompensation ro = 1
1548334945.59 set shootingmode Auto (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fa2013536c0>]
1548334952.02 exposurecompensation ro = 0
1548334952.02 set shootingmode Manual (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fa201353650>]
1548334971.46 exposurecompensation ro = 1

Here with SLEEPTIME = 2 and ENDWITHAUTO=1 (after calling the script once previously to set up an end Auto mode):

1548335175.04 Start settings (sleeptime=2, endwithauto=1):
  shootingmode = Auto
  exposurecompensation ro = 0
1548335175.04 set shootingmode Auto
1548335177.57 exposurecompensation ro = 0
1548335177.57 set shootingmode Manual
1548335180.2 exposurecompensation ro = 1
1548335180.2 set shootingmode Auto (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7f194e6036c0>]
1548335186.71 exposurecompensation ro = 0
1548335186.71 set shootingmode Manual (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7f194e603650>]
1548335206.35 exposurecompensation ro = 1
1548335206.35 set shootingmode Auto

Same, with SLEEPTIME = 1 - again, the read-only property of the exposurecompensation property is not returned correctly after a sleep(1), but is correctly returned after wait_for_event, which however takes a picture (and may take even longer to wait out):

1548335254.15 Start settings (sleeptime=1, endwithauto=1):
  shootingmode = Auto
  exposurecompensation ro = 0
1548335254.15 set shootingmode Auto
1548335255.55 exposurecompensation ro = 0
1548335255.55 set shootingmode Manual
1548335256.84 exposurecompensation ro = 0
1548335256.84 set shootingmode Auto (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fb6233886c0>]
1548335263.54 exposurecompensation ro = 0
1548335263.54 set shootingmode Manual (wait_for_event takes photo here?!)
[2, <Swig Object of type 'CameraFilePath *' at 0x7fb623388650>]
1548335282.96 exposurecompensation ro = 1
1548335282.96 set shootingmode Auto
jim-easterbrook commented 5 years ago

You're right that this problem is unlikely to be caused by python-gphoto2. I've not heard of wait_for_event causing a picture to be taken before. With the cameras I've used I have to call wait_for_event repeatedly until I get the event I'm interested in or a GP_EVENT_TIMEOUT event. Have a look at the time_lapse.py example for a typical usage. (This example also includes some configuration manipulation.)

sdaau commented 5 years ago

Many thanks for the response @jim-easterbrook :

You're right that this problem is unlikely to be caused by python-gphoto2.

Indeed, so I posted https://github.com/gphoto/libgphoto2/issues/358

I've not heard of wait_for_event causing a picture to be taken before

Neither was I able to find anything about that browsing around on the Net; still, I repeatedly experience this... maybe because it (Canon S3 IS) is an old camera

Have a look at the time_lapse.py example for a typical usage. (This example also includes some configuration manipulation.)

Thanks - while this didn't fix the issue with wait_for_event taking a picture, it did force me to rewrite, and revealed how I can wait for exposurecompensation.readonly to change after a change of shootingmode - and that is with the good old sleep polling check; here is test-sm-ec2.py:

from __future__ import print_function

from contextlib import contextmanager
import os
import subprocess
import sys
import time

import gphoto2 as gp

@contextmanager
def configured_camera():
    # initialise camera
    camera = gp.Camera()
    camera.init()
    # if needed:
    #gp.gp_camera_capture_preview(camera) # extend lens (raise mirror?); else nothing quite works on Canon S3 IS; actually without it, program will fail with "GPhoto2Error [-2] Bad parameters"
    try:
        # adjust camera configuratiuon
        cfg = camera.get_config()
        capturetarget_cfg = cfg.get_child_by_name('capturetarget')
        capturetarget = capturetarget_cfg.get_value()
        capturetarget_cfg.set_value('Internal RAM')
        imageformat_cfg = cfg.get_child_by_name('imageformat')
        imageformat = imageformat_cfg.get_value()
        imageformat_cfg.set_value('Unknown value 0001') # Canon S3 IS has no 'Small Fine JPEG'
        camera.set_config(cfg)
        # use camera
        yield camera
    except Exception as ex:
      print(type(ex).__name__, ex)
    finally:
        # reset configuration
        capturetarget_cfg.set_value(capturetarget)
        imageformat_cfg.set_value(imageformat)
        camera.set_config(cfg)
        # free camera
        camera.exit()

def empty_event_queue(camera):
    while True:
        type_, data = camera.wait_for_event(10)
        #print(type_, data)
        if type_ == gp.GP_EVENT_TIMEOUT:
            return
        if type_ == gp.GP_EVENT_FILE_ADDED:
            # get a second image if camera is set to raw + jpeg
            print('Unexpected new file', data.folder + data.name)

def main():
    global shootingmode_cfg
    with configured_camera() as camera:
        cfg = camera.get_config()

        shootingmode_cfg = cfg.get_child_by_name('shootingmode')
        shootingmode = shootingmode_cfg.get_value()
        exposurecompensation_cfg = cfg.get_child_by_name('exposurecompensation')
        exposurecompensation = exposurecompensation_cfg.get_value()
        print("{} {}: shootingmode '{}', exposurecompensation ro={}".format(time.time(), "Start", shootingmode, exposurecompensation_cfg.get_readonly()))

        shootingmode_cfg.set_value("Manual")
        camera.set_config(cfg)

        cfg = camera.get_config()

        shootingmode_cfg = cfg.get_child_by_name('shootingmode')
        shootingmode = shootingmode_cfg.get_value()
        exposurecompensation_cfg = cfg.get_child_by_name('exposurecompensation')
        exposurecompensation = exposurecompensation_cfg.get_value()
        oldecro = exposurecompensation_cfg.get_readonly()
        man01ts = time.time()
        print("{} {}: shootingmode '{}', exposurecompensation ro={}".format(man01ts, "After Manual 01", shootingmode, oldecro))

        shootingmode_cfg.set_value("Auto")
        camera.set_config(cfg)

        # nothing - does not in itself get a fresh readonly value:
        #cfg = camera.get_config()
        #camera.set_config(cfg)

        cfg = camera.get_config()
        child_count = cfg.count_children()
        testarr = []
        for child in cfg.get_children():
          child_type = child.get_type()
          testarr.append("{} {}".format(child_type, child.get_name()))

        #~ empty_event_queue(camera) # takes picture w/ Canon S3 IS (via wait_for_event)

        cfg = camera.get_config()

        shootingmode_cfg = cfg.get_child_by_name('shootingmode')
        shootingmode = shootingmode_cfg.get_value()
        exposurecompensation_cfg = cfg.get_child_by_name('exposurecompensation')
        exposurecompensation = exposurecompensation_cfg.get_value()
        ecro = exposurecompensation_cfg.get_readonly()
        while (oldecro == ecro):
          time.sleep(0.01)
          cfg = camera.get_config() # ecro will not change value without this
          exposurecompensation_cfg = cfg.get_child_by_name('exposurecompensation')
          ecro = exposurecompensation_cfg.get_readonly()
          print(ecro)
        aut01ts = time.time()
        print("{} {}: shootingmode '{}', exposurecompensation ro={} (delta={:.3f})".format(aut01ts, "After Auto 01", shootingmode, exposurecompensation_cfg.get_readonly(), aut01ts-man01ts))

    return 0

if __name__ == "__main__":
    sys.exit(main())

This is what it outputs:

$ python2 test-sm-ec2.py
1548364594.27 Start: shootingmode 'Auto', exposurecompensation ro=0
1548364596.54 After Manual 01: shootingmode 'Manual', exposurecompensation ro=1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
0
1548364598.68 After Auto 01: shootingmode 'Auto', exposurecompensation ro=0 (delta=2.139)

I've seen the delta go as low as 1.188 sec, but usually it tends to be around 2 sec. EDIT: Also, going from "Auto" to "Manual" shootingmode is fine (no delay/wait needed to see exposurecompensation ro change), its going from "Manual" to "Auto" that is a problem (?!)

Since with this I can get to the change of exposurecompensation.readonly without using wait_for_event - I do not need to worry about wait_for_event taking a picture in this case, though it would be great to solve that too.

So I guess this solves my issue on the python side of things - though I'd love to hear if there are any ideas, so as to why do I experience this (they couldn't reproduce the issue with a different camera and my script over at https://github.com/gphoto/libgphoto2/issues/358)