multiple calls to Camera.capture() give "I/O in progress", or the same file

[AntiSol]

Hi.

I'm seeing some weirdness when I try to capture more than one image in a session.

Firstly, I don't see any way other than doing Camera.exit() to tell gphoto2 that I'm done with I/O. If I don't call this, I get an "I/O in progress" error if I call capture a second time.

If I DO call exit() after saving a captured file, subsequent calls to capture() return the same file, even though the camera is capturing a second file (I can tell this because if I call capture about 5 times I get a "storage full" error).

I have looked at a couple of C implementations (notably entangle) and it does not seem to call exit() after capturing an image. So what is the best way to tell gphoto2 that I've finished doing I/O?

If I 'del c' and reinitialize a new camera object this seems to work more reliably, but I still only ever get back one file name from c.capture()

Please see the below test/example code and comments. I'd appreciate any input as to what I'm doing wrong and what I should be doing instead. Thanks! :)

I am using python 2.7, libgphoto2 2.5.9, and libgphoto2_port 0.12.0 (the versions packaged with ubuntu 16.04). My camera is a Nikon D3200.

import gphoto2 as gp
import os

c = gp.Camera()
c.init()

#capture an image:
i = c.capture(gp.GP_CAPTURE_IMAGE)
#get the captured file:
f = c.file_get(i.folder,i.name,gp.GP_FILE_TYPE_NORMAL)
#save to disk:
dest = os.path.join("/tmp",i.name)
f.save(dest)
c.file_delete(i.folder,i.name)

print("saved to '%s'" % dest)

#image saved. good. but what now?

#capture another image:
try:
    #This causes gphoto2.GPhoto2Error: [-110] I/O in progress
    i2 = c.capture(gp.GP_CAPTURE_IMAGE)
except Exception as ex:
    print("Can't capture second image: %s" % ex)

#OK, so that caused an error. Another approach:
c.exit()

#if we don't do init here we get a nasty crash when we call capture() again:
# *** Error in `python': double free or corruption (fasttop): 0x0000000000c7af10 ***
# Aborted (core dumped)
c.init()

i2 = c.capture(gp.GP_CAPTURE_IMAGE)

f2 = c.file_get(i2.folder,i2.name,gp.GP_FILE_TYPE_NORMAL)
dest2 = os.path.join("/tmp","take2-" + i2.name)
f.save(dest2)
# we've saved a second file, but 
#/tmp/{i.name} and /tmp/take2-{i2.name} are the same file:
# the file info returned by the second call to capture()
# is the same as the first.
# on my system: diff /tmp/capt0000.jpg /tmp/take2-capt0000.jpg gives no output
# (identical). 
# 
# Note that this is after supposedly deleting the file back on line 14!
#
# meanwhile, another file was written into memory on the camera - 
# if you put this second capture in a loop you'll get 'storage full' 
# after a few iterations (assuming you don't get a core dump)

print("saved to '%s'" % dest2)

#Here, I usually get:
# '*** Error in `python': double free or corruption (fasttop): 0x0000000000d20a10 ***'
# but sometimes I get a segfault
c.exit()

[jim-easterbrook]

Firstly, c.exit() will leave the camera in an undefined state until you call c.init() again, so a crash is unsurprising. I ought to find a way to make it fail more gracefully though.

The multiple capture problem looks a bit like issue #51. Does the file name change if you don't delete the first one?

gphoto2 2.5.9 is nearly three years old. The current version is 2.5.19. You really should upgrade if you can.

[AntiSol]

Thanks for the response.

So c.exit() and then reinitialising is the best way to tell the library I'm done with I/O? I've also tried putting in a 'def f' and 'del i' in there but this doesn't make any difference. Interestingly, as far as I can tell, this doesn't seem to be what entangle is doing if you look at entangle_camera_capture_image in https://gitlab.com/entangle/entangle/blob/master/src/backend/entangle-camera.c

No, the name doesn't change if I don't delete the file, I just added the call to file_delete as it illustrates that something very weird is going on.

I'm currently trying to compile the latest version of libgphoto2 to see if it behaves any better. Are you able to call capture() multiple times with whatever version/camera you have?

[jim-easterbrook]

I've not tried multiple captures, I generally don't use capture. You shouldn't need to call c.exit() at all, unless you want to release the camera so another application can use it. If you haven't already done so, it's worth browsing the libgphoto2 api. http://www.gphoto.org/doc/api/gphoto2-camera_8h.html#a6f19c4ea385641fb972e779badf48ae1

Looking at the change log I can see two possibly related fixes: 2.5.10: Nikon: fixed problem with SDRAM capture 2.5.13: Various bugfixes in Nikon, Canon capture http://gphoto.org/news/

[AntiSol]

I'd really appreciate if you could take the time to try out my test code and confirm whether others have the same problem :)

OK, but if I shouldn't have to call exit then how do I tell it that I'm done with I/O to prevent the "I/O in progress" error?

I've been using the api docs as reference. I can't find any indication that I need to do anything after saving the file to tell the library I'm done with I/O...?

Still trying to compile the latest version. dependency hell.

[jim-easterbrook]

I don't have the same make or model of camera. I've no idea what's causing your "I/O in progress" errors. Reinitialising the camera is a likely workaround for bugs in libgphoto2, but shouldn't be needed.

I don't have time today but will try to look further into this tomorrow. I'm mostly concerned about your double free or corruption errors, as I'd like the Python interface to be more rugged.

[AntiSol]

If you tried with a different model of camera that would at least indicate whether the issue is specific to my camera driver or not.

I've tried on another machine which has gphoto2 2.5.16 and I get pretty much the same results - I/O error on the second capture and the same file name. It doesn't segfault and core dump quite as often though.

I can't figure out how to install a newer version on my primary machine without it uninstalling every single program that depends on libgphoto2-port12 (Which is everything useful and camera-related, and some others, including wine!)

I also can't figure out how other programs like entangle are doing multiple captures successfully. As far as I can tell entangle isn't doing anything substantially different to what I'm trying, which makes me think this is specific to the python bindings.

[jim-easterbrook]

I've tried this with my Canon 100D, setting the capture target to RAM first, as I think that's what you're using. This hangs up the camera - it displays "recording" on it's screen and won't switch off. The only way out is to pull the battery.

If I do it capturing to memory card there is (obviously) a longer gap between captures. If I comment out the c.file_delete command it all works OK, once I'd corrected the second file save so it saves the second file f2 instead of the first one f.

Having the c.file_delete causes I/O errors sometimes, but I've not been able to do it reliably. This is all with or without the camera reinitialisation.

(libgphoto2 version 2.5.18, python-gphoto2 1.8.3)

[AntiSol]

thanks for taking the time to look into it :)

So you're not hitting the except part of the try block? you're not getting the IO error?

ha, doing f.save instead of f2.save will cause issues, can't believe I didn't spot that typo. But unfortunately f2.save still gives me an identical file.

Seems like the library isn't particularly solid. I wonder how other programs are doing it. I'll keep digging and see what I can find.

I managed to compile from git and get the latest version installed. I don't see any nasty segfaults or aborts anymore, but the behaviour is the same.

I also might have a viable workaround - call trigger_capture and use wait_for_event to handle the files being added. But this has another weird bug where shooting in raw+jpeg gives me capt0000.jpg and capt0001.nef for the same photo. grrrr.

Still, I doubt it's an issue in the python implementation, the gphoto mailing list or whatever they use is probably a better place for me to ask questions.

Again, thanks for taking the time to look into it for me, I appreciate it.

You can close this if you want, or if you prefer I can come back with anything I discover.

Cheers

[jim-easterbrook]

I commented out the try / except and go straight to the second capture. Your script as written will do 3 captures if you don't have an I/O error. And I mostly don't have I/O errors.

[AntiSol]

ok. thanks.

[jim-easterbrook]

When I do get them, it's in the file_delete command, not the following capture, so probably not your problem. I'm also shooting raw+jpeg - I think this might confuse libgphoto2 as each capture has two names. My camera doesn't much like the script deleting just one of the files.

[AntiSol]

Ooh, Good suggestion, I'll try switching to just raw or just jpg and see if that makes a difference. Thanks!

[AntiSol]

eureka!

it turns out this is indeed related to shooting in raw+jpg. Setting it to just shoot in one or the other stops this from happening.

after looking at the gphoto2 source I was able to see what it's doing. it turns out that you need to clear out the event queue on the camera after each capture. It doesn't matter if you don't actually download the file referenced by the 'file added' event, you just need to process the event. The following code works just fine. Note the absence of exit() and init()

Thanks again for your help!

import gphoto2 as gp
import os

c = gp.Camera()
c.init()

def event_text(event_type):
    if event_type == gp.GP_EVENT_CAPTURE_COMPLETE: return "Capture Complete"
    elif event_type == gp.GP_EVENT_FILE_ADDED: return "File Added"
    elif event_type == gp.GP_EVENT_FOLDER_ADDED: return "Folder Added"
    elif event_type == gp.GP_EVENT_TIMEOUT: return "Timeout"
    else: return "Unknown Event"

for i in range(1,10):
    print("Capture #%1d" % i)

    i2 = c.capture(gp.GP_CAPTURE_IMAGE)

    f2 = c.file_get(i2.folder,i2.name,gp.GP_FILE_TYPE_NORMAL)
    dest2 = os.path.join("/tmp",i2.name)

    f2.save(dest2)
    print("saved to '%s'" % dest2)

    #empty the event queue
    typ,data = c.wait_for_event(200)
    while typ != gp.GP_EVENT_TIMEOUT:

        print("Event: %s, data: %s" % (event_text(typ),data))

        if typ == gp.GP_EVENT_FILE_ADDED:
            fn = os.path.join(data.folder,data.name)
            print("New file: %s" % fn)
            #self.download_file(fn)

        #try to grab another event
        typ,data = c.wait_for_event(1)

c.exit()

[jim-easterbrook]

Well done! That bit about clearing the event queue could be the answer to quite a few problems, so I'll try to remember it.

PS An alternative to your event_text function which I think is tidier would be:

event_texts = {
    gp.GP_EVENT_CAPTURE_COMPLETE: "Capture Complete",
        gp.GP_EVENT_FILE_ADDED: "File Added",
    gp.GP_EVENT_FOLDER_ADDED: "Folder Added",
    gp.GP_EVENT_TIMEOUT: "Timeout",
    }

def event_text(event_type):
    if event_type in event_texts:
        return event_texts[event_type]
    return "Unknown Event"

[AntiSol]

yeah I suspect it will sort out a couple of other weird things that I'm currently doing exit and init to work around.

yeah that is a much nicer pattern. Thanks.

[AntiSol]

Sorry to bug you again. This is not really the same issue but I didn't want to open another one. Any thoughts on how to end preview mode?

I can call something like "previewimg = camera.capture_preview()" repeatedly to get a live preview no problem, but I can't seem to find any way to get the camera mirror to go back down other than calling exit, which isn't ideal because doing that resets the capture file names back to capt0000.jpg, meaning subsequent captures will overwrite anything I captured before previewing.

I've been searching around and I can't seem to figure out how others are doing it. the gphoto2 source isn't helpful because it just calls exit when it's done. the libgphoto2 api docs don't mention anything like this in the gp_camera_capture_preview docs. Emptying the event queue doesn't help, nor does doing "del previewimg"

If you don't know that's cool and I'll keep searching, I just thought you might have an idea.

[jim-easterbrook]

No idea I'm afraid. You're not the first person to ask - see #53.

[AntiSol]

ha. thanks anyway

[jim-easterbrook]

This discussion has prompted me to write a simple time lapse script, committed in bc686bd.

[jim-easterbrook]

PS I've just discovered that on my Canon SLR I can switch the camera to & from live view with the settings->output configuration item, setting it to TFT or Off. I don't know if this is any help with switching out of preview mode.

[AntiSol]

I didn't have time to try running your timelapse script but I read through it and it looks like it should work.

WRT exiting preview mode, I figured this out ages ago. I must have forgotten to comment. On my Nikon you set the 'viewfinder' config item to 0 to exit live view / preview mode.

But I appreciate the info about your Canon, I'll add it into my project so that it supports live preview for both Nikon and Canon.

[StyXman]

I'm getting a similar error, and I implemented the event queue emptying, but I sill get the same. Should I open a separate issue?

[jim-easterbrook]

Strictly speaking it's only an issue with python-gphoto2 if it doesn't happen with an equivalent C program. This is really about how to use the libgphoto2 library, and I know no more than any other libgphoto2 user.

[StyXman]

I got to work around it by exit()ing the camera and creating+init()ing a new one.