Faster way to copy files from camera

[RodrigoCatto]

System OS: Ubuntu 22 Python 3.10.6 libgphoto2-2.5.30 python-gphoto2 2.3.4 camera: Canon EOS R

Your problem I'm trying to download all files from the camera to my computer. But, when I compare the time that it takes to download from the python interface vs CLI, I see the python interface takes longer.

I did this test using both the copy-files.py example and the CLI gphoto2 --get-all-files. Most of the comparisons I did, the interface took around 2x more. Here is one example: copy-files.py -> 108.97s gphoto2 --get-all-files -> 50.95s

Is there a fast way to download all the files from the camera using the python interface?

[jim-easterbrook]

The copy-files.py example does more than simply copying the files. It gets the timestamp of the file on the camera before copying the file to a directory created using the timestamp. (It's also a bit of a mess.)

If you write a Python script that simply copies all the files on the camera to one directory on the computer with no other messing about then I would expect it to take the same amount of time.

[RodrigoCatto]

Hi @jim-easterbrook

I tried to write a Python script that simply copies all the files on the camera to one directory, and removed the timestamp and checking if file is there already (since I'm just testing the time that it takes to download all images at once).

def main():

    locale.setlocale(locale.LC_ALL, '')
    logging.basicConfig(
        format='%(levelname)s: %(name)s: %(message)s', level=logging.WARNING)
    callback_obj = gp.check_result(gp.use_python_logging())
    camera = gp.check_result(gp.gp_camera_new())
    gp.check_result(gp.gp_camera_init(camera))
    print('Getting list of files from camera...')
    camera_files = list_camera_files(camera)
    if not camera_files:
        print('No files found')
        return 1
    print('Copying files...')
    t0 = time.time()
    for path in camera_files:
        folder, name = os.path.split(path)
        dest = os.path.join(PHOTO_DIR, name)
        camera_file = gp.check_result(gp.gp_camera_file_get(camera, folder, name, gp.GP_FILE_TYPE_NORMAL))
        camera_file.save(dest)

    gp.check_result(gp.gp_camera_exit(camera))
    t_eleapsed = time.time() - t0
    print("Eleapsed time: {:.2f}s".format(t_eleapsed))
    return 0

Even with this code, the time to save images comparing to the CLI gphoto2 --get-all-files is still around two times slower. I did some digging and found that this is the part taking more time to process (Almost 1s per JPEG photo on my Canon R): gp.check_result(gp.gp_camera_file_get(camera, folder, name, gp.GP_FILE_TYPE_NORMAL))

Do you think that there is a way to optimize my script more?

[jim-easterbrook]

I'm surprised it's that much slower. Your version of the script is probably as quick as it can be. I'd need to have a look at the source code of the gphoto2 command to see what it's doing differently.

It might be possible to do it quicker by having a separate thread to do the camera_file.save, but using threads usually creates more problems than it solves.

[jim-easterbrook]

I've just been having a look at how the gphoto2 command saves files (https://github.com/gphoto/gphoto2/blob/master/gphoto2/main.c#L575). It's all a bit confusing but I think the main difference is that it creates a CameraFile from a C file descriptor (using gp_file_new_from_fd) and passes that to gp_camera_file_get.

This isn't currently possible with python-gphoto2 as it calls gp_file_new in the gp_camera_file_get wrapper to create the CameraFile.

[jim-easterbrook]

In commit 14055c7 I allow passing a CameraFile to gp_camera_file_get. I tried copying files using a CameraFile created from a file descriptor but didn't get any speed improvement.

[RodrigoCatto]

Thanks for digging into it @jim-easterbrook. I'll do some testing here and send you the results, but it was still a good find.

[RodrigoCatto]

Closing this issue as we ended up moving towards other camera hardware. Thanks again @jim-easterbrook for taking the time to look into it. If I stumble across this again, I'll make sure to test it.