Struggling with polling for new files

[connyg]

I might be on the wrong track with this. But I am trying to poll for new files and then copy all over to a Raspberry Pi. And it seems that a loop that would continuously call "list_camera_files" (as in the copyfiles example) would stay at the same number of files, it does not see the new files. I found in the orignal C library that for this there exists a call "filesystem_reset", but I don't know how to call it from Python, I don't seem to have a "filesys" variable that the call requires.

I was also trying to cover my use case by waiting for a "file added" event, but ran into various issues with that. Filenames are not the IMG_xxxx ones that I want to have (the original camera file names), it seems that the camera creates a capture_xxxx first and then renames them to the final name. And if I wait for the event and do a file copy at that stage then I seem to intercept that file. That's why I rather want to wait for the finished filename by polling the directory.

Or am I thinking wrong?

[connyg]

With this code:

    try:
        while True:
            event_type, data = gp.check_result(gp.gp_camera_wait_for_event(camera, 1000)) #,
gp.GP_EVENT_FILE_ADDED))
            if event_type == gp.GP_EVENT_FILE_ADDED:
                print(data.folder, "/", data.name)
                copy_files(camera)
                #copy_file(camera, data.folder, data.name)
            time.sleep(0.1)

    except KeyboardInterrupt:
        print("end")
        pass

    gp.check_result(gp.gp_camera_exit(camera))

I am getting this result:

/ / capt0000.jpg Getting list of files from camera... Copying files... /capt0000.jpg -> /mnt/diskstation/Lumix/Canon/2021_04_08/ WARNING: gphoto2: (gp_context_error) You need to specify a folder starting with /store_xxxxxxxxx/ WARNING: gphoto2: (gp_camera_file_read [gphoto2-camera.c:1735]) 'gp_filesystem_read_file (camera->fs, folder, file, type, offset, buf, size, context)' failed: -1 Traceback (most recent call last): File "ImgPickup.py", line 170, in sys.exit(main()) File "ImgPickup.py", line 158, in main copy_files(camera) File "ImgPickup.py", line 110, in copy_files offset, view[0:chunk_size])) File "/usr/local/lib/python3.7/dist-packages/gphoto2/result.py", line 111, in check_result raise gphoto2.GPhoto2Error(error) gphoto2.GPhoto2Error: [-1] Unspecified error

So the file has no directory associated and cannot be copied. Is that also a cache issue, that GPHOTO2 stores the wrong folder reference?

[jim-easterbrook]

You haven't shown what copy_files does. The gp.GP_EVENT_FILE_ADDED function returns the folder and file name of the new file. This is what you have to copy. It does not get renamed later by the camera.

I think names like capt0000.jpg mean you are capturing to RAM rather than SD card, which is also why there isn't a folder.

[connyg]

Hi Jim,

thanks for answering!

Mmh, ok, I will double check this another time today. The cooy_files is the identical function as in the examples.

And re my other question: can I do a filesystem_reset to see new files? Or is it the Fs not intended to allow polling?

Regards, Konrad

Am 09.04.2021 um 08:10 schrieb Jim Easterbrook @.***>:



You haven't shown what copy_files does. The gp.GP_EVENT_FILE_ADDED function returns the folder and file name of the new file. This is what you have to copy. It does not get renamed later by the camera.

I think names like capt0000.jpg mean you are capturing to RAM rather than SD card, which is also why there isn't a folder.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jim-easterbrook/python-gphoto2/issues/118#issuecomment-816435493, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABL2YVGCK6QHCEBUYLLNGYLTH2K3NANCNFSM42TZW45A.

[jim-easterbrook]

OK, I understand. I had not considered using copy_files to copy new files as they are added. I have no idea if a filesystem_reset would help. If you are currently capturing to RAM rather than SD card then the file system contents won't change anyway.

[connyg]

Not capturing to RAM. I am taking the picture on the camera, by pressing the there. And on my Raspberry Pi I am waiting for the picture to copy the picture immediately to a NAS for review on a screen or Macbook. If I keep looping with copy_files the new file is not seen, as the file system is cached by the C code of libgphoto2. So I would need to reset the cache.

And if I work with waiting for the file added event I get this other file name that seems to be a temporary file on the Canon camera. It seemed to me that the camera saves the file to this and renames it later. And if I incept the event then the file does not seem to be persisted after I copy it. But I would prefer to also have it on SD as usual. So I like the polling method a bit better.

Am 09.04.2021 um 08:42 schrieb Jim Easterbrook @.***>:



OK, I understand. I had not considered using copy_files to copy new files as they are added. I have no idea if a filesystem_reset would help. If you are currently capturing to RAM rather than SD card then the file system contents won't change anyway.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://github.com/jim-easterbrook/python-gphoto2/issues/118#issuecomment-816450998, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ABL2YVCBCFZGVP6B5MFMJSTTH2OVXANCNFSM42TZW45A.

[jim-easterbrook]

I've only ever seen picture names like capt0000.jpg when capturing to RAM, but different cameras do different things. What happens if you try to copy the capt0000.jpg file as soon as your loop detects it?

[connyg]

Some experiments ahead: I tried it in C code in the same manner.

Checked out the examples first (tether and capture). And with the C capture example it would capture two times, one time to memory and one time to file. And it produces the same captxxxx filename every time, without a folder name - the C lib reports it the same way. So with tether (waiting for file event) and capture the camera uses the captxxx filename. So that's issue #1 - the captxxx name. It's behavior of the camera.

The polling of the filesystem every 10 seconds, using sample-afl.c as a basis. It would also not see new captures done with the camera in the next filesystem listing. Then added gp_filesystem_reset and ... surprise ... it would also not see the new files / captures!! That means for issue #2: given the filesystem reset actually works (and I don't have reason to believe it would not) it's possible a behavior of the camera that it does now show new files in the same connection!?

So, bottom line: for my use case I will need to go the tether route and work with the captxxxx filename and take the image directly from the camera upon capture event. I was hoping to avoid having to take care of non-duplicate filenames and just use the names off the camera, but that doesn't seem possible.

[jim-easterbrook]

I think you should raise this on one of the gphoto mailing lists and / or file a bug with libgphoto2. As you say, it's not expected behaviour so either we both misunderstand what's supposed to happen or there's a bug in libgphoto2.