search

kristofg / rifec

Receive Images From Eye-Fi Cards
GNU General Public License v2.0
53 stars 16 forks source link

multiple file transfer #5

Open planet127 opened 11 years ago

planet127 commented 11 years ago

With every new foto all existing fotos of the cards are retransfered. Setting "upsyncallowed" to "true" at line 1229 helps to avoid this

kristofg commented 11 years ago

That is weird, I have never seen any of my cards behave like that. They seem to keep track of upload status per file, so that they don't upload the same file twice. I do get duplicates and triplicates on occasion, but they seem to be caused by upload transactions not completing properly.

It seems like you've found a solution, but I really would like to understand a bit more about what is happening here before changing the default value.

Would it be possible to get a card log and rifec.log with LogLevel=trace taken just after this has happended? (Note: You may want to edit out your upload key from the logs before sharing it.)

Does the card do this all the time, ie. also after it has just been reformatted?

bulislaw commented 11 years ago

On Wed, May 8, 2013 at 9:54 PM, Kristoffer Gleditsch < notifications@github.com> wrote:

That is weird, I have never seen any of my cards behave like that. They seem to keep track of upload status per file, so that they don't upload the same file twice. I do get duplicates and triplicates on occasion, but they seem to be caused by upload transactions not completing properly.

It seems like you've found a solution, but I really would like to understand a bit more about what is happening here before changing the default value.

Would it be possible to get a card loghttp://forums.eye.fi/viewtopic.php?f=4&t=3602#p14910and rifec.log with LogLevel=trace taken just after this has happended? (Note: You may want to edit out your upload key from the logs before sharing it.)

Does the card do this all the time, ie. also after it has just been reformatted?

— Reply to this email directly or view it on GitHubhttps://github.com/kristofg/rifec/issues/5#issuecomment-17633129 .

I have the same problem on my media center (raspberry pi with raspbian) in the same time it works ok with my laptop.

Bartek Szatkowski

kristofg commented 11 years ago

I have the same problem on my media center (raspberry pi with raspbian) in the same time it works ok with my laptop.

OK, so several people are seeing this - that's interesting, thanks.

Unfortunately, since I'm not seeing this on any of my computers (Synology NAS and a laptop for occasional testing), I'm not likely to make any progress without logs.

seberler commented 11 years ago

Hello Kristoffer, I confirm the problem on a raspberry pi type B. Probably there is an timing issue on slower systems? The Pi isn't really fast. I would like to send you a rifec.log and a screen shot of the directory listing there the files are should stored. Could you provide an email-address for sending this?

kristofg commented 11 years ago

seberler: A log would be great! If at all possible, a card log of the same time span would be very helpful.

Perhaps you can create a Gist containing the log, and share the link to it? I would like to keep my Github-related communications on Github if possible.

seberler commented 11 years ago

Hello, following the link to the gist, containing the logs. I'm hoping, two files are included (I had some problems to upload/insert the logs, so that the gist would be updated). https://gist.github.com/seberler/9db14ec415c8e55e4a54 Thanks for Your quick reply

kristofg commented 11 years ago

Thanks! I've downloaded the files to my computer, and will take a look at them when I get a spare evening.

kosie commented 11 years ago

I'm having the same issue with a Raspberry Pi - Model B. Pictures are retransferred every time the eye-fi card is powered up. This does not happen using Eye-Fi Center with my Windows PC. Eye-Fi firmware is 5.2006 (happened with 5.0019 as well)

Although the transfer is successful at the server, the eye-fi card log is full of entries like

[xx:xx] Uploaded xx bytes at offset 0 for "/public/DCIM/100DOXIE/IMG_xxxx.JPG" as file ID xx (desktop delivery)... [xx:xx] Returning from upload function with error 200 ("/public/DCIM/100DOXIE/IMG_xxxx.JPG").

The eye-fi card seems not to recognize successfully transferred files.

kosie commented 11 years ago

Snippets of my logs are at https://gist.github.com/anonymous/6404363

kosie commented 11 years ago

Maybe this is indeed a timing issue? I wonder about a new connection (StartSession) from the eye-fi card while the original transfer is still not finished (calculation of integrity digest in progress).

kristofg commented 11 years ago

Ok, I've looked at the logs. I have to admit I am not a lot wiser.

@seberler: Unfortunately, all the failures in these logs seem to be caused by the retry-limit: When trying filename.1, filename.2, etc., the script will stop at .100. If there are more than 100 instances of the same file name, it will fail at once. There is no information here about why the first 99 transfers failed.

@kosie: I looked at your logs as well:

Looking at the receiver process with PID 7857, the upload begins at 13:00:33. The photo data was received by 13:00:53. The integrity digest was calculated by 13:01:00, and the UploadPhotoResponse message was sent 13:01:02. Assuming the file in question was uploaded again afterwards (it is a very short log snippet), less than half a minute seems far too little to cause a time out.

Thanks for the logs, people. I'll try to sit down and try to reproduce this locally some time, but I don't know when that'll be. If anyone has any ideas in the meantime... :)

seberler commented 11 years ago

Thanks for reply. I tested again with changes in rifec.config. Standard is SocketTimeout=600, values 100 or 2000 does not have any affect. I took a photo only with 640x480pixels, ca. 140 kByte... Now I set the value back to standard (=600). The cardlog what I posted has many entries "Returning from upload function with error 200". What does it mean? Today the the usb-drive, there the images should be stored, was filled with temporary files. If the usb-drive becomes full, there comes the error codes 357 and 360 one line after another. Edit: No, the USB-drive was not full. But I can't imagine, what it was. I changed from the 2GB-SD-Card (Raspbian Boot-"Drive") to an 8GB-SD-Card, but there was no change in this issue.

kristofg commented 11 years ago

What card models are you using, by the way?

seberler commented 10 years ago

At first, I used a 2GB-SD-Card (Raspbian Boot-"Drive"), class 2. Now I tried an 8GB-class-10-SD-Card, but there was no change in this file transfer issue. Also a change of the old 64-MB-USB-Stick against an 8GB-Kingston-USB2.0-Stick result in no change. Then I tried: I changed the USB-drive to a mounted NAS-Drive. Now, the pictures are stored (no fails in the Log), but the jpg-Files are unreadable.

kristofg commented 10 years ago

I'm sorry, I didn't phrase the question very well: What Eye-Fi card model are you using?

seberler commented 10 years ago

Hello, I'm using a Sandisk Eye-Fi-WiFi-Card . The Card is not marked X2, but it is very likely, that it will be equivalent to a X2. The firmware version is 5.0019. By the way, rifec works great with this eye-fi-card, running on ubuntu 10.04 with a normal desktop pc (AMD Sempron 3000+).

seberler commented 10 years ago

Hello, I tried this Perl-software, found at the link in your https://github.com/kristofg/rifec :

This software works well (as I can confirm in the some short tests). Very, very quickly and the "slow" Raspberry seems to be quick enough. Perhaps worth a view?

kristofg commented 10 years ago

@seberler it's very interesting that the other script works. Based on the original problem description and the logs, I don't think this is a timeout issue. I suspect some kind of protocol problem. I don't know when (or if) I'll get the time to sit down and look properly at this, though.

seberler commented 10 years ago

Hello, no problem at all. The code seems to be a kind of "quick and dirty", but it works at the base. I was intended to test different implementations of the eyefi-"Server". Just to sort out any hardware-issues or performance-problems.

kosie commented 10 years ago

@kristofg The other script works in my case too. There must indeed be some kind of protocol issue. Using wireshark I could not identify major differences in the communication. The only real difference I see is the Eye-Fi card connecting multiple times with RIFEC while the small perl script only accepts one connection at a time.

kristofg commented 10 years ago

After upgrading my cards to firmware 5.2010, I've started seeing this on my Synology NAS as well. I'll try to take a look at this if I get a quiet evening during Christmas, but no promises. If anyone has any bright ideas about the root cause for this behavior, I'm all ears.

kristofg commented 10 years ago

I just upgraded my Synology NAS to DSM 5.0, which upgraded Perl from 5.8.6 to 5.18.1. It may just be a fluke, but the upgrade seems to have helped: Old pictures are flowing from the camera to the NAS as I write this. I don't know what to make of that, but I guess upgrading Perl is worth a try.

My Eye-Fi cards are quite old now, and small and slow compared to new cards, so they are headed for the shelf. The Pro series cards are not sold in Europe at the moment, so it's back to old style SD cards for me. If anyone figures this bug out, I'm very interested in hearing about it, but I'm not going to spend any more time on it. Sorry, people. :)

seberler commented 10 years ago

Thanks for reply; I can understand your decision. Thanks a lot anyway for working on this problem; maybe an perl update, depending on the kind of base system, would be solve the problem.