OpenArchive / Save-app-ios

Secure Mobile Media Preservation
https://open-archive.org/
GNU General Public License v3.0
11 stars 3 forks source link

Videos not uploading on version 28 testflight #127

Closed foundscapes closed 5 years ago

foundscapes commented 5 years ago

July 1, 2019 Natalie Test Flight Version 28 Iphone 6s/iOS 12.3.1

  1. On Wifi
  2. Opened application
  3. Added 5 videos to NCTest19
  4. They showed up in the queue 20mb - 450mb
  5. Clicked edit to order according to size
  6. None starting uploading / just hanging with the spinning circles - still 10 mins later, nothing was uploaded
  7. 30 mins later - Added new folder “NCTestJuly19” and added one small photo to it, still hanging in upload screen w/ others IMG_2712 IMG_2713
n8fr8 commented 5 years ago

@foundscapes One thing to try in the cases is to swipe the app away (to kill it), and open it again. See if that helps unstuck whatever is tsuck.

Second, I would try to upload each video by itself to see if a particular video is making it get stuck. If you upload a bunch, it is hard to debug which video might be causing the hang.

@tladesignz on that note, if there is an error accessing a particular video, we should not continue to retry uploading it. The retry should really only occur under network error conditions.

foundscapes commented 5 years ago

@n8fr8 I have closed, reopened, uninstalled, reinstalled, and then uploaded each file individually, which was successful, but there were discrepancies between file size on each device and how long they took to upload.

Then I uploaded the same files again in a batch, and they also worked, but this time, they sent at about the same rate.

If one video is causing a problem, then we need an error to let people know which and why.

foundscapes commented 5 years ago

Same problems as yesterday with videos, no problem with photos July 1st, now photos not uploading, even individually on July 2nd. Kept app open the entire time.

Closed/reopened app on ipad mini and iphone 6s and one photo was uploaded from each and videos seem to upload individually

Uploading indiv videos from first bulk upload, status:

  1. Guitar Wolf (21 seconds - 29.6mb iphone), did not upload individually on either device first time, after closing/reopening app, it uploaded fine on both (under 2 mins)
  2. Kitten in Tunis w/ Waleed (15 seconds - 20mb on iphone, 29.6 mb on ipad ), uploaded fine on both (under a minute)
  3. Tunis highway footage (26 seconds - on ipad mini it shows file was 21 mb and on iphone 6s it was 50.6 mb - same file!) uploaded fine on both (under 3 mins)
  4. Tunis cab ride in town (52 seconds, 100 mb on both) uploaded fine on both (under 6 mins)
  5. Doves 3:55 (12.5mb on ipad, 20 mb on iphone 6s?) iphone 10:57 - 11:12 (15 mins) // ipad 10:57 - 11:29 (32 mins)

Questions: why are same files different sizes on each device? Why do they take different amounts of time to upload (ex: doves file was smaller on ipad, but took 17 mins longer to upload than on iphone 6s)

Uploading all 5 files in bulk: both hung on last upload even though status showed it was done Iphone 6s: 11:18 - 12:03 (45 mins)

  1. Guitar wolf 29.8mb
  2. Kitten in tunis 29.6mb
  3. Tunis highway 50.6 mb
  4. Tunis cab ride in town 100mb
  5. Doves 454.6mb

Ipad Mini: 11:30 - 12:03 (33 mins)

  1. Guitar wolf 29.8mb
  2. Kitten in tunis 29.6mb
  3. Tunis highway 50.6 mb
  4. Tunis cab ride in town 100mb
  5. Doves 454.6mb
foundscapes commented 5 years ago

IMG_0026 File hanging after it says its uploaded, happened on both devices

foundscapes commented 5 years ago

When I closed app on both devices and reopened, the status circle was repopulating again, so unsure why it didn't finish upload.

tladesignz commented 5 years ago

I'll had an interesting experience: File was almost done (99%), then went back to 62%: (see progress value).

[BackgroundUploadManager] #progress tracker changed for Upload: [id=1917F3C4-0232-4170-B235-1FFAA203E283, order=0, progress=0.97, paused=false, tries=0, lastTry=nil, error=nil, assetId=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, asset=Asset: [id=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, created=2019-07-03 12:25:45 +0000, uti=com.apple.quicktime-movie, title=nil, desc=nil, location=nil, notes=nil, tags=nil, mimeType=video/quicktime, filename=19-06-12 02-57-17 1643.mov, file=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.mov, thumb=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.thumb, phassetId=nil, publicUrl=nil, isReady=true, isUploaded=false]])
[BackgroundUploadManager] #yapDatabaseModified
[BackgroundUploadManager] #uploadNext 5 items in upload queue
[BackgroundUploadManager] #uploadNext already one uploading
[BackgroundUploadManager] #progress tracker changed for Upload: [id=1917F3C4-0232-4170-B235-1FFAA203E283, order=0, progress=0.98, paused=false, tries=0, lastTry=nil, error=nil, assetId=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, asset=Asset: [id=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, created=2019-07-03 12:25:45 +0000, uti=com.apple.quicktime-movie, title=nil, desc=nil, location=nil, notes=nil, tags=nil, mimeType=video/quicktime, filename=19-06-12 02-57-17 1643.mov, file=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.mov, thumb=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.thumb, phassetId=nil, publicUrl=nil, isReady=true, isUploaded=false]])
[BackgroundUploadManager] #yapDatabaseModified
[BackgroundUploadManager] #uploadNext 5 items in upload queue
[BackgroundUploadManager] #uploadNext already one uploading
[BackgroundUploadManager] #progress tracker changed for Upload: [id=1917F3C4-0232-4170-B235-1FFAA203E283, order=0, progress=0.99, paused=false, tries=0, lastTry=nil, error=nil, assetId=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, asset=Asset: [id=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, created=2019-07-03 12:25:45 +0000, uti=com.apple.quicktime-movie, title=nil, desc=nil, location=nil, notes=nil, tags=nil, mimeType=video/quicktime, filename=19-06-12 02-57-17 1643.mov, file=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.mov, thumb=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.thumb, phassetId=nil, publicUrl=nil, isReady=true, isUploaded=false]])
[BackgroundUploadManager] #yapDatabaseModified
[BackgroundUploadManager] #uploadNext 5 items in upload queue
[BackgroundUploadManager] #uploadNext already one uploading
[BackgroundUploadManager] #progress tracker changed for Upload: [id=1917F3C4-0232-4170-B235-1FFAA203E283, order=0, progress=0.62, paused=false, tries=0, lastTry=nil, error=nil, assetId=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, asset=Asset: [id=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, created=2019-07-03 12:25:45 +0000, uti=com.apple.quicktime-movie, title=nil, desc=nil, location=nil, notes=nil, tags=nil, mimeType=video/quicktime, filename=19-06-12 02-57-17 1643.mov, file=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.mov, thumb=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.thumb, phassetId=nil, publicUrl=nil, isReady=true, isUploaded=false]])
[BackgroundUploadManager] #yapDatabaseModified
[BackgroundUploadManager] #uploadNext 5 items in upload queue
[BackgroundUploadManager] #uploadNext already one uploading
[BackgroundUploadManager] #progress tracker changed for Upload: [id=1917F3C4-0232-4170-B235-1FFAA203E283, order=0, progress=0.63, paused=false, tries=0, lastTry=nil, error=nil, assetId=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, asset=Asset: [id=01961ECD-BCA1-4EBA-8836-8877D5CCAC11, created=2019-07-03 12:25:45 +0000, uti=com.apple.quicktime-movie, title=nil, desc=nil, location=nil, notes=nil, tags=nil, mimeType=video/quicktime, filename=19-06-12 02-57-17 1643.mov, file=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.mov, thumb=file:///private/var/mobile/Containers/Shared/AppGroup/534616B9-6584-41B1-B1BC-83D4A49D339D/assets/01961ECD-BCA1-4EBA-8836-8877D5CCAC11.thumb, phassetId=nil, publicUrl=nil, isReady=true, isUploaded=false]])

Happened on real device iPhone 7 with iOS 12.3.1 via LTE.

This is something we can't control. It may be a network packet loss, which makes the server reset to a last known "good" state and have the client resend from there.

foundscapes commented 5 years ago

Which file was it? Should we create a popup that lets folks know when this happens?

tladesignz commented 5 years ago

Ok, so I found various little problems, why this didn't work, yet, which are all fixed now. I'll release a new TestFlight.

However, please note:

When iOS takes over with an upload and suspends our app (~ 3 minutes after app close), it will only call back our app, when the user revisits it.

That means, there's the following situation possible:

Result: File is on the server, app doesn't know about it, uploads file again.

The only thing I can imagine which could help here, is to always check, if a file is already on the server before upload, and if yes, quit the upload job as successful.

However, the question is: Is it possible, that there is a file at the exact place we want to put it, which is not our file and should be overwritten?

We can't check, if file is identical, as that would mean downloading it. Which is a lengthy process with videos, of course.

We could check file size, though and maybe some timestamp.

However, if the user manages to actually try to manually upload a file which already is on the server, that could become confusing, or wouldn't it?

The more I think of it, the more I'm convinced it's actually a good idea, considering our folder structure with timestamps, which makes it very unlikely, that this will ever happen...

Anyway, @n8fr8, @foundscapes, please comment.

tladesignz commented 5 years ago

I regarded your silence as approval, and implemented my idea. New TestFlight will follow.

foundscapes commented 5 years ago

I'm not sure how to answer your question, @n8fr8 ?

foundscapes commented 5 years ago

There is still a difference in size of the same files between devices, iphone 6s / ipad mini, respectively: 14.7mb vs 29.6mb, 41.1mb vs 39.2mb I don't know why, since they are the same files.

n8fr8 commented 5 years ago

It is a good idea to check the server to see if the file has already uploaded. My question is this @tladesignz are we sure the file size reported by the server matches our local file size? I guess only testing will tell!

@foundscapes that is likely due to compression being different or some compression happening by iCloud when you synced the files between devices. This code / issue is not related to how different devices handle compression, just about reliable / resumeable uploads.

foundscapes commented 5 years ago

@n8fr8 ah that makes sense..I think the file I sent over airdrop maintained the same size.

tladesignz commented 5 years ago

My question is this @tladesignz are we sure the file size reported by the server matches our local file size? I guess only testing will tell!

A copy of a file is always an exact one on computers. If not, it's a faulty one and should be removed and retransmitted.

Since copies are exact, the file size must be the same. Everything else is either an execution error, a protocol design failure or a violation of a protocol... I can understand your uneasiness, but there's really no wiggle room for interpretation. And besides: I tested it. It's fine. ;-)

@foundscapes Yeah, recompression is most probably the explanation of the file size differences.