Crashes every few hundreds/thousands files

[changemenemo]

So @marinofaggiana do you receive logs for the crashes with the new version or not ? Because you specifically asked to not send you logs. I'm sorry but it's a phone. Not rooted. Not a full OS like linux or windows. So if any crashes are happening it can't be to only system since it's an embedded system. And the problem you were talking about in the other thread was that you weren't receiving any logs about the crashes. 1/ So do you have any new input about these ? 2/ do you receive automatically crashes logs now so that you disabled the manual sending feature ?

What's the status about this crashes we are experiencing ? Since I'm not the only one and you are avoiding any interaction with me recently

[marinofaggiana]

Hi no no, nothing about you ! Nay thanks for your help, I have several task every day open and I can't dedicate too many time dia single issue. Do you have news about this ? Do you have try with the last TestFlight version ?

[changemenemo]

@marinofaggiana 1/ still crashing with the last one from testflight. 2/ Still don't get the crash window pop up to send you the logs(you never answered if it was normal or not) 3/ still crashing with my new version of nextcloud 18.0.7 just updated today to that version

[mmischitelli]

Exactly like @boistordu says. Nextcloud 19.0.2 and while trying to upload, crashes every 100-200 files more or less. And, while uploading, the whole app is completely unresponsive

[changemenemo]

@mmischitelli do you have also some files in the list that does not show their thumbnails and which are apparently on hold ?

[mmischitelli]

@boistordu Can't really tell now, but I don't think I saw pending files. I only had jpgs, heic, mov, mp4.

[changemenemo]

@mmischitelli no problem it's to guess if my pending photos were the problem or not but so it isn't.

[marinofaggiana]

What's do you mean for 100 files ?? please write how I can reproduce the issue

[mmischitelli]

What's do you mean for 100 files ?? please write how I can reproduce the issue

Basically just set up phone sync with a phone with hundreds of photos/videos on it. In my case, I have an iPhone 7+ still in high performance mode (original battery with health level > 80%) with roughly 4000 media files (mixed jpg/heic/mov/mp4). NextCloud 19.0.2 running in docker on unraid with postgres 11 db.

[changemenemo]

And me as you know @marinofaggiana it s a 18.0.7 on arm board installed with yunohost. The client is an iPhone XR with thousands of photos videos mixed for my auto sync. I deactivate compatibility mode for the photo.

And there still isn't any crash report pop up at crash of the apps

[mmischitelli]

Maybe this has to do something with the crash we're seeing:

[LOG] Could not write to database: Error Domain=io.realm Code=9 "mmap() failed: Cannot allocate memory size: 29999104 offset: 67108864" UserInfo={NSLocalizedDescription=mmap() failed: Cannot allocate memory size: 29999104 offset: 67108864, Error Code=9}

[marinofaggiana]

mmm, interesting, out of memory ....

[changemenemo]

So I m happy to see that the crash report has come back as a feature. Crashes still happening

[marinofaggiana]

I have find a possible issue please test with the last 3.0.7 (build 6)

[changemenemo]

@marinofaggiana I just test it. Still the same crashes. So not related to the display module of transfers. I've filled out the crash pop up form. Hope you did receive the logs

[lukasmaurer83]

I have the same issue here, the app starts crashing after syncing aprox. 100 files, if you want to make a big „initial sync“ and have over 25.000 files (most of them images), this is almost impossible, since you have to sit next to the phone all the time, and reset the app. Please fix this, because it makes the app really unusable, thanks in advance.

Btw. @marinofaggiana I would also like to send some kind of crash report to assist you in finding the issue, but I never got to see the „crash pop up form“ @boistordu mentioned. I also elevated the loglevel to maximum, and also had this error prior to the crash:

[LOG] Could not write to database: Error Domain=io.realm Code=9 "mmap() failed: Cannot allocate memory size: 46235648 offset: 134217728" UserInfo={NSLocalizedDescription=mmap() failed: Cannot allocate memory size: 46235648 offset: 134217728, Error Code=9}

[marinofaggiana]

I have open an Issue for a alloc memory on Database ... I wait a reply

[changemenemo]

I have open an Issue for a alloc memory on Database ... I wait a reply

On nextcloud issues itself you mean ?

[marinofaggiana]

No no it's an issue of iOS database used, no nextcloud.

[changemenemo]

Mmmmh okey sorry for the misunderstanding. Keep us informed please as soon as you got news

[marinofaggiana]

Good news with the last 3.0.7 (build 8)

[lukasmaurer83]

Thanks for your efforts, the new testflight version has significantly reduced tendencey towards crash on sync, and the alloc memory error in the logs is gone. Still had two crashes, but far less than before.

[changemenemo]

Yeah same here. It I didn't have the time to go all the way and I still didn t test the auto upload in background etc. I m just testing the upload of the library. When I have more time I will try to go all the way with the 6k files that I have left to upload. So stay tuned.

[changemenemo]

Okey I had a crash after 1500 files because there is a lack of storage now. So I guess the other crashes are more or less resolved.

@marinofaggiana can you tell me why the temporary files are not deleting themselves ?

So I guess the process is the following,

• one by one the photos from the photo library get copied in the nextcloud app's folder
• then one by one uploaded
• then what ?

This copy isn't deleted and stays in the app folder ? Is there a reason why it doesn't get deleted ?

Also there is this option to get rid of the photos when they get uploaded, but that's for the photo of the photo library itself no? Not only for the copies ...

So could you please make an update that deletes the copy of the photo library just after getting uploaded ?

ÉDIT : Also after this crash there is no credentials retained in the app, it's like it has been reset. Even if the app is still with 28G of data

[schmurtzm]

I can confirm that there is a lot less crash (may be not but see below...) with the last test flight (3.0.7 (8)). But when I make an "Upload the whole camera roll" with more than 13000 photos, it stuck systematically at 9724 photo in queue... I tried many time (reinstall the ios app) with the same result.

Fun fact : if I take a new picture, this one is correctly uploaded. If I "force the start" of the upload of a file in the top of the queue it crash. And if I "force the start" in the middle of the queue the file is uploaded but nothing more....

I don't find any way to upload the rest of the queue :-(

[changemenemo]

I can confirm that there is a lot less crash (may be not but see below...) with the last test flight (3.0.7 (8)).

But when I make an "Upload the whole camera roll" with more than 13000 photos, it stuck systematically at 9724 photo in queue...

I tried many time (reinstall the ios app) with the same result.

Fun fact : if I take a new picture, this one is correctly uploaded.

If I "force the start" of the upload of a file in the top of the queue it crash.

And if I "force the start" in the middle of the queue the file is uploaded but nothing more....

I don't find any way to upload the rest of the queue :-(

How is the storage it takes on your phone ?

[schmurtzm]

How is the storage it takes on your phone ?

50Go available... seems to be enough ;-) Very annoying problem....

[marinofaggiana]

@strugee

[changemenemo]

After 6k of files uploaded while the app being active( so no background upload activated), without any crashes I think we can declare that the app is free of memory crashes problem and close the ticket.

[changemenemo]

I think we need to reopen this issue. It came back. Every few hundreds of files the app is crashing while being active in foreground. iPhone XR 15G space left on the iPhone so it s not that.

The instance I m doing the test to is a 4 core 10G ram on a xeon gold and with ssd as storage support. 10G network in a data center. So it can't be the instance which is problematic here.

I m again not sure that multiple uploads at the same time is such a good idea again... don't forget we don't have unlimited RAM. Did you get the logs uploads ?

[changemenemo]

I ve sent you a mail with a link to doWnload the log

[schmurtzm]

Just in case, check if you don't have a corrupted video (take a look in slow motion videos in pririty). Take a look to my issue here, may be it is similar ?

Anyway, even if the file is corrupted it should not crash nextcloud app, the file should be skipped. Agree to say that multiple upload is not always a good idea, it also increases fragmentation on HDD server. May be it should be a parameter like on FTP client software ?

[sethigoldy]

I am getting almost exactly the same behaviour. My scenario is I have like 12K+ photos on icloud server when I start Upload the whole camera roll it starts creating archive and after that starts uploading but in middle sometimes around 3k or 6k it just stop uploading any photos/videos. I can see the last lines in the logs are saying:

2021-02-23 14:07:45 Network request body: None
2021-02-23 14:07:51 Automatic upload, no new assets found

But I still currently have like 3610 files pending, there's no error in the logs (I searched for it). I even enabled Disable Local Cache but nothing seems to work. And I am sure all of my files are free of corruption as they are uploaded to Apple icloud servers and I can view every one of them.

Let me know if you guys need anything else regarding it.

iOS version : 3.2.0.30
Nextcloud server : 20.0.7

By the way I am grateful to each one of the contributors of Nextcloud (You guys/gals Rock)