OutOfMemoryError

[nascentt]

fatal error: java.util.concurrent.executionexception: java.lang.OutOfMemoryError: Failed to allocate

not sure if this is because of a memory leak or something, it seems to only be a 12mb file it tried to load. I have seen this happen before also.

[nascentt]

This particular device is a 4gb RAM dedicated purpose tablet.

[mvysny]

The problem with Android is that regardless the OS memory, it only provides 16mb of memory to an app, or 24mb if the app says it needs more. I have configured PhotoCloud to use the android:largeHeap but that's all I can do unfortunately - Android decides how much memory the app receives. I thought it is 24mb on all devices, but apparently this value differs per-device.

It would be interesting to see what value the getLargeMemoryClass() function returns on your device.

I am trying hard to load just the necessary images, but at any point I have to hold at least two-three images in memory (when you fling to history). I also downscale the images so that they fit on your screen. However that's not possible with RAWs. It would be interesting to see where exactly the OOM occurred. I'll try to check the current error reportings.

[mvysny]

Unfortunately Crashlytics is really horrible and hides old reports, so I can't access the OOM reports. It would be great if you could obtain the stacktrace crash from your device and paste it here.

Yet, I'm not sure what I can do. Apparently if the app is only given 24 megs of RAM, then it's impossible to load two 12 mb images along with the app. Maybe other software worked around this limitation by using native code (since that should be subject to a different memory restriction set), but I have no idea in this regard unfortunately.

[nascentt]

Tell me what to do and I'll do my best to help diagnose. Crazy you only get 16mb - the device has 4gb ram. The images are huge (dslr) so it sort of makes sense.

The bigger problem is the app doesn't continue after the error. Would be nice if you discarded the images in memory then start again rather than just stay at the error screen.

I plan on giving tablets to family as photoframes for Christmas but the errors that halt images showing will confuse them as they're elderly. I plan on it being non-interactive.

On 17 Dec 2017 9:25 p.m., "Martin Vysny" notifications@github.com wrote:

Unfortunately Crashlytics is really horrible and hides old reports, so I can't access the OOM reports. It would be great if you could obtain the stacktrace crash from your device and paste it here.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#issuecomment-352286750, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7ery3KmXXNNHatN-Zse_Z3y6g59aYks5tBYa2gaJpZM4RExdN .

[mvysny]

Tell me what to do and I'll do my best to help diagnose. Crazy you only get 16mb - the device has 4gb ram. The images are huge (dslr) so it sort of makes sense.

Yup I agree it's crazy :(

Are those DSLR images RAWs by any chance? If yes, it should help if you convert them to jpegs - Android can then load jpegs and downscale them at the same time - that's very memory efficient because the images are downscaled to the tablet's resolution as they are loaded. Thus they are downgraded to, say 2mpx (with fullHD displays), but since nobody's going to zoom them in, that's enough. And 2mpx photo takes way less memory. Unfortunately for technical reasons RAWs cannot be downscaled-on-load (since there is no such library which supports that).

The bigger problem is the app doesn't continue after the error. Would be nice if you discarded the images in memory then start again rather than just stay at the error screen.

The problem with OutOfMemory is that not only the loader fails, the other concurrently running parts of the app typically fail as well (since there's no memory), and the worst thing is that they do so in a random fashion. Thus, the app may be in some half-crashed unpredictable state after OutOfMemory. Therefore it's generally very dangerous to auto-repeat the OutOfMemory in particular. Of course PhotoCloud repeats less serious I/O errors (such as temporary cloud failures, or temporary internet outages), but it's really not recommend to repeat OutOfMemory.

I plan on giving tablets to family as photoframes for Christmas but the errors that halt images showing will confuse them as they're elderly. I plan on it being non-interactive.

I understand and I agree - the app should just run without any issues since it will probably run on a tablet hanged on a wall which nobody touches anymore. And it should exactly do that, with the exception of OutOfMemory errors...

[mvysny]

Fixing #56 should help a bit with the memory usage. Please try to upgrade to 1.10.5

[mvysny]

With PhotoCloud 1.10.5 you can check how much memory Android gives to PhotoCloud (it's in the About dialog). I've found out that on my phone with 2gb of memory PhotoCloud receives a generous 256 MB; on my other phone with 3gb of memory PhotoCloud receives 512 MB. I have never experienced out of memory on my phones.

Can you check how much memory PhotoCloud receives on your device please?

[nascentt]

384mb

On 23 Dec 2017 4:04 p.m., "Martin Vysny" notifications@github.com wrote:

With PhotoCloud 1.10.5 you can check how much memory Android gives to PhotoCloud (it's in the About dialog). I've found out that on my phone with 2gb of memory PhotoCloud receives a generous 256 MB; on my other phone with 3gb of memory PhotoCloud receives 512 MB. I have never experienced out of memory on my phones.

Can you check how much memory PhotoCloud receives on your device please?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#issuecomment-353733958, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7erfI4y6vQDo5AphBZs3XiPjaHH7lks5tDSSngaJpZM4RExdN .

[mvysny]

That should be more than enough memory to run Photocloud...

[mvysny]

This bug report illustrates the problem with handling out of memory errors quite well:

Fatal Exception: java.lang.OutOfMemoryError: OutOfMemoryError thrown while trying to throw OutOfMemoryError; no stack trace available

:)

[nascentt]

the issues been happening more and more frequently in recent days, really bizarre. 2GB free currently

[nascentt]

Something interesting I noted today regarding this. For the last 24 hours my server was offline, so the photoframe couldnt access the photo share. This obviously meant that it cycled through the cached images. After 24 hours I got the server back up. The tablet took a minute or so to recognise the server and started streaming new pictures again. Within minutes BAM "OutOfMemory". So I close all apps, free memory. Restart the slideshow and a few minutes passes BAM.

24 hours no issues... no access to photo share. A few minutes of access to photoshare and errors reoccur.

Surely that's not a coincidence right? Whether it means that the cached photos are small enough to fit in the memory whereas the photos it tries to load from the server to big....or it's an issue with using too much ram traversing the server heirachy...or something else.

That seems to give some sort of clue. Also I use non default cache settings, no idea if that might alter anything, such as thumbnail cache or photo cache? Maybe I should set those as default again?

On 26 December 2017 at 05:12, Martin Vysny notifications@github.com wrote:

This bug report illustrates the problem with handling out of memory errors quite well:

Fatal Exception: java.lang.OutOfMemoryError: OutOfMemoryError thrown while trying to throw OutOfMemoryError; no stack trace available

:)

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#issuecomment-353918761, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7elQmelmtqgUhuCXo3x5fQk_hAXbcks5tEIAkgaJpZM4RExdN .

[mvysny]

Surely that's not a coincidence right?

Most probably it's not a coincidence. The photos are always cached downscaled to the resolution of the device screen. That makes sense since typically your device just hangs on a wall showing pictures; you typically do not zoom those pictures in. Downscaled versions also take up significantly less space in the cache.

As you say, apparently showing cached (downscaled) versions of images works rock solid for 24 hours straight; showing full-sized images from your storage fails with OOM. Apparently there is something wrong with the image decoder (since the decoder should downscale loaded images to the device resolution on-the-fly and should thus not take up lots of memory).

I have optimized the server hierarchy traversal and it should now use rather small amount of RAM; I'm guessing the largeness of those photos is the culprit here.

1. Are you sure that those large images aren't RAWs? RAW decoding does not support on-the-fly downscale and takes up enormous amount of memory. RAW support is currently experimental and is known to fail with OOM on large photos. Please try to not to slideshow RAWs and let's see if it helps.
2. What kind of images are you showing? I need pixel resolution and a format (is it JPEG only?)

To pinpoint the problem:

1. Try to only show a folder with a handful (say, 100) of JPEGs with original size in the slideshow, no RAWs; please let me know if this controlled environment still crashes with OOM or not.
2. If it still crashes, try imagemagick or other mass downscaler software to downscale a couple of images to, say, 1080p, and only show that folder, to see if it helps.

[nascentt]

hmm I do have the "Include RAWs in slideshow unticked. SO it should avoid any RAW files right?

I can try step 2, but I guess due to having thousands of images I need to test with individual folders first to confirm a confirm that is actually affected.

[nascentt]

images are from various devices, but mostly smartphone, and dslr

On 7 January 2018 at 16:32, nascent nascent.signup@gmail.com wrote:

hmm I do have the "Include RAWs in slideshow unticked. SO it should avoid any RAW files right?

I can try step 2, but I guess due to having thousands of images I need to test with individual folders first to confirm a confirm that is actually affected.

[mvysny]

hmm I do have the "Include RAWs in slideshow unticked. SO it should avoid any RAW files right?

True; PhotoCloud should ignore raw images.

I can try step 2, but I guess due to having thousands of images I need to test with individual folders first to confirm a confirm that is actually affected.

I'd suggest to find a folder with biggest image, root the stream there and try the Slideshow whether it will crash or not.

A different question just came to my mind: are you perhaps using symlinks? Could it be that symlinks to parents are used, which creates an infinitely deep tree?

[nascentt]

Interesting. I do have a symlink - although it shouldn't cause an infinite loop, as it points to another location... Let me remove it for now. I'll search for the largest files and put thme in a folder and test. I do have some stray mp4 and raw in some of thee folders, does the app ignore any movies be default?

On 7 January 2018 at 16:37, Martin Vysny notifications@github.com wrote:

hmm I do have the "Include RAWs in slideshow unticked. SO it should avoid any RAW files right?

True; PhotoCloud should ignore raw images.

I can try step 2, but I guess due to having thousands of images I need to test with individual folders first to confirm a confirm that is actually affected.

I'd suggest to find a folder with biggest image, root the stream there and try the Slideshow whether it will crash or not.

A different question just came to my mind: are you perhaps using symlinks? Could it be that symlinks to parents are used, which creates an infinitely deep tree?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#issuecomment-355834962, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7eqB8bUxm6V3CuzimKRYWSJ6MjbfNks5tIPLBgaJpZM4RExdN .

[mvysny]

Yes, the app should ignore all files except "png", "gif", "bmp", "jpg", "jpeg"

[nascentt]

I just checked, the symlink was already gone, it was probably the first thing I tried myself.

I guess I'll try random subfolders/large files until I find a folder that causes the crash.

On 7 January 2018 at 16:53, Martin Vysny notifications@github.com wrote:

Yes, the app should ignore all files except "png", "gif", "bmp", "jpg", "jpeg"

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#issuecomment-355835974, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7ejfIsN-vBKdZGO_vjFdlqG2RHAR8ks5tIPaYgaJpZM4RExdN .

[mvysny]

Thank you very much for your patience. At this point unfortunately I have no idea what may be the cause of the OOM. Perhaps it is an image that is somehow corrupted; maybe a set of images. Unfortunately I can't reproduce the OOM and hence I can't hunt for such image myself, otherwise I would have done so. Meanwhile I will revisit the image loading code, perhaps there is something I have missed?

[mvysny]

I'm thinking: to remedy this, maybe I can introduce an option to download photos at half the resolution of the screen. I believe that the quality of the photos won't suffer much and it certainly beats PhotoCloud crashing with OutOfMemory randomly...

[mvysny]

Added the setting in PhotoCloud 1.10.10; it's in Settings / Slideshow / Advanced. Closing; please feel free to reopen if you succeed in finding that pesky photo which causes this memory-related crash to occur.

[nascentt]

Just to update. I haven't had a single OutOfMemory error since enabling this. It might be a coincidence as it might not have hit whichever folder that was causing it, but when I'd get it I'd then get it 30 times that day and that hasn't happened yet.

On 19 January 2018 at 18:44, Martin Vysny notifications@github.com wrote:

Closed #53 https://github.com/mvysny/photocloud-frame-slideshow/issues/53.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#event-1433017371, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7ehD9SDi6QuUoFlXdQsKNqN9trRwfks5tMOJ7gaJpZM4RExdN .

[nascentt]

Edit: July 2018 update: Have not seen this error since january. Still have the half resolution option Unticked, so this has been fixed somehow. Weird.

One more update. Not a single OutOfMemory error in the past 1-2 months. Despite getting them on an hourly basis around christmas. I also don't have the half resolution option enabled. Nor did I ignore any directories.

On 23 January 2018 at 19:43, nascent nascent.signup@gmail.com wrote:

Just to update. I haven't had a single OutOfMemory error since enabling this. It might be a coincidence as it might not have hit whichever folder that was causing it, but when I'd get it I'd then get it 30 times that day and that hasn't happened yet.

On 19 January 2018 at 18:44, Martin Vysny notifications@github.com wrote:

Closed #53 https://github.com/mvysny/photocloud-frame-slideshow/issues/53.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/mvysny/photocloud-frame-slideshow/issues/53#event-1433017371, or mute the thread https://github.com/notifications/unsubscribe-auth/ABF7ehD9SDi6QuUoFlXdQsKNqN9trRwfks5tMOJ7gaJpZM4RExdN .