Discussion: Integrating sync quality settings

[DeveloperOl]

I have some images in the cloud which I would like to synchronize to my phone to keep in my pocket. However the camera used for taking these pictures has a lot of MP and therefore the images are between 10-20Mb per image. This results in folder sizes around 40Gb for one vacation.

This is way to big for keeping on my phone and I used to solve this by keeping a compressed version on my phone and the original files in the cloud. However I think an app like lespas might be the perfect solution for this problem.

Additional context This issue is mainly intended as a place for discussion in the beginning. I have no experience how lespas is working under the hood but I am willing to put effort in a PR if you consider this feature usefull and implementable. Also if you see any problems which might occur using the current sync algorithm I would be happy to open a discussion.

My idea so far would be: allow setting compression / max resolution per album (saved in json). when changing the setting resync album. when syncing download image -> convert -> mark as saved (cloud images not touched). when uploading original image must be kept until upload succeeds

[scubajeff]

Lespas tackle this with 2 features combined, 'remote album' and 'remote album cache size setting'.

When an album is 'remote', Lespas won't store it's media files in your device, instead, everything, the thumbnail, the original picture in full resolution, are downloaded from server on the fly. (You should really follow the faq on how to setup the right resolution thumbnail pre-generation) Those data downloaded from server will stay in a LRU cache maintained by underlaying okhttp library. You can set it's size limit in Lespas's setting.

With these, you can enjoy your works in their full glory without worrying squeezing your precious device disk space.

Be awared that the size limit is only for remote album pictures, the app itself will also use tens of megabytes of cache, and if you have videos, that will further set you back by around 100MB+ for video streaming. I took the default setting of 800MB, the actual app storage usage right now is around 1GB.

[DeveloperOl]

I understand the principle of caching remote albums, however, this does not tackle the initial problem. Large pictures take forever to download (if not cached) and nextcloud takes some time to generate thumbnails as well (if not using a cache plugin). Further, if I want to show the pictures to someone I need to load 50Gb when looking at every picture. That's where my idea of a "low resolution" download option comes from. Currently the only option I have is keeping low and high res images in separate albums with the low res album synced. However this means duplicated images and a lot of error Potential for me.

[scubajeff]

Unless there is already a copy of low resolution picture on server or there is a piece of server app which generate this picture on the fly, Lespas has no way to download 'it'.

50GB for a trip is hugh. I have more than a thousand pictures in Lespas, and the total size is about 14GB.

Picture with size of 10-20MB in compressed state will need roughly 200MB memory to display it. Thrilled to know that Lespas actually works.

[DeveloperOl]

At least not a way to download it yet! If you would Accept this feature/the solution I described above I would try implementing it and make a PR. Love this project so far and it solves a lot of my problems! But I also found another bug which leads to an instant crash of the app which I would investigate first!

[steviehs]

Currently the only option I have is keeping low and high res images in separate albums with the low res album synced. However this means duplicated images and a lot of error Potential for me.

May be have a look at my scripts, which are reffered in the readme: phonepics.py and digiphoto.py

digiphoto.py puts all pictures in original resolution untouched (except the name, which is desired) in one folder structure and phonepics.py puts the reduced/resized pics in another flat folder structure for syncing to lespas. Using this process - my life in pictures is now approx 14000 picts in approx 360 flat folders (for lespas) and still only taking around 6 Gigs.

[DeveloperOl]

@steviehs Nice work I like the approach. However one problem remains for me. I take some photos with a physical camera and some on the phone. For the pictures on the phone I use lespas itself to sync them into the cloud (because I can do it anywhere anytime offline when I have time and it syncs when going online again). If I would use this workflow with your script I would end up with high res pictures in the flat hirachy which would mean a total mess because these picture are missing in the high res structure and still take up lots of space.

Oh and ofcourse I could use the official nextcloud app for the sync but I have experienced syncing problems many times and I'm afraid to delete images on my phone which are not synced. I trust lespas more lol. I could use the native nextcloud selection tool but then I cannot use my phone gallery app.

@scubajeff My favorite would still be a more native approach where lespas does the sync "on the fly".

[scubajeff]

@DeveloperOl Yes, if there is a 'companion' app runs on NC server exposing APIs to Lespas client to do all sorts of things, like shrinking large images, collecting meta data, providing a more integrated file sharing API, that would be the best solution. But sadly, I have zero knowledge on PHP.

BTW, I think @steviehs scripts does generate folders, 'flat' doesn't mean all photos in only one folder. As far as I know, @steviehs has hundreds of albums in Lespas.

[DeveloperOl]

Oh a companion app is a very nice idea! I haven't thought about that. I will look into that! I am not a PHP expert either but I know it better than kotlin lol.

When uploading images that would mean a small change though. A) upload image, delete locally and redownload from companion (with quality changes) Or B) Upload Image and convert locally (use companion app only for redownloading)

My preference would be B to reduce network traffic and improve nextcloud performance. Do you agree?

[scubajeff]

@DeveloperOl do you mean those large picture files are generated by your phone rather than digital camera? For the later situation, I think @steviehs scripts could very well be the starting point. I had some reading on NC's developer doc before, seems to me, their app is a wrapper on PHP codes, like manifest.xml in android world but a bit more complicated.

[DeveloperOl]

No the large images are from the digital camera. But for consistency reasons I would say either all images should be compressed or none. Not just some, that would confuse users I'd say.

[scubajeff]

If we go down this path, e.g., I found someone who can help me out on the server end, an option setting the size limit triggering shrinking will be enough.

[DeveloperOl]

I can look into this. But to be clear, the images remain full sized on the nextcloud side and are just compressed for lespas

[DeveloperOl]

Another Idea on this Topic after doing some research.

Maybe it is smarter not to use a self built companion app (another piece of software which would need maintainance) and instead allow a setting to download the 1024p aspect ration preview. In my tests the aspect ratio images still have a nice quality on the phone (and even on a computer). The compression ratio for me was around ~95-97% per image (eg 160KB instead of 5.8MB). Tested using: https://[baseurl]/core/preview?fileId=[fid]&x=1024&y=1024&a=true on recommended preview- generator settings. My idea is, to make this an optional setting. I can do the implementation when I have time if you accept @scubajeff

Advantages: Integrates with recommended nextcloud preview-generator settings Therefore no additional dependencies/maintainance needed Fast download speed Low storage on Phone Cloud images will remain full quality and untouched fully optional

Disadvantages: Pictures added directly from the phone will not be affected (this could be fixed by re-downloading after upload)