search

vauvenal5 / yaga

Nextcloud Yaga - A Nextcloud first gallery app for Android.
https://vauvenal5.github.io/yaga-docs/
GNU General Public License v3.0
136 stars 9 forks source link

Only showing a few images in "home view" even with recursive on #97

Open zerodogg opened 3 years ago

zerodogg commented 3 years ago

My nextcloud account has 8606 images. I've got it added to Yaga with recursive enabled. The home view lists 36 images. I'm not able to see any clear pattern as to which images it picks for display, it's not the most recent. The only files that always show up are those at the root of my Nextcloud. The same occurs if I select the root in «browse view» and then open up in «focus» mode, it doesn't appear to load recursively.

Is there a limit on how deep recursive mode will go? If I open up my «InstantUpload» folder in «focus»-mode I get a lot more images (not quite sure if it's all of them, though).

vauvenal5 commented 3 years ago

Can you tell me which Yaga version you are using?

zerodogg commented 3 years ago

0.22.3 from f-droid

vauvenal5 commented 3 years ago

Are you able to retrieve logcat logs?

zerodogg commented 3 years ago

Yes, I can get logcat. I tried looking at logcat while yaga was syncing just now, and that's quite the information overload so if you're looking for specifics, let me know.

I did find this (server and username removed) multiple times in my log:

03-16 07:25:51.587  7955  8068 I flutter : [W] NextcloudFileManager - Emiting list! (nc://.../InstantUpload/)
03-16 07:25:51.590  7955  7980 I flutter : [E] ForegroundWorker - Error in forground worker: NoSuchMethodError: The getter 'microsecondsSinceEpoch' was called on null.
03-16 07:25:51.590  7955  7980 I flutter : Receiver: null
03-16 07:25:51.590  7955  7980 I flutter : Tried calling: microsecondsSinceEpoch
03-16 07:25:51.590  7955  7980 I flutter : #0      DateTime.compareTo (dart:core-patch/date_patch.dart:94)
03-16 07:25:51.590  7955  7980 I flutter : #1      SortManager._sortListByDateModified.<anonymous closure> (package:yaga/managers/isolateable/sort_manager.dart:179)
03-16 07:25:51.590  7955  7980 I flutter : #2      Sort._dualPivotQuicksort (dart:_internal/sort.dart:238)
03-16 07:25:51.590  7955  7980 I flutter : #3      Sort._doSort (dart:_internal/sort.dart:60)
03-16 07:25:51.590  7955  7980 I flutter : #4      Sort.sort (dart:_internal/sort.dart:33)
03-16 07:25:51.590  7955  7980 I flutter : #5      ListMixin.sort (dart:collection/list.dart:355)
03-16 07:25:51.590  7955  7980 I flutter : #6      SortManager._sortListByDateModified (package:yaga/managers/isolateable/sort_manager.dart:178)
03-16 07:25:51.590  7955  7980 I flutter : #7      SortManager._sortFileFolders (package:yaga/managers/isolateable/sort_manager.dart:144)
03-16 07:25:51.590  7955  7980 I flutter : #8      SortManager.sortList (package:yaga/managers/isolateable/sort_manager.dart:136)
03-16 07:25:51.590  7955  7980 I flutter : #9      IsolatedFileManager.listFileLists.<anonymous closure> (package:yaga/managers/isolateable/isolated_file_manager.dart:58)
03-16 07:25:51.590  7955  7980 I flutter : #10     _MapStream._handleData (dart:async/stream_pipe.dart:219)
03-16 07:25:51.590  7955  7980 I flutter : #11     _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:157)
03-16 07:25:51.590  7955  7980 I flutter : #12     _RootZone.runUnaryGuarded (dart:async/zone.dart:13
vauvenal5 commented 3 years ago

This is an interesting one. The error you are seeing would suggest that you have pictures without a date modified. :thinking:

Edit: Since this is very unlikely there has to be another issue causing this. Which view type are you using when this happens? Does the behavior change when switching view types?

Edit2: Do you have any files with a image mime type which do not have a date modified?

lolrepeatlol commented 3 years ago

I also have the same issue. I actually have to browse to each individual image in the browse screen before they show up in the home screen, and that's when I have recursive mode on. When I don't, nothing shows up at all.

vauvenal5 commented 3 years ago

@lolrepeatlol can you please also send me some logs? There is a send logs function by now in the global settings.

vauvenal5 commented 3 years ago

@lolrepeatlol is it possible that your home view target directory is wrongly configured? Can you check your view specific settings in the top-right menu when you are in the home view?

@zerodogg the only case where I was able to reproduce missing files was when the network connectivity is too bad and requests time out. Once you have updated to version v0.23.1 or higher can you please retry and send me new logs with the integrated send logs-function?

lolrepeatlol commented 3 years ago

@lolrepeatlol is it possible that your home view target directory is wrongly configured? Can you check your view specific settings in the top-right menu when you are in the home view?

oops! that seems to be it, hahaha. not to be mean, but that wasn't really made clear from the start. looks like i should get to work on the ux 🙃

vauvenal5 commented 3 years ago

@lolrepeatlol to be fair, it is actually mentioned in the quickstart guide. But yes, I am myself not very happy about the way it works currently.

dln949 commented 3 years ago

I am confused.

I have the same problem that zerodogg posted here: I have around 6,000 photos on my server, but the yaga app seems to just randomly display only about 100 of them. Based on what was written above, I don't know what the proposed fix is.

version 23.10

nextcloud home path looks right to me

vauvenal5 commented 3 years ago

Can you provide your logs and a screenshot of your view local settings?

Akruidenberg commented 3 years ago

Same problem here. Not all my photos are shown.

vauvenal5 commented 3 years ago

Can you provide your logs and a screenshot of your view local settings?

robert-wettstaedt commented 3 years ago

Same here. Photos are only showing when I manually open them in Browse View.

Here are screenshots of the settings:

Screenshot_20210429-072016_Yaga 2 Screenshot_20210429-072026_Yaga 2 Screenshot_20210429-072037_Yaga

And here is my log (after starting Yaga, I reconfirmed all the settings):

2021-04-29 07:23:01.422465 INFO LogErrorFileHandler - Opened file
2021-04-29 07:23:01.567798 SHOUT LogErrorFileHandler - ------- APP INFO -------
2021-04-29 07:23:01.567942 SHOUT LogErrorFileHandler - environment: release
2021-04-29 07:23:01.567974 SHOUT LogErrorFileHandler - version: 0.25.0
2021-04-29 07:23:01.568005 SHOUT LogErrorFileHandler - appName: Yaga
2021-04-29 07:23:01.568032 SHOUT LogErrorFileHandler - buildNumber: 2500
2021-04-29 07:23:01.568067 SHOUT LogErrorFileHandler - packageName: com.github.vauvenal5.yaga
2021-04-29 07:23:01.568094 SHOUT LogErrorFileHandler - ------- END APP INFO -------
2021-04-29 07:23:02.225254 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (start)
2021-04-29 07:23:02.243139 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (received list - #images: 1)
2021-04-29 07:23:02.243985 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - manager key)
2021-04-29 07:23:02.244156 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - event key)
2021-04-29 07:23:08.479954 WARNING FileListLocalManager - fba5dcf0-a8aa-11eb-b288-2348cb2a170c (start)
2021-04-29 07:23:08.482822 WARNING FileListLocalManager - fba65220-a8aa-11eb-a2a7-db0b34316353 (start)
2021-04-29 07:23:08.485134 WARNING FileListLocalManager - fba6c750-a8aa-11eb-bf4d-b9b640447898 (start)
2021-04-29 07:23:08.497563 WARNING FileListLocalManager - fba5dcf0-a8aa-11eb-b288-2348cb2a170c (received list - #images: 0)
2021-04-29 07:23:08.498017 WARNING FileListLocalManager - fba65220-a8aa-11eb-a2a7-db0b34316353 (received list - #images: 0)
2021-04-29 07:23:08.503825 WARNING FileListLocalManager - fba6c750-a8aa-11eb-bf4d-b9b640447898 (received list - #images: 0)
2021-04-29 07:23:08.958833 WARNING FileListLocalManager - fba6c750-a8aa-11eb-bf4d-b9b640447898 (received list - #images: 0)
2021-04-29 07:23:08.959109 WARNING FileListLocalManager - fba6c750-a8aa-11eb-bf4d-b9b640447898 (done - manager key)
2021-04-29 07:23:08.959162 WARNING FileListLocalManager - fba6c750-a8aa-11eb-bf4d-b9b640447898 (done - event key)
2021-04-29 07:23:08.961506 WARNING FileListLocalManager - fba5dcf0-a8aa-11eb-b288-2348cb2a170c (received list - #images: 0)
2021-04-29 07:23:08.961754 WARNING FileListLocalManager - fba5dcf0-a8aa-11eb-b288-2348cb2a170c (done - manager key)
2021-04-29 07:23:08.961803 WARNING FileListLocalManager - fba5dcf0-a8aa-11eb-b288-2348cb2a170c (done - event key)
2021-04-29 07:23:08.964153 WARNING FileListLocalManager - fba65220-a8aa-11eb-a2a7-db0b34316353 (received list - #images: 0)
2021-04-29 07:23:08.964385 WARNING FileListLocalManager - fba65220-a8aa-11eb-a2a7-db0b34316353 (done - manager key)
2021-04-29 07:23:08.964458 WARNING FileListLocalManager - fba65220-a8aa-11eb-a2a7-db0b34316353 (done - event key)
2021-04-29 07:23:10.716317 WARNING SettingsManagerBridge - UriPreference {
  fixedOrigin=true,
  value=nc://<URL>/Pictures/Camera/,
  key=nc:mapping:remote,
  title=Remote Path,
  enabled=true,
}
2021-04-29 07:23:14.369288 WARNING SettingsManagerBridge - MappingPreference {
  remote=UriPreference {
    fixedOrigin=true,
    value=nc://<URL>/Pictures/Camera/,
    key=nc:mapping:remote,
    title=Remote Path,
    enabled=true,
  },
  local=UriPreference {
    fixedOrigin=true,
    value=file://device.local/Android/data/com.github.vauvenal5.yaga/files/,
    key=nc:mapping:local,
    title=Local Path,
    enabled=true,
  },
  syncDeletes=BoolPreference {
    value=true,
    key=nc:mapping:syncDeletes,
    title=Sync Server Deletes,
    enabled=true,
  },
  value=false,
  key=nc:mapping,
  title=Root Mapping,
  enabled=true,
}
2021-04-29 07:23:14.370068 SEVERE ForegroundWorker - Error in forground worker
2021-04-29 07:23:14.370068 SEVERE ForegroundWorker - #0      MappingManager._removeFromTree (package:yaga/managers/isolateable/mapping_manager.dart:64)
#1      MappingManager._removeFromTree (package:yaga/managers/isolateable/mapping_manager.dart:68)
#2      MappingManager._removeFromTree (package:yaga/managers/isolateable/mapping_manager.dart:68)
#3      MappingManager._removeFromTree (package:yaga/managers/isolateable/mapping_manager.dart:68)
#4      MappingManager.handleMappingUpdate (package:yaga/managers/isolateable/mapping_manager.dart:53)
#5      new MappingManager.<anonymous closure> (package:yaga/managers/isolateable/mapping_manager.dart:35)
#6      _RootZone.runUnaryGuarded (dart:async/zone.dart:1546)
#7      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
#8      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271)
#9      _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart:123)
#10     _WhereStream._handleData (dart:async/stream_pipe.dart:195)
#11     _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:153)
#12     _RootZone.runUnaryGuarded (dart:async/zone.dart:1546)
#13     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
#14     _DelayedData.perform (dart:async/stream_impl.dart:591)
#15     _StreamImplEvents.handleNext (dart:async/stream_impl.dart:706)
#16     _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:663)
#17     _microtaskLoop (dart:async/schedule_microtask.dart:40)
#18     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49)
#19     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:120)
#20     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:185)

2021-04-29 07:23:18.451062 WARNING SettingsManagerBridge - BoolPreference {
  value=false,
  key=category:view:recursive,
  title=Load Recursively,
  enabled=true,
}
2021-04-29 07:23:18.451428 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (start)
2021-04-29 07:23:18.453580 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (received list - #images: 0)
2021-04-29 07:23:18.453894 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - manager key)
2021-04-29 07:23:18.453964 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - event key)
2021-04-29 07:23:18.893781 WARNING SettingsManagerBridge - BoolPreference {
  value=true,
  key=category:view:recursive,
  title=Load Recursively,
  enabled=true,
}
2021-04-29 07:23:18.894030 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (start)
2021-04-29 07:23:18.898110 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (received list - #images: 1)
2021-04-29 07:23:18.898777 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - manager key)
2021-04-29 07:23:18.899176 WARNING FileListLocalManager - f7eb9410-a8aa-11eb-8442-4911d42d0ae5 (done - event key)
dln949 commented 3 years ago

I have screen after screen that looks like this (but then I also have screens that display just fine - sometimes a picture looks like what you see here, and other times the same picture displays just fine) 10cd3039-1f1d-4fb4-ad72-8a57806e92b0

vauvenal5 commented 3 years ago

@robert-wettstaedt your case is easy: In your Home View Settings (your 3rd screen) set the path to the respective Nextcloud path. For example 'https://your.cloud.com/Pictures/Camera' and everything should be fine. See currently your Home View points to a local device path and can therefor only show local files.

@dln949 you have an entirely different problem. In your case the list of files is complete. They just do not get updated once the preview has been downloaded. I moved your issue to #132.

Zeromax commented 3 years ago

I can confirm the error stated above.

  1. Fresh installation
  2. Connected to my Nextcloud
  3. Select photo folder like in your first setup description
  4. Nothing is shown

In yaga isolated log I see a list of my folders and also the Recursive flag works. Yaga tries to load the Pictures, but stops working because of this error:

2021-07-10 10:28:20.758617 INFO NextCloudService - Mapping Photos/2019/05/IMG_20190514_182246.jpg
2021-07-10 10:28:20.758830 INFO NextCloudService - Mapping Photos/2019/05/IMG_20190514_182255.jpg
2021-07-10 10:28:20.758960 SEVERE NextCloudService - LateInitializationError: Field 'lastModified' has not been initialized.
2021-07-10 10:28:20.758960 SEVERE NextCloudService - #0      NextCloudService.list.<anonymous closure> (package:yaga/services/isolateable/nextcloud_service.dart)
#1      _MapStream._handleData (dart:async/stream_pipe.dart:213)
#2      _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:153)
#3      _RootZone.runUnaryGuarded (dart:async/zone.dart:1546)
#4      _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
#5      _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271)
#6      _ForwardingStreamSubscription._add (dart:async/stream_pipe.dart:123)
#7      _WhereStream._handleData (dart:async/stream_pipe.dart:195)
#8      _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:153)
#9      _RootZone.runUnaryGuarded (dart:async/zone.dart:1546)
#10     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
#11     _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271)
#12     _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:733)
#13     _StreamController._add (dart:async/stream_controller.dart:607)
#14     _StreamController.add (dart:async/stream_controller.dart:554)
#15     _RootZone.runUnaryGuarded (dart:async/zone.dart:1546)
#16     _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341)
#17     _IterablePendingEvents.handleNext (dart:async/stream_impl.dart:538)
#18     _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:663)
#19     _microtaskLoop (dart:async/schedule_microtask.dart:40)
#20     _startMicrotaskLoop (dart:async/schedule_microtask.dart:49)
#21     _runPendingImmediateCallback (dart:isolate-patch/isolate_patch.dart:120)
#22     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:402)
#23     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:426)
#24     _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:184)

It looks like the field lastModified is not set. Is it possible to fix this?

vauvenal5 commented 2 years ago

Yes I have to handle missing lastModified gracefully. The question is when missing, what value to set there. See some of the sorting is based on last modified.

Zeromax commented 2 years ago

I think then fall back to file created date. IMO this is the most reasonable value to use if date modified is not set.

Or do you have no date then?

zerodogg commented 2 years ago

I'd say last modified > creation date > 0, falling back to a zero value for sorting purposes when all else fails. Even just treating it as 0 would be an improvement over the current crash, since that makes the app unusable.