Unable to view photos

[robertcontois]

When clicking on a photo to view, the lightbox should open showing the larger version of the photo. The front end makes a call to /api/photos/aadd85dc9c9c91dd65d319a806b94a7b1/ and returns a 500 error and the lightbox sits waiting to load the image.

The following error is displayed in the backend log:

Running backend server... [2021-01-08 14:41:26 +0000] [309] [INFO] Starting gunicorn 20.0.4 [2021-01-08 14:41:26 +0000] [309] [INFO] Listening at: http://0.0.0.0:8001 (309) [2021-01-08 14:41:26 +0000] [309] [INFO] Using worker: gevent [2021-01-08 14:41:26 +0000] [313] [INFO] Booting worker with pid: 313 [2021-01-08 14:41:26 +0000] [314] [INFO] Booting worker with pid: 314 14:41:37 Worker rq:worker:86d2de7e24b24bf0a4a132ffca5af635: started, version 1.6.1 14:41:37 Subscribing to channel rq:pubsub:86d2de7e24b24bf0a4a132ffca5af635 14:41:37 Listening on default... 14:41:37 Cleaning registries for queue: default Not Found: /api/jobs/ Not Found: /api/jobs/ Bad Request: /api/nextcloud/listdir/ 127.0.0.1 - - [2021-01-08 14:43:48] "POST /build/ HTTP/1.1" 200 139 0.044669 Internal Server Error: /api/photos/aadd85dc9c9c91dd65d319a806b94a7b1/ Traceback (most recent call last): File "/miniconda/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner response = get_response(request) File "/miniconda/lib/python3.8/site-packages/django/core/handlers/base.py", line 115, in _get_response response = self.process_exception_by_middleware(e, request) File "/miniconda/lib/python3.8/site-packages/django/core/handlers/base.py", line 113, in _get_response response = wrapped_callback(request, callback_args, callback_kwargs) File "/miniconda/lib/python3.8/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view return view_func(*args, kwargs) File "/miniconda/lib/python3.8/site-packages/rest_framework/viewsets.py", line 125, in view return self.dispatch(request, *args, *kwargs) File "/miniconda/lib/python3.8/site-packages/rest_framework/views.py", line 509, in dispatch response = self.handle_exception(exc) File "/miniconda/lib/python3.8/site-packages/rest_framework/views.py", line 469, in handle_exception self.raise_uncaught_exception(exc) File "/miniconda/lib/python3.8/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception raise exc File "/miniconda/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch response = handler(request, args, kwargs) File "/miniconda/lib/python3.8/site-packages/rest_framework_extensions/cache/decorators.py", line 54, in inner return this.process_cache_response( File "/miniconda/lib/python3.8/site-packages/rest_framework_extensions/cache/decorators.py", line 83, in process_cache_response response = view_method(view_instance, request, *args, *kwargs) File "/code/api/views.py", line 193, in retrieve return super(PhotoViewSet, self).retrieve(args, **kwargs) File "/miniconda/lib/python3.8/site-packages/rest_framework/mixins.py", line 56, in retrieve return Response(serializer.data) File "/miniconda/lib/python3.8/site-packages/rest_framework/serializers.py", line 548, in data ret = super().data File "/miniconda/lib/python3.8/site-packages/rest_framework/serializers.py", line 246, in data self._data = self.to_representation(self.instance) File "/miniconda/lib/python3.8/site-packages/rest_framework/serializers.py", line 515, in to_representation ret[field.field_name] = field.to_representation(attribute) File "/miniconda/lib/python3.8/site-packages/rest_framework/fields.py", line 1870, in to_representation return method(value) File "/code/api/serializers.py", line 107, in get_similar_photos res = search_similar_image(obj.owner,obj) File "/code/api/image_similarity.py", line 15, in search_similar_image np.frombuffer(bytes.fromhex(photo.encoding)), dtype=np.float32) TypeError: fromhex() argument must be str, not None

[derneuere]

Related to #26

Did your scan finish or are you still scanning?

[robertcontois]

I don't see the notification that says "Scan completed" after pressing the button to scan, but the job shows in the admin area as having found 3 photos (the correct amount) and having a duration of a few seconds. This behavior of not being able to view the photo occurs in the demo site as well.

I've only been testing with 3 photos and it was working fine (I could view the photo) up until I updated dev commits from the past couple days or so into my fork.

[derneuere]

Okay. It tries to read image.embedding which is calculated in image._im2vec() but it returns none. Did your scan crash once?

[robertcontois]

I updated my error message above to include all of the lines from when the backend starts. It's a fresh install, empty database, new build. There are no error messages on the backend logs to indicate the build crashes.

[derneuere]

Could you look into your ownphotos.log in your log folder? Maybe there are more errors.

[robertcontois]

I figured out part of the problem, if you reuse the same _librephotosmedia folder with thumbnails already in it, the process for adding the new images crashes. I would expect that thumbnails would be overwritten if importing a new file with the same hash but it looks like that might not be the case. There's a lot of noise in my ownphotos.log, I'll try to post a clean version just with this issue.

[robertcontois]

Here is my ownphotos.log

2021-01-08 15:53:18,645 : directory_watcher.py : handle_new_image : 48 : INFO : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: handling image /data/IMG_3136.JPG 2021-01-08 15:53:18,662 : directory_watcher.py : handle_new_image : 90 : ERROR : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: could not load image /data/IMG_3136.JPG. reason: The 'thumbnail_big' attribute has no file associated with it. Traceback (most recent call last): File "/code/api/directory_watcher.py", line 70, in handle_new_image photo._generate_captions() File "/code/api/models.py", line 185, in _generate_captions image_path = self.thumbnail_big.path File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 56, in path self._require_file() File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 38, in _require_file raise ValueError("The '%s' attribute has no file associated with it." % self.field.name) ValueError: The 'thumbnail_big' attribute has no file associated with it. 2021-01-08 15:53:18,671 : directory_watcher.py : handle_new_image : 48 : INFO : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: handling image /data/sample1.jpg 2021-01-08 15:53:18,677 : directory_watcher.py : handle_new_image : 90 : ERROR : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: could not load image /data/sample1.jpg. reason: The 'thumbnail_big' attribute has no file associated with it. Traceback (most recent call last): File "/code/api/directory_watcher.py", line 70, in handle_new_image photo._generate_captions() File "/code/api/models.py", line 185, in _generate_captions image_path = self.thumbnail_big.path File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 56, in path self._require_file() File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 38, in _require_file raise ValueError("The '%s' attribute has no file associated with it." % self.field.name) ValueError: The 'thumbnail_big' attribute has no file associated with it. 2021-01-08 15:53:18,679 : directory_watcher.py : handle_new_image : 48 : INFO : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: handling image /data/sample4.jpg 2021-01-08 15:53:18,684 : directory_watcher.py : handle_new_image : 90 : ERROR : job e08b1a05-a8dc-4ac5-9449-8e59f98fa004: could not load image /data/sample4.jpg. reason: The 'thumbnail_big' attribute has no file associated with it. Traceback (most recent call last): File "/code/api/directory_watcher.py", line 70, in handle_new_image photo._generate_captions() File "/code/api/models.py", line 185, in _generate_captions image_path = self.thumbnail_big.path File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 56, in path self._require_file() File "/miniconda/lib/python3.8/site-packages/django/db/models/fields/files.py", line 38, in _require_file raise ValueError("The '%s' attribute has no file associated with it." % self.field.name) ValueError: The 'thumbnail_big' attribute has no file associated with it. 2021-01-08 15:53:18,685 : directory_watcher.py : scan_photos : 169 : INFO : Added 4 photos 2021-01-08 15:53:18,686 : image_similarity.py : build_image_similarity_index : 28 : INFO : builing similarity index for user admin

[derneuere]

This is the function in question. I don't generate a thumbnail, if there is already a thumbnail, which is correct. But I should add an association to the thumbnail if there is no thumbnail saved yet: https://github.com/LibrePhotos/librephotos/blob/42f5ebc4f286e3da0bcadfc05004622b46ec3f29/api/models.py#L274-L306