FileNotFoundError: [Errno 2] No such file or directory: '/data/media/user_2/

[danielnetop]

Hi, I'm setting up a new phone and the android app is not working, however on my current phone I don't get any kind of error when using the etesync application. Posting on the server repo because this is the error I'm getting on the server with the debug flag as true.

Not sure what's wrong, can you help with it?

Full log of one of the collections:

Internal Server Error: /api/v1/collection/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/item/
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 179, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 114, in view
    return self.dispatch(request, *args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "./django_etebase/views.py", line 310, in list
    "data": serializer.data,
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 760, in data
    ret = super().data
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 260, in data
    self._data = self.to_representation(self.instance)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 677, in to_representation
    return [
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 678, in <listcomp>
    self.child.to_representation(item) for item in iterable
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 529, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 529, in to_representation
    ret[field.field_name] = field.to_representation(attribute)
  File "/usr/local/lib/python3.9/site-packages/rest_framework/relations.py", line 533, in to_representation
    return [
  File "/usr/local/lib/python3.9/site-packages/rest_framework/relations.py", line 534, in <listcomp>
    self.child_relation.to_representation(value)
  File "./django_etebase/serializers.py", line 119, in to_representation
    with open(obj.chunkFile.path, "rb") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/data/media/user_2/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/si/vSh_iSBHJbUqqtEW1YrNJolBGQny0HWc97-yjgWhk'
[pid: 40|app: 0|req: 2/2] 172.17.0.1 () {42 vars in 721 bytes} [Mon Nov 30 23:52:04 2020] GET /api/v1/collection/tqw-R3i_3-Zz9xubJjB9gqv-u0CHLsNR/item/? => generated 14817 bytes in 361 msecs (HTTP/1.0 500) 6 headers in 202 bytes (1 switches on core 0)

[tasn]

Hey,

The main issue here is: why is there no such file or directory? Are you maybe using Docker and have not marked the data volume to be saved? If so, which Docker image is it? Please report it to them! I'm pretty sure that's what's going on.

Though also interesting is: why is it not showing up in the Android client? It shows like it's a real server error which should make the client complain. :|

-- Tom

[danielnetop]

Hi, I'm using the "official" one by victorrds, I've comment on a similar ticket there, when upgrade to etebase (etesync 2.0) I didn't know we'd needed to save that user information, it wasn't specified anywhere, I'll probably try to migrate from the app where it works, and will try to migrate all that data into another user and see if it solves. Thank you

[tasn]

Yeah, so it's probably that. I think He updated the README about it.

[victor-rds]

Sorry for the late comment, I was running my own backup here, for anyone using my image and having this error:

This was not caused by a "minor upgrade", this was caused by a mistake from my image, not server problem it was my fault.

When I first created the Dockerfiles and images there was no mention in the server repository about the MEDIA_ROOT directory role in the data persistence, and when looked into the /etebase_server/settings.py file:

# Static files (CSS, JavaScript, Images)
# https://docs.djangoproject.com/en/3.0/howto/static-files/

STATIC_URL = "/static/"
STATIC_ROOT = os.environ.get("DJANGO_STATIC_ROOT", os.path.join(BASE_DIR, "static"))

MEDIA_ROOT = os.environ.get("DJANGO_MEDIA_ROOT", os.path.join(BASE_DIR, "media"))
MEDIA_URL = "/user-media/"

And made the wrong assumption that this directory contained "web related files" and chose to keep it in the /srv/etebase right besides the "static" folder, and that was a mistake.

Fast forward to the issue #42, thanks to @thgoebel, we discussed about the MEDIA_ROOT, and @tasn explained that the actual data goes there and that database only holds metadata, he also fixed the docs on the server repository, but I already published my images, I should've asked Tom before making my on baseless assumptions.

This will not happen again, and is always good to backup before upgrade.

[danielnetop]

@tasn I've just migrated to a new user, using the app, import from the old to the new one and I'm still getting this error when going to pim.etesync.com

FileNotFoundError at /api/v1/collection/AcyatV7xOjdPlHp44KgsewwmL8yDm7Ig/item/
[Errno 2] No such file or directory: '/data/media/user_3/AcyatV7xOjdPlHp44KgsewwmL8yDm7Ig/XM/pTe844rm2QDa1380YLZgQ7-zl3Webux2VTOdpYtgc'

And right now I see some information on that folder for the user, but not that specific folder

[tasn]

If I had to guess: you haven't put that folder on persistent storage still, so it again reset between docker restarts. You need to have it on persistent storage, the same way the database is.

[danielnetop]

I think I know what happened here, I deleted the calendar on the app and then I deleted the folder on the server. Because looking at the current folders I have the amount of folders corresponding to my number of calendars and contact books

[tasn]

Don't manually delete data behind Etebase's back, it's a recipe for trouble. :P

[danielnetop]

Yeah, but since I deleted the calendar on the app I thought that it was supposed to forget about all that reference :/

But the directories stay there and on the metadata as well apparently

[danielnetop]

Going to create a new user and this time to not mess with it.

If I use the admin dashboard to delete the old users, is it safe to delete them?

[danielnetop]

Or is it better to start fresh and remove the old database and folders?

[victor-rds]

Do not create new users, start another server instance, with a clean database and media directory.

[tasn]

Better start from scratch, but there's no problem with deleting users from the dashboard.

[danielnetop]

Thank you, will do that and I'll comment back afterwards. Thanks

[danielnetop]

Hi @tasn, @victor-rds Already finished the migration to a new directory and server.

It's apparently working well 😄

Thank you so much for your help, and also help me understand as well what happened.

Going now update my backup script to include the media directory.

[tasn]

Great to hear, and thanks for the update!