search

usememos / memos

An open-source, lightweight note-taking solution. The pain-less way to create your meaningful notes. Your Notes, Your Way.
https://usememos.com
MIT License
34.05k stars 2.46k forks source link

Recurring bug with docker container: failed to get user: interrupted (9) #3922

Closed ozls closed 3 days ago

ozls commented 1 month ago

Describe the bug

Hello, I'm selfhosting memos (with sqlite db) and I have a recurring bug that I don't know how to solve. From time to time, I will get a popup saying failed to get user: interrupted (9) on the web interface. After this, the interface displays a blank page. In the logs of the docker container I get those lines:

2024/09/13 16:44:59 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/09/13 16:44:59 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="failed to get user: interrupted (9)"
2024/09/13 16:44:59 ERROR server error method=/memos.api.v1.MemoService/ListMemoProperties error="failed to get user: interrupted (9)"
2024/09/13 16:45:00 ERROR server error method=/memos.api.v1.MemoService/ListMemos error="failed to get user: interrupted (9)"
2024/09/13 16:45:00 ERROR server error method=/memos.api.v1.MemoService/ListMemoTags error="failed to get user: interrupted (9)"
2024/09/13 16:45:07 ERROR server error method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile error="failed to get user: interrupted (9)"
2024/09/13 16:45:07 ERROR server error method=/memos.api.v1.AuthService/GetAuthStatus error="failed to get user: interrupted (9)"

Most of the time it is solved by restarting the container, but it can be very inconvenient if I'm not home. I have no clue what triggers it.

Any idea what is the problem is? Can I do anything more to help find the root cause?

Steps to reproduce

I have no clue

The version of Memos you're using.

v0.22.5

Screenshots or additional context

No response

RoccoSmit commented 1 month ago

What type of resources (attachments) are you viewing? Images? And, do you get these errors when looking at new resources (ones you haven't looked at before) or does it happen with for e.g. reload the home page? Did you have this issue in v0.22.4?

ozls commented 1 month ago

In this case I have just one resource in the latest memos, a screenshot. I only ever attach photos or screenshots. It had been displayed before without issue, I don't know what changed. It definitely happened before 0.22.5.

RoccoSmit commented 1 month ago

I was wondering if the new thumbnail feature was causing this, but if it happens before 0.22.5 then it's something else

sherlcok314159 commented 1 month ago

0.22.2 also has this problem

RoccoSmit commented 1 month ago

I've not been able to reproduce this, if I had to take a stab in the dark it would be something to do with your browser telling the server to close the connection (context), the server being mid db activity and not unlocking the db and the subsequent db requests not being allowed due to the lock e.g. getting the user. but the code looks like its handling the connection closure so not sure and also not sure why your browser would tell the server to close the connection

image

Are you viewing large images?

ozls commented 1 month ago

I had this issue with both my laptop (Firefox) and my phone (Vanadium, a Chromium fork) so I don't think it comes from the browser. My images are all below 3MB.

But now that I think about it, it might have something to do with the way the container gets stopped. I run watchtower every night for updates, and before 0.22.5 I had memos on the stable tag, which means it got restarted frequently. Indeed memos has been running without issue for a few days, but I just managed to make it crash:

  1. Browse my memos a little bit, load resources without issue
  2. Stop and remove the container
  3. Start up the container
  4. Browse memos as fast as possible, as soon as the web UI was available

I managed to view a few images without issue, but one specific memos with three pictures made it crash. The images never appeared, no more memos could be loaded, the same log lines appeared and a blank screen was displayed on refresh.

Just after this I stopped, removed and started the container once more. I just waited a little bit, and then I was able to browse memos without any crash. I hope this helps!

remsch commented 1 month ago

I share the same problem (complete overlap, in my opinion), running the container version of Memos (v.0.22.5) on Coolify v.4.0.0-beta.325. I have two separate instances on two subdomains, both on SQLite, one runs fine, while the other doesn't. The problems occurred when I added another user I since deleted. Since deletion didn't resolve the problem, I suspect another issue.

RoccoSmit commented 1 month ago

@ozls do you always have the same successful web request before the first ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary or are they different

sherlcok314159 commented 1 month ago

Any progress? You may need to upload several images (100~ kb or 2 MB+) to reproduce this problem. I tried several versions and the problem remains.

ozls commented 1 month ago

@RoccoSmit The bug did not happen since I stopped removing and restarting the container on a regular basis so I'm confident I found the trigger. Nevertheless I made it happen once more to try to answer your question. I had to click "load more" a few times, and then the the memo with three pictures made it crash again. Here is the complete log from container start to crash:


███╗   ███╗███████╗███╗   ███╗ ██████╗ ███████╗
████╗ ████║██╔════╝████╗ ████║██╔═══██╗██╔════╝
---
██╔████╔██║█████╗  ██╔████╔██║██║   ██║███████╗
██║╚██╔╝██║██╔══╝  ██║╚██╔╝██║██║   ██║╚════██║
Server profile
██║ ╚═╝ ██║███████╗██║ ╚═╝ ██║╚██████╔╝███████║
version: 0.22.5
data: /var/opt/memos
╚═╝     ╚═╝╚══════╝╚═╝     ╚═╝ ╚═════╝ ╚══════╝
dsn: /var/opt/memos/memos_prod.db
addr: 
port: 5230
mode: prod
driver: sqlite
---
Version 0.22.5 has been started on port 5230
---
See more in:
👉Website: https://usememos.com
👉GitHub: https://github.com/usememos/memos
---
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.InboxService/ListInboxes
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:24 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:25 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:25 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:25 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:25 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:25 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:26 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:26 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:27 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:27 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:27 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:27 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:28 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:29 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:29 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:29 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:30 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:30 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:30 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:30 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:31 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:31 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:31 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/09/25 06:40:31 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/09/25 06:40:31 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/09/25 06:40:34 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="failed to get user: interrupted (9)"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="failed to get user: interrupted (9)"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: interrupted (9)"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.MemoService/ListMemoProperties error="failed to get user: interrupted (9)"
2024/09/25 06:40:34 ERROR server error method=/memos.api.v1.MemoService/ListMemoTags error="failed to get user: interrupted (9)"
2024/09/25 06:40:37 ERROR server error method=/memos.api.v1.MemoService/ListMemos error="failed to get user: interrupted (9)"
2024/09/25 06:40:40 ERROR server error method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile error="failed to get user: interrupted (9)"
2024/09/25 06:40:40 ERROR server error method=/memos.api.v1.AuthService/GetAuthStatus error="failed to get user: interrupted (9)"
2024/09/25 06:40:44 ERROR server error method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile error="failed to get user: interrupted (9)"
2024/09/25 06:40:44 ERROR server error method=/memos.api.v1.AuthService/GetAuthStatus error="failed to get user: interrupted (9)"
Jofroop commented 1 month ago

I have the same problem. For me it seems to be triggered right after posting a memo but I'm not completely sure, as I haven't tested it enough yet. I only restart my container when this issue happens and I never move it so I don't think that that's the trigger for me.

I also get the blank page when memos stops working, the failed to get user: interrupted (9), and whenever I try to log in after this bug happens it says Failed to find user by userame <my-username>.

I've also installed Memos through Docker, and the problem persists until I restart the Docker container, and quickly returns. I'm using v0.22.5. Here are my logs when the error happens:


version: 0.22.5

███╗   ███╗███████╗███╗   ███╗ ██████╗ ███████╗
████╗ ████║██╔════╝████╗ ████║██╔═══██╗██╔════╝
██╔████╔██║█████╗  ██╔████╔██║██║   ██║███████╗
██║╚██╔╝██║██╔══╝  ██║╚██╔╝██║██║   ██║╚════██║
██║ ╚═╝ ██║███████╗██║ ╚═╝ ██║╚██████╔╝███████║
╚═╝     ╚═╝╚══════╝╚═╝     ╚═╝ ╚═════╝ ╚══════╝
data: /var/opt/memos
dsn: /var/opt/memos/memos_prod.db
addr:
port: 5230
mode: prod
driver: sqlite
---
Version 0.22.5 has been started on port 5230
---
See more in:
👉Website: https://usememos.com
👉GitHub: https://github.com/usememos/memos
---
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.UserService/GetUserAvatarBinary
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.InboxService/ListInboxes
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.UserService/ListUserAccessTokens
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:25 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:29 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/10/01 20:26:29 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/10/01 20:26:29 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:29 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:26:29 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/10/01 20:26:50 INFO OK method=/memos.api.v1.WorkspaceSettingService/SetWorkspaceSetting
2024/10/01 20:27:38 INFO OK method=/memos.api.v1.IdentityProviderService/ListIdentityProviders
2024/10/01 20:27:40 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/10/01 20:27:40 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/10/01 20:27:40 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:27:40 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:27:40 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/10/01 20:27:53 INFO OK method=/memos.api.v1.UserService/ListUserAccessTokens
2024/10/01 20:27:54 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/10/01 20:27:54 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/10/01 20:27:54 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:27:54 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:27:54 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/10/01 20:27:57 INFO OK method=/memos.api.v1.WebhookService/ListWebhooks
2024/10/01 20:28:00 INFO OK method=/memos.api.v1.UserService/ListUsers
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.WorkspaceService/GetWorkspaceProfile
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.AuthService/GetAuthStatus
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.WorkspaceSettingService/GetWorkspaceSetting
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.UserService/GetUserSetting
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.UserService/GetUserAvatarBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.UserService/GetUserAvatarBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.InboxService/ListInboxes
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:28:26 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/10/01 20:31:25 INFO OK method=/memos.api.v1.InboxService/ListInboxes
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemoProperties
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/10/01 20:33:00 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/10/01 20:33:01 INFO OK method=/memos.api.v1.MemoService/ListMemoTags
2024/10/01 20:33:01 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:33:01 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:33:01 INFO OK method=/memos.api.v1.MemoService/ListMemos
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to find memo by ID: 0x4000856790"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="rpc error: code = Internal desc = failed to get resource: context canceled"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="failed to get user: interrupted (9)"
2024/10/01 20:33:01 INFO OK method=/memos.api.v1.ResourceService/GetResourceBinary
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.MemoService/ListMemoProperties error="failed to get user: interrupted (9)"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.ResourceService/GetResourceBinary error="failed to get user: interrupted (9)"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.MemoService/ListMemoProperties error="failed to get user: interrupted (9)"
2024/10/01 20:33:01 ERROR server error method=/memos.api.v1.MemoService/ListMemoTags error="failed to get user: interrupted (9)"
2024/10/01 20:33:07 ERROR server error method=/memos.api.v1.MemoService/CreateMemo error="failed to get user: interrupted (9)"
2024/10/01 20:33:10 ERROR server error method=/memos.api.v1.MemoService/CreateMemo error="failed to get user: interrupted (9)"```
Kellermaan commented 1 month ago

also get the same error failed to get user: interrupted (9) is it fixed now?

Kellermaan commented 1 month ago

also get the same error failed to get user: interrupted (9) is it fixed now?

And when I get this error, I need to log in again.

RoccoSmit commented 4 weeks ago

I tried, but could not reproduce the issue on my end for debugging

Kellermaan commented 3 weeks ago

I tried, but could not reproduce the issue on my end for debugging

you are welcomed to use my instance for debugging.

Kellermaan commented 3 weeks ago

do we have any update on this?

Jofroop commented 3 weeks ago

I have no evidence but I suspect it may be connected to the 'explore' page. I noticed the other day that loading it crashed the instance. However, I've been getting almost no crashes recently. Perhaps the issue mostly fixes itself after some time?

Kellermaan commented 3 weeks ago

I have no evidence but I suspect it may be connected to the 'explore' page. I noticed the other day that loading it crashed the instance. However, I've been getting almost no crashes recently. Perhaps the issue mostly fixes itself after some time?

make sense. it look all fine until I click the explore page.

sherlcok314159 commented 3 days ago

Why closing it? It is an important bug!

RoccoSmit commented 2 days ago

Still not able to replicate to debug/fix

What are the memory resources allocated to your docker instances. OP mentioned their issue stopped when they stopped restarting the container. Wondering if even if you don't restart your container, if the docker container runs out of memory and docker then restarts the container in the background

ozls commented 1 day ago

Not being able to reproduce is not a valid reason to close an issue imo, as the issue is still here. But your project, your rules! I want to clarify that in my case memory is no issue, and also want to highlight that just restarting never made anything crash for me. What made it crash was, as I understand it: restarting + super fast loading of memos with pictures in my browser. Just as an additional test, I tried the proven "scroll super fast and load everything" method again, with the container already running for a long time, and nothing crashed.