search

ggogel / seafile-containerized

A fully containerized deployment of Seafile for Docker, Docker Swarm and Kubernetes.
150 stars 21 forks source link

Not working: SSO, GC, EMAIL #264

Closed kpinarci closed 6 days ago

kpinarci commented 1 month ago

I have set up this Seafile instance, intending to use it in a production environment. However, I have encountered several issues.

  1. SSO I integrated Seafile with Keycloak for SSO. When attempting to log in via SSO, I am redirected to Keycloak. After successfully logging in on Keycloak, I am redirected back to Seafile, but I receive the following error message: "An error has occurred. Please contact the administrator."

    In the Seahub log, I see the following entry:

    24-07-27 00:46:47,709 [ERROR] seahub.oauth.views:154 oauth_callback HTTPSConnectionPool(host='keycloak.example.com', port=443): Max retries exceeded with url: /realms/HomeLab/protocol/openid-connect/token (Caused by NameResolutionError("<urllib3.connection.HTTPSConnection object at 0x7339e23e68f0>: Failed to resolve 'keycloak.example.com' ([Errno -3] Try again)"))

    Yes. The proxy headers are set correctly and I have tried different settings multiple times, but the issue persists. Yes. I added the keycloak IP and domain into seahub /etc/hosts.

  2. Seafile GC Not Working I transferred 50 GB of files from Synology to Seafile using rclone. Due to an unspecified issue with rclone, the server ended up occupying approximately 100 GB. However, Seafile only shows 40 GB in use. Using the command docker system df -v, I discovered that seafile-data is using 92.55 GB. The recycle bin is empty. I attempted to start the garbage collection manually but unsuccessfully.

    Local Volumes space usage:
VOLUME NAME               LINKS     SIZE
seafile_seafile-data      2         92.55GB
seafile_seafile-mariadb   1         301.8MB
seafile_seahub-avatars    2         21.03kB
seafile_seahub-custom     2         0B
seafile_caddy-config      1         3.34kB
seafile_caddy-data        1         8.717kB

    Inside the Seahub container, I executed bash /opt/seafile/seafile-server-latest/seahub/scripts/seaf-gc.sh and received the following error message:

    awk: /opt/seafile/seafile-server-latest/seahub/scripts/pro/python/seafevents/seafevents_api.py: No such file or directory
Starting seafserv-gc, please wait ...
/opt/seafile/seafile-server-latest/seahub/scripts/seaf-gc.sh: line 108: /opt/seafile/seafile-server-latest/seahub/scripts/seafile/bin/seafserv-gc: No such file or directory
seafserv-gc run done

  3. Sending Email Notifications, not working. As described in the documentation, I added the SMTP configuration for sending email notifications. However, emails are not being sent.

Has anyone else encountered these issues, or have I configured something incorrectly? I would appreciate any feedback.

ggogel commented 1 month ago

For 1 and 3 see #211

For 2, gc needs to be executed in the seafile-server (backend) container not seahub (frontend). You should start gc with the included script /scripts/gc.sh, as described in the Wiki. This will keep the container alive while the gc is running. Otherwise, the container will detect that seafile-server stopped and will restart.

kpinarci commented 1 month ago

Thank you very much for the prompt response.

Unfortunately, I overlooked the wiki, but I appreciate the pointer.

  1. I was able to resolve the email issue by configuring the external network. Thanks!
  2. I set up SSO for Seahub using the external network as described in the wiki. Now I can reach the Keycloak server, but the login still doesn’t work. I followed the OIDC configuration from this guide. Please see the logs for details.
  3. GC: I executed the command docker exec seafile-server /scripts/gc.sh as described. Unfortunately, it hangs and doesn’t progress even after 8 hours. When I interrupt the command and restart the compose, the docker volume still occupies around 92.54 GB.

seafile-hub logs:

seafile-hub        | 2024-07-27 20:40:17,171 [ERROR] django.request:241 log_response Internal Server Error: /oauth/callback/
seafile-hub        | Traceback (most recent call last):
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/thirdpart/requests/models.py", line 971, in json
seafile-hub        |     return complexjson.loads(self.text, **kwargs)
seafile-hub        |   File "/usr/local/lib/python3.10/json/__init__.py", line 346, in loads
seafile-hub        |     return _default_decoder.decode(s)
seafile-hub        |   File "/usr/local/lib/python3.10/json/decoder.py", line 337, in decode
seafile-hub        |     obj, end = self.raw_decode(s, idx=_w(s, 0).end())
seafile-hub        |   File "/usr/local/lib/python3.10/json/decoder.py", line 355, in raw_decode
seafile-hub        |     raise JSONDecodeError("Expecting value", s, err.value) from None
seafile-hub        | json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
seafile-hub        |
seafile-hub        | During handling of the above exception, another exception occurred:
seafile-hub        |
seafile-hub        | Traceback (most recent call last):
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/thirdpart/django/core/handlers/exception.py", line 55, in inner
seafile-hub        |     response = get_response(request)
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/thirdpart/django/core/handlers/base.py", line 197, in _get_response
seafile-hub        |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/seahub/oauth/views.py", line 88, in _decorated
seafile-hub        |     return func(request)
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/seahub/oauth/views.py", line 158, in oauth_callback
seafile-hub        |     user_info_json = user_info_resp.json()
seafile-hub        |   File "/opt/seafile/seafile-server-latest/seahub/thirdpart/requests/models.py", line 975, in json
seafile-hub        |     raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
seafile-hub        | requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Browser

Page unavailable
Sorry, but the requested page is unavailable due to a server hiccup.

Our engineers have been notified, so check back later.

brower-url: 

https://cloud2.example.de/oauth/callback/?state=N3khShYhcmIeQhBOlgBUCFQuUAc9tt&session_state=10a687b1-cbbb-45c4-a640-1ee87726bfa6&iss=https%3A%2F%2Fkeycloak.example.de%2Frealms%2FHomeLab&code=1685bc26-6e52-4267-a9ef-6701c3171158.10a687b1-cbbb-45c4-a640-1ee87726bfa6.af088220-372e-4713-8974-77264a31bf69

GC seafile-server logs:

eafile CE: Stop Seafile to perform offline garbage collection.
Stopping socat...

Stopping seafile server ...
Done.
Waiting for the server to shut down properly...
Kill remaining processes with SIGKILL signal.

Starting seafserv-gc, please wait ...
2024-07-27 22:39:15 gc-core.c(590): GC version 1 repo shared(f0b3e14c-594b-4dc8-a6e0-60aece4e6ecb)
2024-07-27 22:39:15 gc-core.c(417): GC started. Total block number is 1422.
2024-07-27 22:39:15 gc-core.c(41): GC index size is 1024 Byte.
2024-07-27 22:39:15 gc-core.c(442): Populating index.
2024-07-27 22:39:15 gc-core.c(187): Populating index for repo f0b3e14c.
2024-07-27 22:39:16 gc-core.c(271): Traversed 191 commits, 1422 blocks.
2024-07-27 22:39:16 gc-core.c(460): Scanning and deleting unused blocks.
2024-07-27 22:39:16 gc-core.c(500): GC finished. 1422 blocks total, about 1422 reachable blocks, 0 blocks are removed.

2024-07-27 22:39:16 gc-core.c(590): GC version 1 repo test(da59767b-6574-4fcc-9d16-e49bc3848b2d)
2024-07-27 22:39:16 gc-core.c(395): No blocks. Skip GC.

2024-07-27 22:39:16 gc-core.c(590): GC version 1 repo it-stuff(da437deb-df45-47a8-aadc-ad5951d5b179)
2024-07-27 22:39:16 gc-core.c(417): GC started. Total block number is 16016.
2024-07-27 22:39:16 gc-core.c(41): GC index size is 8008 Byte.
2024-07-27 22:39:16 gc-core.c(442): Populating index.
2024-07-27 22:39:16 gc-core.c(187): Populating index for repo da437deb.
ggogel commented 1 month ago

You're welcome.

Regarding the garbage collection, the logs look normal. I think cleaning up this much data just takes a significant amount of time. Creating a new library might be faster.

In the the logs regarding OAuth it looks like the callback from your Keycloak returns a broken JSON. The first property doesn't seem to have a value. I'm using OAuth with Entra ID and it works without any problems.

kpinarci commented 1 month ago

@ggogel you were right—the garbage collection ran, which likely took a long time due to the large amount of data. Unfortunately, it didn’t free up any disk space. Despite my efforts, I haven’t been able to clean up the Seafile data. There’s still around 92GB of used space, even though only 40GB is shown in the Seafile UI. I plan to remove and redeploy the volumes.

Could you offer any advice on the best way to transfer data from Synology to Seafile? I tried using rclone, but it ran continuously without completing.

Additionally, the issue with Keycloak persists. I’ve intercepted and examined the JWT, and everything seems fine so far. I have several other tools that work perfectly with Keycloak, but Seafile is still problematic. I would appreciate it if anyone could share their working configuration for reference.

Thank you again for all your support.

ggogel commented 1 month ago

Is your config not formatted correctly?

Here is mine:

ENABLE_OAUTH = True
OAUTH_ENABLE_INSECURE_TRANSPORT = True
OAUTH_CLIENT_ID = "***"
OAUTH_CLIENT_SECRET = "***"
OAUTH_REDIRECT_URL = 'https://seafile.gogel.me/oauth/callback/'
OAUTH_PROVIDER_DOMAIN = 'login.microsoftonline.com'
OAUTH_AUTHORIZATION_URL = 'https://login.microsoftonline.com/***/oauth2/v2.0/authorize'
OAUTH_TOKEN_URL = 'https://login.microsoftonline.com/***/oauth2/v2.0/token'
OAUTH_USER_INFO_URL = 'https://graph.microsoft.com/oidc/userinfo'
OAUTH_SCOPE = ["User.Read", "profile", "email", "openid",]
OAUTH_ATTRIBUTE_MAP = {
        "email": (True, "email"),
        "id": (False, "not used"), 
        "name": (False, "name")
}
kpinarci commented 1 month ago

Thank you! It now works perfectly with Keycloak. I will share my working Seafile-Keycloak configuration in case anyone else needs it.”

# SSO Keycloak settings
ENABLE_OAUTH = True
OAUTH_ENABLE_INSECURE_TRANSPORT = True
ACTIVATE_AFTER_REGISTRATION = True
ENABLE_SIGNUP = True
OAUTH_CREATE_UNKNOWN_USER = True
OAUTH_ACTIVATE_USER_AFTER_CREATION = True
OAUTH_ENABLE_INSECURE_TRANSPORT = True

OAUTH_CLIENT_ID = "seafile******"
OAUTH_CLIENT_SECRET = "******"
OAUTH_REDIRECT_URL = 'https://seafile.example.de/oauth/callback/'

OAUTH_PROVIDER_DOMAIN   = 'keycloak.example.de'
OAUTH_AUTHORIZATION_URL = 'https://keycloak.example.de/realms/homelab/protocol/openid-connect/auth'
OAUTH_TOKEN_URL         = 'https://keycloak.example.de/realms/homelab/protocol/openid-connect/token'
OAUTH_USER_INFO_URL     = 'https://keycloak.example.de/realms/homelab/protocol/openid-connect/userinfo'
OAUTH_SCOPE = ["profile", "email", "openid"]
OAUTH_ATTRIBUTE_MAP = {
    "email": (True, "email"),
    "id":    (False, "not used"),
    "name":  (False, "name")
}
ggogel commented 1 month ago

Glad to hear that you got it working :)

Regarding getting your files into Seafile. I think for the rclone seafile backend you need to enable seafdav. Using seafdav, you need to authenticate with the local user password, not OAuth.

Another way would be using one of the clients that Seafile offers, like SeaDrive. You could install SeaDrive on your computer and then copy all the files from your synology there.