Give a proper error/improve documentation

[tomlawesome]

Summary

I'm trying to use the shlink backend docker image. It works, seemingly, in that I can boot into the container without errors, create/disable API keys. When I go to the web URL pointed at the container, it forwards to the expected address. However, if I try to connect to the shlink container from the web-app it just will not connect. It doesn't seem to matter what I do.

It would be great if there was some more guidance on the settings, and trouble shooting/what to look for in the documentation because it is not clear what the actual problem is as no real error code is given.

Ideally failure to connect would give some kind of error code that may hint why it has failed to connect. No response from host, bad API key etc.

https://i.ibb.co/jHVqy6w/shlink.png

  shlink-be:
    image: shlinkio/shlink:stable
    container_name: shlink-be
    restart: always
    security_opt:
      - no-new-privileges:true
    profiles:
      - isolation
    networks:
      - Mars
    ports:
      - 8120:8888
    environment:
      - SHORT_DOMAIN_HOST=<my short domain>
      - SHORT_DOMAIN_SCHEMA=https
      - PORT=8888
      - DB_DRIVER=${SHLINK_DB_DRIVER}
      - DB_NAME=${SHLINK_DB}
      - DB_USER=${SHLINK_USER}
      - DB_PASSWORD=${SHLINK_PASS}
      - DB_HOST=postgres
      - DB_PORT=${SHLINK_PORT}
      - DISABLE_TRACK_PARAM=no-track
      - DELETE_SHORT_URL_THRESHOLD=30
      - VALIDATE_URLS=true
      - INVALID_SHORT_URL_REDIRECT_TO=https://<url>/oops
      - REGULAR_404_REDIRECT_TO=https://<url>/oops
      - BASE_URL_REDIRECT_TO=https://<url>
#      - REDIS_SERVERS=redis
      - BASE_PATH=/
      - WEB_WORKER_NUM=64
      - TASK_WORKER_NUM=32
      - VISITS_WEBHOOKS=
      - DEFAULT_SHORT_CODES_LENGTH=6
      - GEOLITE_LICENSE_KEY=xEvsJeSJfO09BfJg
#      - MERCURE_PUBLIC_HUB_URL=https://merc.<url>
#      - MERCURE_INTERNAL_HUB_URL=http://mercure
#      - MERCURE_JWT_SECRET='<secret>'
      - ANONYMIZE_REMOTE_ADDR=true
      - REDIRECT_STATUS_CODE=301
      - REDIRECT_CACHE_LIFETIME=90
      - AUTO_RESOLVE_TITLES=true
      - TRACK_ORPHAN_VISITS=false

[acelaya]

For this time, you can probably check the browser's console to see what's going on. Once you know the actual error, you can probably find something here https://shlink.io/documentation/troubleshooting/

That said, that error screen could probably be improved.

What shlink-web-client does when trying to connect to a server is first calling the /rest/health endpoint. If that request fails in any way, that error screen is displayed.

The most tricky part is that, unless the request actually reached shlink, it's hard to display a more specific error.

For example, something reported many times is errors trying to connect to an instance not using https, or not using a trusted https certificate. That will be blocked by the browser, but the rejected promise does not provide the info to know anything other than the fact that the request failed.

But I guess at least the errors returned by shlink could be displayed verbatim, and fall back to a generic message otherwise.

Would you be open to contribute this improvement to the project?

[acelaya]

I'm giving it a second thought, and this might be even trickier to improve.

The possible things that could make Shlink unreachable:

• Wrong API key: the health endpoint does not require API key, which means it will not result in that screen, but continue and fail later. This needs to be addressed separately.
• Typo in the URL: This will not result in a specific error where you can tell for sure there was a typo. It would be easy to mistake with wrong DNS config, etc.
• Non matching encryption between Shlink and shlink-web-client: due to security reasons, the browser will block the requests unless both of them use unencrypted http or https with a proper certificate. Again, this does not result in an error containing the info to display. You can just check the browser's console to see the actual error. This was documented in the troubleshooting section:
  • https://shlink.io/documentation/troubleshooting/#i-see-mixed-content-errors-in-the-browsers-console
  • https://shlink.io/documentation/troubleshooting/#short-urls-work-but-i-cant-consume-shlink-from-shlink-web-client
• Something in front of Shlink is removing the CORS headers: you won't be able to consume Shlink from a web client unless the proper CORS headers are served. Shlink serves those automatically and out of the box, but sometimes people put a reverse proxy on front of it that removes them. This issue would also reflect on the browser's console, but not provide the info in code. This one is also documented: https://shlink.io/documentation/troubleshooting/#response-to-preflight-request-doesnt-pass-access-control-check-no-access-control-allow-origin-header-is-present

Due to all this, maybe the best approach is to somehow instruct how to open the console as part of the error message, or even add a button that opens it (I guess this might not be possible, or depend on the browser).

[tomlawesome]

I think maybe for someone of my skill level it's difficult to understand that some of the troubleshooting steps apply to my problem, and as you say, how to determine which problem it is.

I'm not sure it is worth the time to include a button to open console etc. I think it's reasonable to expect users to do their own debugging, to research the topic and learn how. I think including the key content of your posts above in a documentation section + a brief mention of the console as a debugging tool would be enough to signpost those with less knowledge to how they can figure out the problem and then use the existing troubleshooting steps to help resolve the issue.

Perhaps the Troubleshooting section would also benefit from topics headings that are digestible at a lower knowledge level.

E.g. the links shared in the post directly above this could be grouped under a heading "Shlink-web-client cannot connect to shlink server"

[acelaya]

I think including the key content of your posts above in a documentation section + a brief mention of the console as a debugging tool would be enough.

I think I agree. I'll try to get it added to the documentation somehow.

E.g. the links shared in the post directly above this could be grouped under a heading "Shlink-web-client cannot connect to shlink server"

Yep, that's a good point. I guess my brain is too technical sometimes, haha.

BTW, did any of this at least help you figure out why you couldn't connect in your particular case?

[tomlawesome]

BTW, did any of this at least help you figure out why you couldn't connect in your particular case?

Yes, thank you. I was able to determine that some of the issues wouldn't affect me (knowing they could be issues). I then looked into using the console of my browser and am able to see that the CORS Access-Control-Allow-Headers is missing. From the troubleshooting, I think that this is most likely due to my reverse proxy Traefik stripping the headers/not forwarding the headers sent by the shlink server. So I have looked into that and added the following to the headers section of my middlewares.yml file for Traefik:

    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlAllowOriginList:
          - 'https://wiki.example.com'
          - 'https://auth.example.com'
          - 'https://shlink.example.com'
          - 'https://tl1.example.com'
          - 'https://shl1.example.com'
        accessControlAllowHeaders:
          - 'content-type'
          - 'x-api-key'
        addVaryHeader: true

To find out which headers to add to 'access'ControlAllowHeaders' I opened the console, observed the CORS error, and noted the header value. I then refreshed the page and the error was there but had changed to a different header value, so I added that.

I am now at a point where I get a 404 error and I think this is my remaining issue.

It should be noted that now that the headers are added to accessControlAllowHeaders in Traefik, the web-client appears to try and connect to the server, where before it failed instantly. Likely due to not having authorisation from the server to the browser for the CORS request.

For anyone reading this thread in future, two useful resources for reading and learning: https://doc.traefik.io/traefik/v2.0/middlewares/headers/ https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Access-Control-Allow-Headers

[acelaya]

Cool thanks!

Every time this issue has been reported in the past it was by someone using precisely traefik, so I think I will have to mention that specific tool in the docs.

[tomlawesome]

Cool thanks!

Every time this issue has been reported in the past it was by someone using precisely traefik, so I think I will have to mention that specific tool in the docs.

No worries, these can also be set as labels and depends how the individual chooses to set up their Traefik. I would say most users probably use labels rather than files (i use files, with some labels).

Traefik have some good examples on their docs:

labels:
  - "traefik.http.middlewares.testheader.headers.accesscontrolallowmethods=GET,OPTIONS,PUT"
  - "traefik.http.middlewares.testheader.headers.accesscontrolalloworigin=origin-list-or-null"
  - "traefik.http.middlewares.testheader.headers.accesscontrolmaxage=100"
  - "traefik.http.middlewares.testheader.headers.addvaryheader=true"

I think correct syntax for an accessControlAllowHeaders labels would be as follows (but these are guesses!):

 - "traefik.http.middlewares.testheader.headers.accesscontrolallowheaders=content-type,x-api-key

Users will also likely want to add referrerPolicy to their setup, as this another CORS issue, where my server was set to "same-origin" only, and therefore not allowing CORS at all: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referrer-Policy

    middlewares-secure-headers:
      headers:
        accessControlAllowMethods:
          - GET
          - OPTIONS
          - PUT
        accessControlAllowOriginList:
          - 'https://wiki.example.com'
          - 'https://auth.example.com'
          - 'https://shlink.example.com'
          - 'https://tl1.example.com'
          - 'https://shl1.example.com'
        accessControlAllowHeaders:
          - 'content-type'
          - 'x-api-key'
        addVaryHeader: true
        referrerPolicy: "strict-origin-when-cross-origin"

[tomlawesome]

To clarify, when you say both shlink client and shlink server must both be either http or https (but both the same), does this extend all the way to the container, or only as far as the reverse proxy?

[acelaya]

Only to the party which is publicly exposed. The communication between traefik and the Shlink server won't affect the client.

[tomlawesome]

Thanks.

Currently not getting anything from the health endpoint:

[2021-04-21T15:08:38.044312+00:00] [NULL] Access.ERROR - 192.168.10.11 - - "GET /rest/v2/health HTTP/1.1" 404 979
[2021-04-21T15:08:38.110766+00:00] [NULL] Access.INFO - 192.168.10.11 - - "GET /favicon.ico HTTP/1.1" 200 16958
[2021-04-21T15:11:57.418782+00:00] [NULL] Access.ERROR - 192.168.10.11 - - "GET /rest/health HTTP/1.1" 404 979
[2021-04-21T15:11:57.505744+00:00] [NULL] Access.INFO - 192.168.10.11 - - "GET /favicon.ico HTTP/1.1" 200 16958

[acelaya]

I guess this should more or less help others facing similar issues https://shlink.io/documentation/troubleshooting/#shlink-cannot-be-accessed-from-shlink-web-client-or-other-web-clients

[acelaya]

And now I have just realized this was more or less already documented https://shlink.io/documentation/troubleshooting/#short-urls-work-but-i-cant-consume-shlink-from-shlink-web-client 😅

[tomlawesome]

And now I have just realized this was more or less already documented https://shlink.io/documentation/troubleshooting/#short-urls-work-but-i-cant-consume-shlink-from-shlink-web-client 😅

I don’t think that one covers it really. The new section is great, really helpful I think for others in the same position and at a similar knowledge level! Thanks for taking the time to add to the docs.