How to handle CSRF verification?

r-franzke commented 5 months ago

Summary

Not sure if this is a feature request or just something missconfigured on my server. Basically I always get a CSRF verfification failed error, as I guess the CSRF token is missing in the request.

It would be very nice to get some advice about this, as I really like to use your client instead of writing something myself.

Context

The Error I got:

Could not upload file (title="" type="" tags="[]" error="request failed with status code 403:
<!DOCTYPE html>
<html lang="en">
<head>
  <meta http-equiv="content-type" content="text/html; charset=utf-8">
  <meta name="robots" content="NONE,NOARCHIVE">
  <title>403 Forbidden</title>
  <style type="text/css">
    html * { padding:0; margin:0; }
    body * { padding:10px 20px; }
    body * * { padding:0; }
    body { font:small sans-serif; background:#eee; color:#000; }
    body>div { border-bottom:1px solid #ddd; }
    h1 { font-weight:normal; margin-bottom:.4em; }
    h1 span { font-size:60%!;(MISSING) color:#666; font-weight:normal; }
    #info { background:#f6f6f6; }
    #info ul { margin: 0.5em 4em; }
    #info p, #summary p { padding-top:10px; }
    #summary { background: #ffc; }
    #explanation { background:#eee; border-bottom: 0px none; }
  </style>
</head>
<body>
<div id="summary">
  <h1>Forbidden <span>(403)</span></h1>
  <p>CSRF verification failed. Request aborted.</p>

  <p>You are seeing this message because this site requires a CSRF cookie when submitting forms. This cookie is required for security reasons, to ensure that your browser is not being hijacked by third parties.</p>
  <p>If you have configured your browser to disable cookies, please re-enable them, at least for this site, or for “same-origin” requests.</p>

</div>

<div id="explanation">
  <p><small>More information is available with DEBUG=True.</small></p>
</div>

</body>
</html>
")

Out of Scope

List aspects that are explicitly not part of this feature

Further links

URLs of relevant Git repositories, PRs, Issues, etc.

Acceptance criteria

Given a precondition
When an action happens
Then a result is expected

Implementation Ideas

If applicable, shortly list possible implementation ideas

ccremer commented 5 months ago

That's ... very strange. The paperless-ngx docs don't mention the requirement of CSRF token when using the API. What's your authentication method? If you're authenticating via user+password, does it work when switching to Token method?

Also, can you provide (redacted) logs of the output and the exact CLI command/config? It's also interesting to know the Paperless-ngx version, maybe something changed between versions.

cdoepmann commented 5 months ago

I'm experiencing the same issue. My config.yaml is as follows:

consume-delay: 1s
consume-dir: ""
content: archive
incremental: false
log-level: 0
overwrite: false
target-path: ""
token: "XXXXX"
unzip: false
url: "http://127.0.0.1:8000/"
username: "XXXXX"

(I'm using just username + password for authentication.)

I'm seeing this issue with paperless-ngx versions 2.3.3 and 2.4.0. Haven't tested others.

r-franzke commented 5 months ago

I`m using 2.4.2 via the docker-compose setup. I also run a nginx proxy in front of it. But also when I call it locally via http, without the nginx proxy, I get the same Error.

Based on this line, I woul expect the CSRF check to be disabled for the uplad endpoint: https://github.com/paperless-ngx/paperless-ngx/blob/dev/src/paperless/urls.py#L174

Can you tell us a bit about your setup @ccremer? Maybe we can find a difference.

ccremer commented 5 months ago

I can't reproduce this. I've tried version 2.2 and 2.4. Both token method and username+password method work in my case. My Url is set to the publicly accessible one with a valid TLS cert.

In my setup I'm using Let's Encrypt with Caddy as a proxy in front of it and have set the PAPERLESS_URL variable, which also sets PAPERLESS_CSRF_TRUSTED_ORIGINS if not overridden. https://docs.paperless-ngx.com/configuration/#hosting-and-security

Maybe you could look closer into these options?

r-franzke commented 5 months ago

The PAPERLESS_URL variable is set correctly. It is also working fine via the browser.

Are you also using the docker setup or the barematal installation?

r-franzke commented 5 months ago

I found the problem... :D

I had to remove the trailing / from the URL.

This was what I have changed.

- paperless-cli upload  --token <token> --url https://<my-paperless-host>/ <my-file>
+ paperless-cli upload  --token <token> --url https://<my-paperless-host> <my-file>

So basically I send https://<my-paperless-host>//api/documents/post_document/ which paperless is not able to match on the except_csrf annotation.

Thanks for your help!

r-franzke commented 5 months ago

I'm experiencing the same issue. My config.yaml is as follows:
consume-delay: 1s
consume-dir: ""
content: archive
incremental: false
log-level: 0
overwrite: false
target-path: ""
token: "XXXXX"
unzip: false
url: "http://127.0.0.1:8000/"
username: "XXXXX"
(I'm using just username + password for authentication.)

I'm seeing this issue with paperless-ngx versions 2.3.3 and 2.4.0. Haven't tested others.

I guess this is also the same problem here, as there is also a trailing /.

ccremer commented 5 months ago

I found the problem... :D

I had to remove the trailing / from the URL.

ah, well that wasn't obvious. But is stated in the docs for this Env var :)

Do not include a trailing slash.

I guess we could print out a warning if there's a trailing slash in the config though.

cdoepmann commented 5 months ago

Removing the trailing / indeed solves the issue for me as well – thank you!

ccremer / paperless-cli