taoensso / sente

Realtime web comms library for Clojure/Script
https://www.taoensso.com/sente
Eclipse Public License 1.0
1.74k stars 193 forks source link

Managing CRSF tokens in a cljs/clj app #415

Closed kovasap closed 1 year ago

kovasap commented 1 year ago

I'm just been onboarding my project to sente as per the advice I got on reddit. In doing so, I've had to rework my application so that my web server is serving my frontend so that I can use clojure code to generate my crsf token. This has led to some confusion that I detailed in this post.

Despite this setup otherwise working, I am getting 403 errors saying I have a bad crsf token. I also tried just disabling crsf entirely by passing nil to sente in my frontend (like all the shadow cljs example projects linked in the readme do) instead of my token and removing relevant lines from my server code, but I get the same errors.

Are there any working projects out there that use shadow-cljs and crsf tokens? Also, is there any way to generate the crsf token in my html somehow so that I don't need my api server to serve it and can just use the shadow-cljs web server directly?

ptaoussanis commented 1 year ago

Hi Kovas, should be able to get back to you tomorrow!

ptaoussanis commented 1 year ago

@kovasap Hi Kovas,

I've had to rework my application so that my web server is serving my frontend so that I can use clojure code to generate my crsf token.

Just to clarify: you're asserting that your frontend .js (including compiled Sente) must be served by your Clojure web server? If so, what's led you to believe that?

Your front and back ends will indeed need to agree on a CSRF token, but it's not obvious to me that that would imply the above. I might be misunderstanding something though.

https://www.reddit.com/r/Clojure/comments/10xe952/best_practices_for_serving_web_content/

This link appears to be dead, sorry:

Sorry, this post was removed by Reddit's spam filters. Reddit's automated bots frequently filter posts it thinks might be spam.

I am getting 403 errors saying I have a bad crsf token

Could you please share the exact error that you're seeing? For one, it'd be helpful to understand precisely what software/layer is complaining.

I also tried just disabling crsf entirely by passing nil to sente in my frontend (like all the shadow cljs example projects linked in the readme do)

I'm not familiar with any particular community examples off-hand, but for security reasons you cannot disable CSRF checks from the client side. CSRF checks can only be disabled from the server. There's some info on how to do this in the make-channel-socket-server! docstring.

Are there any working projects out there that use shadow-cljs and crsf tokens?

I'm unfortunately not familiar with shadow-cljs myself, but in principle I don't expect that there should be any fundamental differences between a shadow-cljs project and the reference example.

High-level: your Sente server and Sente client just need to agree on what the valid CSRF token is.

You could start with a simple constant token, just to confirm that your basic machinery is correctly in place. Once you've confirmed that that's working, you can try swap in a real (dynamic) token.

The simplest setup is usually something like this:

  1. Server-side: generate a random token, and embed it in your rendered HTML.
  2. Client-side: grab the token from your rendered HTML.

Again, ultimately, the server and client just need to be able to agree: the client must send the same token that the server is expecting.

This is the approach that the reference example takes.

Step 2 should usually be trivial. Step 1 can be non-trivial to get right. ring-anti-forgery is a common library-level solution.

is there any way to generate the crsf token in my html somehow so that I don't need my api server to serve it

Embedding a token in your HTML should indeed be possible in principle, and is a (the?) common approach. How precisely to generate the token will generally be up to you. You can see the reference example and ring-anti-forgery links above for some pointers.

I hope that helps!

kovasap commented 1 year ago

Hey sorry I didn't realize my reddit post was removed. Here is the text:

I've been building a web app with a client written using Shadow-CLJS and a server based on retit.ring. The template I used (https://github.com/schnaq/cljs-re-frame-full-stack) and other examples I've found seems to suggest having two servers running to run the application. One server just serves the Shadow-CLJS frontend and the other handles the API calls that frontend might make.

Recently, I've been trying to incorporate Sente for some client/server communication and ended up with having the API server serve the frontend code as well. This is primarily because the Sente docs recommend that I add an anti-forgery token generated by a clj library to my HTML, but it seems generally useful to be able to generate some HTML on my server in hiccup format. My current code to do this is at https://github.com/kovasap/cljs-board-game/blob/ae83bf4e90142460b2f09ecca961d7ab581f42b9/src/main/app/api.clj#L48-L74.

Getting the server to serve my frontend code has been surprisingly hard, and I don't even have it fully working yet. Currently I'm experiencing some issues where Shadow-CLJS is complaining that it can't connect to the Shadow-CLJS server that I was running before. Maybe another type of Shadow-CLJS build would resolve these, but then I worry that I wouldn't get the nice iterative development benefits.

This has all made me wonder if this is a good design pattern or if I'm making some obvious newbie mistakes. Any tips/guidance/examples would be much appreciated!

To your questions:

Just to clarify: you're asserting that your frontend .js (including compiled Sente) must be served by your Clojure web server? If so, what's led you to believe that?

Your front and back ends will indeed need to agree on a CSRF token, but it's not obvious to me that that would imply the above. I might be misunderstanding something though.

It seems to me like in order to generate a unique crsf token dynamically I would need some clojure code to modify the HTML I send to the client. This is suggested in the Sente README like:

(let [csrf-token (force ring.middleware.anti-forgery/*anti-forgery-token*)]
  [:div#sente-csrf-token {:data-csrf-token csrf-token}])

If there's another way to accomplish this without the anti-forgery call, I'm curious!

I just tried using a constant token (the string "testing"), and got the same error:

image

ptaoussanis commented 1 year ago

I just tried using a constant token (the string "testing"), and got the same error:

Did you also update the Sente server so that it expects the same (constant) token?

Like I mentioned, you need the Sente server and Sente client to agree on the token. Ultimately what matters is what the server expects: the client needs to send the token that the Sente server is expecting, otherwise the server will reject the request as insecure.

You can see the make-channel-socket-server! docstring for info on how to control the server's token expectation:

(sente/make-channel-socket-server! adapter {<other opts...> :csrf-token-fn (fn [ring-req] "testing")}

The client should then have then have the same token:

(sente/make-channel-socket-client! path (fn [] "testing") {<other opts...>})

Does that make sense? Just in case you're not too clear on what a CSRF token is or how they generally work, it might be helpful to look this up so that you've got an intuitive foundation.

It seems to me like in order to generate a unique crsf token dynamically I would need some clojure code to modify the HTML I send to the client. This is suggested in the Sente README like:

The Sente README and reference example have a server that render HTML responses using Hiccup. In this case it's pretty easy to add a CSRF token to the HTML output as shown in the README and ref example.

Your linked landing-pg-handler looks fine.

I'm not familiar with your routing lib though, or shadow - and would prefer to avoid trying to parse through your whole codebase and setup.

Instead, I'd recommend you try first narrow down where the issue is happening. For example:

  1. Your linked landing-pg-handler is rendering a div with id "sente-csrf-token". Is that div actually present in the HTML you're seeing in your browser? If not, then you know something's off with your rendering, routing, or high-level setup.

  2. If you do see the div in your browser's HTML, then check what the token is (value of the data-csrf-token attribute). I'd recommend initially modding your Sente server as already described so that the token is a constant (e.g. "testing"). Try get that to work.

  3. Once you see the "testing" token in your generated HTML, try confirm that "testing" is also being used by the Sente client. You can use your browser's dev tools to check what request the Sente client is actually making to the server. Is it providing the "testing" token?

  4. If the server HTML includes "testing", and the Sente client request includes "testing" - then the Sente request should succeed. Does your browser confirm this? If not, could be an issue with routing on the server, some other auth layer on the server is unhappy, etc. You'd need to look at the specific error to debug further. If the request does succeed, then your next step would be switching the constant "testing" token to something dynamically generated by the server.

Does that make sense?

I'd encourage you to try and break your debugging down into digestible steps. E.g. what's the first thing to confirm? That the server's HTML response includes a token. Then: is it the token you're expecting? Then: can the client see the token? Then: does the client see the same token that the server is expecting? Etc.

Going step-by-step like this also makes it easier for others to assist - which'll make it more likely that you get quick assistance.

Hope that helps!

kovasap commented 1 year ago

This:

(sente/make-channel-socket-server! adapter {<other opts...> :csrf-token-fn (fn [ring-req] "testing")}

is exactly what I was missing! My issue goes away when adding this option. Strangely, when using (force ring.middleware.anti-forgery/*anti-forgery-token*) I still have the same problem, even when I pass this in to the above function via :csrf-token-fn (fn [ring-req] (force ring.middleware.anti-forgery/*anti-forgery-token*))}. Would you expect this to work?

ptaoussanis commented 1 year ago

is exactly what I was missing! My issue goes away when adding this option.

Great 👍

Would you expect this to work?

Yes - if the middleware is configured correctly - and if the client agrees on the token, then I'd expect that to work. Please see my previous recommendations for how to debug this further.

kovasap commented 1 year ago

Ok still losing my mind a bit over this issue : P.

From this print statement on my server, I see "TOKEN" "P9Ke8dXVemgezjEWttLSPgv+6alirbAO4UX1qPmd6uToDek4AwpR5KZYBLFtRur8IHF0h3bVsrBjbx7k", which matches the token received by the client AND used by the client to send requests to the server:

image

I highly suspect that sente on the server side is not using the same token. My attempts to make it do so have all failed; they all generate unique, different tokens when they are called. Based on the example, it seems like this shouldn't even be necessary - I assume sente automatically checks for the token under the hood somehow?

Any ideas for how I can check which token sente on the server side is expecting?

I was reading at https://github.com/ring-clojure/ring-anti-forgery#usage that the anti forgery strategy tries to use different tokens for each session - maybe sente is not on the same "session" as the html with the token that's sent to the client? Do you know of a way to check this?

ptaoussanis commented 1 year ago

Thanks for the clear links, and descriptions of what you've tried - that makes it easier for me to assist 👍

From this print statement on my server, I see "TOKEN" "P9K<...>", which matches the token received by the client AND used by the client to send requests to the server

Okay, that's a great start. So you've confirmed what the Sente client's expectation of the correct token is 👍

Now you'll want to confirm what the Sente server's expectation of the correct token is.

You can mod your server constructor call to something like this:

 (sente/make-channel-socket-server!
   (get-sch-adapter)
   {:csrf-token-fn
    (fn [ring-req]
      (let [;; Code to get token from ring-req should match
            ;; https://github.com/kovasap/cljs-board-game/blob/4ddeba9bf11890c5d163650150ec440ce6b9d8a5/src/main/app/api.clj#L75
            csrf-token (:anti-forgery-token ring-req)]
        (prn "TOKEN (server)" csrf-token)
        csrf-token))})

With the above in place, you'll have:

  1. Confirmation (via the above print statement) of the token the server is expecting.
  2. Confirmation (via your first print statement) of the token embedded in your HTML.
  3. Confirmation (via your client->server browser request) of the token the client believes the server is expecting.

For everything to work correctly, you'll need all tokens to match: 1=2=3. It sounds like you've already confirmed that 2=3, so the likely problem is that 1!=2.

Can you confirm that 1!=2?

If so, then the next step is to debug why the Ring requests being provided to Sente have a different :anti-forgery-token value than the Ring requests being provided to your landing page handler.

To understand that, you'll need to understand your routing and middleware structure.

Unfortunately two challenges there:

If you're new to Clojure, Ring, middleware, etc. - it's often easiest to start with working examples for your particular framework or library.

Otherwise - your best bet for debugging is often just verbose logging. In your case, you'll want to inspect your Ring requests - in particular the :anti-forgery-token values.

In specific response to some of your points:

I highly suspect that sente on the server side is not using the same token.

Agreed, for the reasons stated above.

Based on the example, it seems like this shouldn't even be necessary - I assume sente automatically checks for the token under the hood somehow?

Correct, if a :csrf-token-fn arg isn't explicitly provided to make-channel-socket-server! then a default fn will be used that will try extract a CSRF token from the Ring request provided to Sente by looking at some of the most typical values (including (:anti-forgery-token ring-req)).

The reason I'm advocating for you providing an explicit :csrf-token-fn here is solely so that you can add logging to help debug. Once you have everything working, you could remove the custom fn.

I was reading at https://github.com/ring-clojure/ring-anti-forgery#usage that the anti forgery strategy tries to use different tokens for each session - maybe sente is not on the same "session" as the html with the token that's sent to the client?

Something like that seems plausible, yes. Though I'm not actually too familiar with ring-anti-forgery off-hand. Last I looked at it years ago, I wasn't a fan of the way it worked - so I tend to use a custom CSRF implementation myself. It's certainly widely used though, there should be lots of examples around.

Do you know of a way to check this?

Logging should be helpful here. Ring requests are just data (maps), and middleware just transform that data (add keys, etc.).

Routes / handlers are ultimately just functions of Ring requests.

You can inspect/print the content of Ring requests, incl. content like :session, :params, :anti-forgery-token, etc.

Hope something above is of some use, good luck!