Cookies: getting, setting, generating

[p-groarke]

Everyone loves cookies! 🍪 nomnom

I know you've stated before it was out of scope, but that was many years ago. Any chance you can bring this back into scope?

Thx and good day, 🍪

[vinniefalco]

LOL.. yeah, we do like cookies. Yes I can reconsider it, but what do you mean by "cookies?" What precisely do you want Beast to do?

[p-groarke]

I was hoping for a builtin way to create cookies and to get/set them in a http response/request.

I am a complete beginner when it comes http itself (learning with beast), so I have no opinion on how that should look.

Cheers and thx for beast, rawr 🐯

[vinniefalco]

I was hoping for a builtin way to create cookies and to get/set them in a http response/request.

Oh, well that's easy, you can do it right now with Beast.

// Set a cookie with the name "mycookie" to the value "123"
http::response<empty_body> res;
res.set(field::set_cookie, "mycookie=123");

Now when a browser sends subsequent requests to the server, there will be a "Cookie" field with all the values that the server sent:

http::request<empty_body> req;
...
// Print each cookie in the request
for(auto param : http::param_list(req[field::cookie]))
    std::cout << "Cookie '" << param.first << "' has value '" << param.second << "'\n";

Hope this helps!

[p-groarke]

Yes a lot! Beast already supports get/set of cookies, very nice :)

Then I'd rephrase my feature request as "cookie generator", to easily create cookies. Though I'm feeling you'll respond out-of-scope, probably rightly so.

[vinniefalco]

What does "cookie generator" mean? Maybe we can enlarge the scope... I'm feeling generous. But seriously, I am generally speaking widening the scope of Beast (although, I am doing it carefully and conservatively).

[p-groarke]

You may close this if you want, I feel I need to learn more before I can be of much help to you.

Basically I was thinking of a cookie factory with serialization/deserialization to http. I do think you'd want to enlarge the scope, as cookies are probably not the only thing that need factory. I'm all for widening the scope of beast, because it is pretty badass so far.

So pseudo code:

// value and onwards are optional
cookie c{name, value, expire, path, domain, secure, httponly};
http::response<empty_body> res;
res.set(field::set_cookie, c);

Is this realistic? Cheers

[vinniefalco]

This sounds pretty reasonable, it could go into <boost/beast/_experimental/http/cookie.hpp> at least, it would need an example program probably a client that connects to a predefined list of public hosts (google, apple, etc) and tries to get cookied up, then prints them. It would of course need tests. But before we get to that this needs more work, what does the declaration of cookie look like? Is there a link to the cookie serialization format (an RFC maybe)?

[p-groarke]

I believe this is the one https://tools.ietf.org/html/rfc6265

[vinniefalco]

https://tools.ietf.org/html/rfc6265

Whoa... that is definitely non-trivial. In fact it looks harder to parse than the entire HTTP message itself! Parsing RFC-compliant timestamps is, as we say in the biz "a pain in the arse." This is still doable but it is a decent sized effort.

[p-groarke]

I don't know how much you can depend on other libraries now that beast is part of boost, but would Hinnant's date library help here? Anyhow, I won't hold my breath. I guess the RFC has made my point it would be nice to have a compliant cookie generator in beast :)

[vinniefalco]

I'm sure Howard wouldn't mind if I borrowed some code but we can't just depend on the entire thing. And for security purposes we really need something that is built in a certain way, to minimize dependencies, in a way that is also friendly to being audited. Maybe there is something out there. Unfortunately I don't have the time to dive into it right now but I am thinking that yes it could be in-scope for _experimental, and if that is successful then in-scope for the public interfaces.

[HowardHinnant]

If it helps, roll-your-own {year, month, day} <-> Unix Time conversion algorithms documented here, public domain: http://howardhinnant.github.io/date_algorithms.html

[vinniefalco]

http://howardhinnant.github.io/date_algorithms.html

Amazing code as usual, but.... there's nothing here for parsing dates and times from strings?

[HowardHinnant]

Correct, bring your own parsing. The algorithms are just the math.

[wangh09]

Hi, is there a way to read cookie from the websocket session? thanks!

[vinniefalco]

is there a way to read cookie from the websocket session?

Yes, see:

https://www.boost.org/doc/libs/1_70_0/libs/beast/doc/html/beast/using_websocket/handshaking.html#beast.using_websocket.handshaking.inspecting_http_requests

[wangh09]

Thanks, it works. BTW I found that the param_list requires a ';' at the beginning, namely ";key1=val1; key2=val2" is valid but "key1=val1; key2=val2" is not.

[vinniefalco]

I found that the param_list requires a ';' at the beginning

Yes, that is required by RFC7230's BNF:

        param-list  = *( OWS ";" OWS param )
        param       = token OWS [ "=" OWS ( token / quoted-string ) ]

[maddanio]

is there a way to read cookie from the websocket session?

Yes, see:

https://www.boost.org/doc/libs/1_70_0/libs/beast/doc/html/beast/using_websocket/handshaking.html#beast.using_websocket.handshaking.inspecting_http_requests

but is there any way to set them on the client end? we would like to implement uniform auth on our apache (which proxies all our ws services) and the easiest way is request headers or cookies. in js cookies is the way since we cant set headers. but in beast it seems we can do neither?

[madmongo1]

A client side cookie jar is beyond the scope of beast. You would need to find an existing library or write what you need.

[maddanio]

is there a way to read cookie from the websocket session?

Yes, see: https://www.boost.org/doc/libs/1_70_0/libs/beast/doc/html/beast/using_websocket/handshaking.html#beast.using_websocket.handshaking.inspecting_http_requests

but is there any way to set them on the client end? we would like to implement uniform auth on our apache (which proxies all our ws services) and the easiest way is request headers or cookies. in js cookies is the way since we cant set headers. but in beast it seems we can do neither?

I take it back, there is the decorator option which does what I need, just didnt understand that, my error.

[dhalsimax]

http::param_list has another anomaly. In case you pass strings like: "; Coockie1=;Cookie2=Value2" where a key has got no value, it returns an empty set. This could be ok if you know, but would be nice if it returns all the keys value pairs with the "it->second" sets to "" whener it is. Firefox for example refires cookies with null values, so in case you set one of this in the response you will be forces to restart the browser. In another case you set "Expiration" attribute of such null value cookies you will be also forced to manually clear cookies on browser. Anyway rfc, as I remember, doesn't forbid such kind of cookies and could be useful to have some of null valued.

[klemens-morgenstern]

As the cookie jar is beyond the scope of beast, can this be closed?

[tyyykkkeeessshhaaaa]

is there a way to read cookie from the websocket session?

Yes, see: https://www.boost.org/doc/libs/1_70_0/libs/beast/doc/html/beast/using_websocket/handshaking.html#beast.using_websocket.handshaking.inspecting_http_requests

but is there any way to set them on the client end? we would like to implement uniform auth on our apache (which proxies all our ws services) and the easiest way is request headers or cookies. in js cookies is the way since we cant set headers. but in beast it seems we can do neither?

If you are finished or have a working sample of your code for proxies with websockets, please send 😄 😄 😄

[tyyykkkeeessshhaaaa]

need to implement proxies into my client

[tyyykkkeeessshhaaaa]

im interested in getting cookies from a response, haven't seen an example of this

[klemens-morgenstern]

It's in the works in my requests library on the client side.

[vinniefalco]

This is how you generate the cookies https://www.wideopeneats.com/our-grandmas-classic-chocolate-chip-cookies/

[p-groarke]

tsssk, unfair

[vinniefalco]

Oops! Didn't even think of that. Here's a softcopy of the recipe

[tyyykkkeeessshhaaaa]

thank you, i have found a new and better passion, screw ssl and websocket in C++

[tyyykkkeeessshhaaaa]

It's in the works in my requests library on the client side.

looks, very nice! will definitely use if possible in my code, although not entirely sure what libraries are best for me. If anyone can provide guidance as I need a way to be able to create several Https clients and websocket clients and have them in separate threads, also needs to be able to modify cookies and headers for requests too and get them from response

for more context this is essentially for web scraping

edit: http is supposed to say https in visualization

[vinniefalco]

there's a beast example that launches a whole bunch of clients and tries to download the home page of the top 10,000 most popular web addresses:

https://github.com/boostorg/beast/tree/develop/example/http/client/crawl

[tyyykkkeeessshhaaaa]

there's a beast example that launches a whole bunch of clients and tries to download the home page of the top 10,000 most popular web addresses:

https://github.com/boostorg/beast/tree/develop/example/http/client/crawl

yes, I have seen this and actually used it in my initial code using this library, however it still lacks proxies with websockets and cookies