search

httpie / cli

🥧 HTTPie CLI — modern, user-friendly command-line HTTP client for the API era. JSON support, colors, sessions, downloads, plugins & more.
https://httpie.io
BSD 3-Clause "New" or "Revised" License
33.37k stars 3.67k forks source link

Support for reading cookies from files #404

Open Bsiyag opened 8 years ago

Bsiyag commented 8 years ago

Hi,

I really like the interface of this util and I do understand that cookies can be passed on in the GET requests as key-value pairs but some websites require quite a few cookies to be able to serve a page. I use the add-on https://chrome.google.com/webstore/detail/cookiestxt/njabckikapfpffapmjgojcnbfjonfjfg?utm_source=chrome-app-launcher-info-dialog that lets you save all the cookies on a webpage into a file that can be used with wget or curl. Alternatively, the cookies set by the server using the set-cookie header primitives can be dumped to a cookie-jar file using both of those tools. Curl lets you read cookies either directly from stdin or from a file with the -b flag.

Would it be possible to incorporate that into this tool?

Thanks, Bhupi

jkbrzt commented 8 years ago

Perhaps it would be possible to use the sessions feature to accomplish something similar?

https://httpie.org/doc#sessions

Bsiyag commented 8 years ago

It's not JSON format. Pasting below a sample cookies file for github. // Example: wget -x --load-cookies cookies.txt https://github.com/jkbrzt/httpie/issues/404#issuecomment-152310720 // .github.com TRUE / FALSE 1482342124 _octo GH1.1.1405997590.1419775245 gist.github.com FALSE / TRUE 0 _gist_session BAh7B0kiD3Nlc3Npbsdas98d7asf6ds87af6a97afDU3Y2YyZDliZTlhYzUzBjsAVEkiEF9jc3JmX3Rva2VuBjsARkkiMUJCMmNzcHhFRzdjZzc0Rml0QWNBN3I5dENoNDJ2TjhlclJBbnFqdWdRQVU9BjsARg%3D%3D--fb3b90fadfebb39c06d90a1c346ac8062cc50e1a gist.github.com FALSE / FALSE 1482342124 _ga GA1.1.749480923.1419775244 .github.com TRUE / TRUE 2073234372.312392 logged_in yes .github.com TRUE / TRUE 2073234372.312392 dotcom_user Bsiyag .github.com TRUE / FALSE 1482342124 _gat 1 github.com FALSE / TRUE 0 tz Europe%2FBerlin .github.com TRUE / FALSE 1503242165 _ga GA1.2.749480923.1419775244 github.com FALSE / TRUE 0 _gh_sess eyJzZXNzaW9uX2lkIjoiMDUwZjAwY2M4NmQ4YWI2ZDc2NjQxZDgxMzllZmE0ZTkiLCJyZWZlcnJhbF9jb2RlIjoiaHR0cDovL21hc2hhYmxlLmNvbS8yMDE0LzEwLzAzLsdgf80b7fsjfadjkahgf898ds9g8mfjknbvxchrwt-9qeo9808ilyukm9078NTExNjUsIl9jc3JmX3Rva2VuIjoibzFTdllmcWtRUWEwL2xSM2lNS3IyeklINEZyZjZKOXFsVzhLQWtBcC81cz0iLCJsYXN0X3dyaXRlIjoxNDQ2MTUxMTk5ODI2LCJjb250ZXh0IjoiLyJ9--4f6b46b822254f94bc5e6dc92113231a0d10f504 github.com FALSE / TRUE 141340798.872784 user_session I1WdfIFii4mpZkZL6z8PKqnFQfPvf8exHVWjGSb8ARdBtC3iVjKEIGZ5EddwWvane_uw6UKdgTRzCTIX

I use this to log into sites that require SSO from a browser; dump cookies and then access more data that I need by using curl. As an example of more info: I can grep for IP addresses of people who added comments.

ryanerwin commented 6 years ago

There are a few plugins for Chrome and Firefox that make it easy to grab your cookies off of a website and drop them into a cookies.txt file understood by curl and wget.

I've been using httpie more and more as it's so much nicer than curl...

For example, here's cookies.txt in the Chrome Web Store. One of my favorite plugins! Comes in handy often for testing, using it with httpie would be so much nicer than curl... ;-)

Bsiyag commented 6 years ago

Yes @ryanerwin , I mentioned the exact same plugin with references to wget and curl while reporting this issue about 3 years ago. I gave up on using httpie after a while due to this shortcoming but I agree that it would be much nicer than curl-ing a resource on web.

ryanerwin commented 6 years ago

I was doing several hours of HTTP testing yesterday and though I love the way HTTPIE looks so clean, there were a few features that just kept bringing me back to curl... Hope this gets implemented at some point because curl can be very rough on the eyes 😄

The other feature in curl that was super handy is --resolve so that you can force a connection via a specific address regardless of how DNS is setup...

Yet I count a total of 214 command line switches in curl... would be very difficult to reimplement all of them somewhere...

Bsiyag commented 6 years ago

The --resolve switch is just an extra feature like most of those flags without which one can still play with host entries and achieve the same results but cookies are almost everywhere on the web and they're rarely just one key-value pair.

dogweather commented 5 years ago

@jakubroztocil This would be a fun project. Would you be interested in a PR for it?

quinncomendant commented 2 years ago

It's not JSON format. Pasting below a sample cookies file for github. […] I use this to log into sites that require SSO from a browser; dump cookies and then access more data that I need by using curl.

This file format is the Netscape HTTP cookie file format. It's commonly supported by curl, wget, youtube-dl, et al. It would make sense for httpie to support it too.

Here's what the curl docs say about it:

Netscape once created a file format for storing cookies on disk so that they would survive browser restarts. curl adopted that file format to allow sharing the cookies with browsers, only to see browsers move away from that format. Modern browsers no longer use it, while curl still does.

The Netscape cookie file format stores one cookie per physical line in the file with a bunch of associated meta data, each field separated with TAB. That file is called the cookiejar in curl terminology. […] Lines that start with # are treated as comments.

Each line that specifies a single cookie consists of seven text fields separated with TAB characters. A valid line must end with a newline character.

Field number, what type and example data and the meaning of it:

  • string example.com - the domain name
  • boolean FALSE - include subdomains
  • string /foobar/ - path
  • boolean TRUE - send/receive over HTTPS only
  • number 1462299217 - expires at - seconds since Jan 1st 1970, or 0
  • string person - name of the cookie
  • string daniel - value of the cookie