benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
674 stars 42 forks source link

Adding HTTP(S) headers not working #112

Closed smartmic closed 3 weeks ago

smartmic commented 3 weeks ago

The --header (or -H) option does not work for me for GET requests. What am I doing wrong here:

$ # adding custom header 'Cookie' is not part of request header ⇒
$ xidel https://httpbin.org/headers -H "Cookie: a=b" -e "$json?*"
{
  "Accept": "text/html,application/xhtml+xml,application/xml,text/*,*/*",
  "Host": "httpbin.org",
  "User-Agent": "Mozilla/5.0 (compatible; Xidel)",
  "X-Amzn-Trace-Id": "…"
}
$ # same with curl works ⇒
$ curl https://httpbin.org/headers --header "Cookie: a=b" 
{
  "headers": {
    "Accept": "*/*", 
    "Cookie": "a=b", 
    "Host": "httpbin.org", 
    "User-Agent": "curl/8.8.0", 
    "X-Amzn-Trace-Id": "…"
  }
}
Reino17 commented 3 weeks ago

You have to put --header / -H in front of the url.

Even curl dictates this (Usage: curl [options...] <url>), but apparently it is more forgiving than xidel.

smartmic commented 3 weeks ago

Oh yes, thanks! I was too easily led astray as the --help page did not distinguish between the --extract and --follow options (which are usually after the url in the examples) and all other options. Maybe a short update in the help page or adding a command synopsis would help (I still hope such a great tool will be maintained and get an update sometime in the future).