benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
674 stars 42 forks source link

xidel, xquery, tuple & download ? #113

Open Baltazar500 opened 2 weeks ago

Baltazar500 commented 2 weeks ago

Hi.

How to use xidel within an xquery request to use elements (or variables) of an array (tuple) as a name and link to download a file without using external utilities like curl ? "x:download" because xidel does not support :(

How to use "array[1]" for url to download and "array[2]' for filename ?

echo '[{"a":"https://example.com/example.zip","b":"Filename"}]'|xidel -se 'for $a in parse-json(($raw))() let $array:=(($a).a,($a).b) return $array[1] || " " || $array[2]'

How to use separate variables as a url and filename to download ?

echo '[{"a":"https://example.com/example.zip","b":"Filename"}]'|xidel -se 'for $a in parse-json(($raw))()let $title:=($a).b let $url:=($a).a return $url || " " || $title'

Reino17 commented 2 weeks ago

Your $array-variable holds a sequence instead of a proper JSON array.

$ echo '["https://videlibri.sourceforge.net/img/xidel-windows-blue.png","xidel_screenshot.png"]' | \
  xidel -se '$json()'
https://videlibri.sourceforge.net/img/xidel-windows-blue.png
xidel_screenshot.png

Here the input is a proper JSON array with a working example url.

$ echo '["https://videlibri.sourceforge.net/img/xidel-windows-blue.png","xidel_screenshot.png"]' | \
  xidel - -f '$json(1)' --download .

This downloads and saves the image as 'xidel-windows-blue.png' to the current dir.
Normally the stdin dash, -, isn't necessary, but you can't use -f without specifying an input.

$ echo '["https://videlibri.sourceforge.net/img/xidel-windows-blue.png","xidel_screenshot.png"]' | \
  xidel - -f '$json(1)' --download '{$json(2)}'

This downloads and saves the image as _'xidelscreenshot.png' to the current dir. Unlike -e, the query you enter for --download (and -d as well) is always treated as an extended string. So to evaluate $json(2), the curly brackets are necessary.

"x:download" because xidel does not support :(

No, because obviously there's --download in the first place. There is however a (cumbersome) way to do it in-query:

$ echo '["https://videlibri.sourceforge.net/img/xidel-windows-blue.png","xidel_screenshot.png"]' | \
  xidel -se '
    file:write-binary(
      $json(2),
      string-to-base64Binary(x:request($json(1))/raw)
    )
  '

file-write-binary() (part of the EXPath File Module, integrated in Xidel)

@benibela At first I wanted to use one of the urls on https://nightly.link/benibela/xidel/workflows/main/master as an example url, but none of them seem to work.

benibela commented 2 weeks ago

file-write-binary()

or file:write-text if it is all text

@benibela At first I wanted to use one of the urls on https://nightly.link/benibela/xidel/workflows/main/master as an example url, but none of them seem to work.

They are deleted after three months :/