Prepend baseurl before following

[Fuzzyma]

Is there a way to prepend the baseurl before every link before following? In my case I have relative links in a json file which I want to follow. I need to add the url to the links before I can follow them.

Is this even possible? Thats my command so far:

xidel file.json -f '$json()["url"]' -e '//html'

[Fuzzyma]

Yes it is possible with transform and concat

xidel file.json -f 'transform($json()("url"), function($e){ concat("http://baseUrl.com", $e) })' -e ...

[benibela]

That's a surprise. I thought transform would only work with XML.

It is supposed to be done with the mapping operator !

xidel file.json -f '$json()("url") ! concat("http://baseUrl.com", .) ' -e ...

or

xidel file.json -f '$json()("url") ! resolve-uri(., "http://baseUrl.com") ' -e ...

[Fuzzyma]

Well that solution looks way better. However: I am already done with my task :D. Thanks anyway! Is this documented somewhere?

Also: Is there a way to retry requests, when they timeout? Because I get errors like this: Error: -3 Internet connection reseted

[benibela]

Is this documented somewhere?

It is just XPath 3: https://www.w3.org/TR/xquery-30/#id-map-operator

Also: Is there a way to retry requests, when they timeout?

--error-handling=xx=retry

for an error code xx. Or xx for all 2 digit codes

[Fuzzyma]

cool thanks!