benibela / xidel

Command line tool to download and extract data from HTML/XML pages or JSON-APIs, using CSS, XPath 3.0, XQuery 3.0, JSONiq or pattern matching. It can also create new or transformed XML/HTML/JSON documents.
http://www.videlibri.de/xidel.html
GNU General Public License v3.0
674 stars 42 forks source link

post request not working #8

Closed Fuzzyma closed 6 years ago

Fuzzyma commented 7 years ago

I need to send a post request. I try so doing:

xidel "http://annuaire.cncc.fr/index.php?page=liste&p=1" --method POST -d "recherche=1&nature_fiche=toutes&nom_societe=&cp_ville=&crcc=0&x=34&y=18" -e '//html'

That doenst work. Also when I split up the patameters:

xidel "http://annuaire.cncc.fr/index.php" --method POST -d "recherche=1" -d "nature_fiche=toutes" -d "nom_societe=" -d "cp_ville=" -d "crcc=0" -d "x=34" -d "y=18" -e '//html'

it always prints **** Retrieving (GET): http://annuaire.cncc.fr/index.php ****. How do I convince xidel to do a post request instead?

benibela commented 7 years ago

You need to place the url after the data.

xidel --method .. -d .. http:/... -e ...

You can think of it like a pipe. Set the data, make the http request, process it. If the url comes first, it makes the http request, sets the data afterwards, but there is nothing to send them to. Another example: you can have two urls, data in the middle, and it is only send to the 2nd url.

Only for some arguments there is a check, if they are at position where they will be ignored, to automatically reorder them (that is why xidel -e ... http://... works)

Fuzzyma commented 7 years ago

mh ok its doing post requests now. However - this works:

curl 'http://annuaire.cncc.fr/index.php?page=liste' --data 'recherche=1&nature_fiche=toutes&nom_societe=&cp_ville=&crcc=0&x=34&y=18'

but this not:

xidel --method="post" -d "recherche=1&nature_fiche=toutes&nom_societe=&cp_ville=&crcc=0&x=34&y=18" "http://annuaire.cncc.fr/index.php?page=liste&p=1" -e '//html'

// EDIT: With "does not work" i mean it doenst return the expected result from the request

benibela commented 7 years ago

What is the expected result?

Fuzzyma commented 7 years ago

When it fails, I get back a website saying that no results where found (in france). I expect to get the table with the results instead

benibela commented 7 years ago

It works for me. Which OS/version are you using?

Fuzzyma commented 7 years ago

Windows 7 64 bit (Xidel 0.9.4)

benibela commented 7 years ago

Then Windows sends happily any method string, while it is sanitized on linux.

It is --method POST, not --method post. Or just leave it

Fuzzyma commented 7 years ago

I dont believe I missed that. I already saw this mistake earlier in all my tests. Thank you very much!