Lua-cURL / Lua-cURLv3

Lua binding to libcurl
MIT License
279 stars 64 forks source link

sorry...How to require ? #179

Closed dauuricus closed 3 years ago

dauuricus commented 3 years ago

i can't get any example code infomation about case of Lua-cURLv3.

I installed Lua-cURL on luarocks

luarocks install Lua-cURL

lua-curl 0.3.13-1 (rockspec) - https://luarocks.org

i wrote

local curl = require(cURL)

:result:

************************
lua: test_redirect.lua:15: module 'cURL' not found:
        no field package.preload['cURL']
        no file '/data/data/com.termux/files/usr/share/lua/5.3/cURL.lua'
        no file '/data/data/com.termux/files/usr/share/lua/5.3/cURL/init.lua'
        no file '/data/data/com.termux/files/usr/lib/lua/5.3/cURL.lua'
        no file '/data/data/com.termux/files/usr/lib/lua/5.3/cURL/init.lua'
        no file './cURL.lua'
        no file './cURL/init.lua'
        no file '/data/data/com.termux/files/usr/lib/lua/5.3/cURL.so'
        no file '/data/data/com.termux/files/usr/lib/lua/5.3/loadall.so'
        no file './cURL.so'
stack traceback:
        [C]: in function 'require'
        test_redirect.lua:15: in main chunk
        [C]: in ?

did i mistake ?

how to require() cURL? lcurl? lua-curl?

moteus commented 3 years ago

cURL is a right name. Seems package.path/package.cpath do not point to a place to where luarocks install modules To check it just find lcurl.so and cURL.lua files in your system

dauuricus commented 3 years ago

Thank you.

OK,now i understand the reason. I made mistake.I notice that when i try to another module install & test require(httpclient) few secounds before.

I installed 2 luarocks.One is pkg install luarocks,other is termux's github clone repo on my termux ./build-package.sh -I luarocks.

so, i remove pkg remove luarocks & rm /data/data/com.termux/files/home/.termux-build/.built-packages/luarocks then i rebuild luarocks and then reinstall Lua-cURL.

OK, now i can require("cURL").

and i want to know same example code ... about handling https response redirect 'location'.

moteus commented 3 years ago

Can check out libcurl doc. E.g. CURLOPT_FOLLOWLOCATION and CURLOPT_MAXREDIRS

dauuricus commented 3 years ago

@moteus thank you! I will read that first.

dauuricus commented 3 years ago

OK. maybe minimum example is

local curl = require('cURL')
c = curl.easy{
    url = "https://example.com"
}
c:perform()
c:close()

--[ c code ?]--
--CURL *curl = curl_easy_init();
--if(curl) {
--  curl_easy_setopt(curl, CURLOPT_URL, "https://example.com");
--
--  /* example.com is redirected, so we tell libcurl to follow redirection */
--  curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);                   --
--  curl_easy_perform(curl);
--}

is cURLv3 syntax same as cURLv2 ?

How to write in Lua code ?

example

curl_easy_getinfo(curl, CURLINFO_REDIRECT_URL, &url);
    if(url)
      printf("Redirect to: %s\n", url);

Rf.https://curl.se/libcurl/c/CURLINFO_REDIRECT_URL.html

dauuricus commented 3 years ago

it looks cURLv2 has

c:setopt_

method. Because this contrasts with "The Easy interface"... https://curl.se/libcurl/c/

I need to know around "curl_easy_getinfo" for redirect url. So I would like to know the format of the methods on v3 around "setopt" and "get info" .

or there is no way to treat "https redirect" same as luasec ?

dauuricus commented 3 years ago

oh I see

https://lua-curl.github.io/lcurl/modules/lcurl.html#easy:getinfo

but i can't do

local curl = require('cURL')

curl.easy()
  :setopt_url("https://example.com")
  :setopt(curl.CURLOPT_FOLLOWLOCATION, 2)
--:setopt(curl.CURLOPT_MAXREDIRS, 3)
  :perform()
  --:getinfo(curl.INFO_RESPONSE_CODE)
--:getinfo(curl.CURLINFO_REDIRECT_URL)
:close()

--CURL *curl = curl_easy_init();
--if(curl) {
--  curl_easy_setopt(curl, CURLOPT_URL, "https://example.com");
--
--  /* example.com is redirected, so we tell libcurl to follow redirection
 */
--  curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L);
--
--  curl_easy_perform(curl);
--}

maybe this(CURLOPT_FOLLOWLOCATION) syntax is other module 'lcurl' case Rf.https://lua-curl.github.io/lcurl/modules/cURL.html

dose require('cURL') allow

 curl_easy_getinfo(curl, CURLINFO_REDIRECT_URL, &url);
    if(url)
      printf("Redirect to: %s\n", url);

??

it's mean same as curl -IL -s https://somewhere.to.go | grep location

I want to do same function as here https://qiita.com/dauuricus/items/a7c756569a3d7bcf46f2 without dependency problem.Now I can't install lua-requests,http on termux (android userland terminal) Lua5.3.Because I can't compile lua-cjson,lua-cqueues yet.

Then I’m testing lua-cURL.There is no depndency problem.

moteus commented 3 years ago
local curl = require('cURL')

local easy = curl.easy{
  url            = "https://example.com",
  followlocation = true,
  maxredirs      = 2,
}

local buffer = {}
easy:setopt_writefunction(table.insert, buffer)

easy:perform()

print(table.concat(buffer))
print('-------------------')
print('code',     easy:getinfo_response_code())
print('redirect', easy:getinfo_redirect_url())

easy:close()

there several way to set options easy:setopt{url = ...} easy:setopt_url(....) easy:setopt(curl.OPT_URL, ...) The last two way allaw set context variable to callback functions (like in example).

dauuricus commented 3 years ago

@moteus Thank you for writing great example code.It is very very helpful. I 've tested just now. wonderful !

I've tested yesterday same test in lua-requests on chromebook(not termux. in debian container with no dependency problems).then lua-requests could't get my target ridirect url.that result was same as luasec ''wrong version number''.

so, awesome.

and then the test result response body was the url I expected, that is, the html text It was redirecting to, but the url address didn't contain the redirect url value. Perhaps the type of redirect that is normally expected is different from the redirect of my target.

-------------------
code    200     nil
redirect        nil     nil

I have been researching illegal Japanese manga websites on servers in China for the past three months. The illegal image content itself is in cloudflare and will probably not be displayed depending on the country region of the IP address (eg. Googlecolab gets 403).

I wrote some program in ruby, python, go, nim that confirms the existence of the page URL by referring to the php redirect destination provided by its site. Due to my curiosity, I want to run them all on termux, but in the case of Nim, there was a problem that the certificate was not recognized in termux. And again, out of curiosity, I wanted to write those programs in Lua, and I just started learning Lua. Even in the case of Lua, when I tried to communicate with https with http,httpclient, I got the same error as in the case of nim on termux. In the case of Lua, this problem could be avoided by giving the program code to specify the location of the certificate (which Nim in Termux can't do), but my knowledge is completely lacking in Lua. I can't reach information ... example code is very helpful for understanding the module 's functionality and limitations for beginner.

dauuricus commented 3 years ago

ok. I get 302 redirected url.

local curl = require('cURL')

local easy = curl.easy{
  url            = "https://somewhereto.go/redirect/?tour=/vol",
  --followlocation = true,
  followlocation = false,
  maxredirs      = 3,
}

local buffer = {}
easy:setopt_writefunction(table.insert, buffer)

easy:perform()

print(table.concat(buffer))
print('-------------------')
print('code',     easy:getinfo_response_code())
print('redirect', easy:getinfo_redirect_url())

easy:close()
-------------------
code    302     nil
redirect        https://somewhereto.go/2311047/  nil
dauuricus commented 3 years ago

Are there same option names and values in v3 ? https://curl.se/libcurl/c/easy_setopt_options.html Ex. CURLOPT_FOLLWLOCATION

for example I wanto

  1. check redirected URL
  2. connect to new location URL (and re set opt to easy object) (3. and more)

below "-->"

local curl = require('cURL')

local easy = curl.easy{
  url            = "https://somewhereto.go/redirect/?tour=/vol",
  --followlocation = true,
  followlocation = false,
  maxredirs      = 3,
}

local buffer = {}
easy:setopt_writefunction(table.insert, buffer)

easy:perform()

local res_code = easy:getinfo_response_code()
local res_location = easy:getinfo_redirect_url()

--print(table.concat(buffer))
print('-------------------')
print('code',     easy:getinfo_response_code())
print('redirect', easy:getinfo_redirect_url())

if res_code == 302 then
        easy:setopt(curl.OPT_URL,res_location)
-->     easy:setopt(curl.CURLOPT_FOLLWLOCATION, 2) -- number or true? / is this correct name? 'CURLOPT_FOLLWLOCATION'
end

easy:perform()

--print(table.concat(buffer))
print('-------------------')
print('code',     easy:getinfo_response_code())

easy:close()

when I use "easy:setopt()" followlocation must be change another name ... like OPT_FOLLWLOCATION or something.

is CURLOPT_FOLLWLOCATION name in cURLv3 ? I picked up that from libcurl 'CURLOPT_FOLLOWLOCATION'. https://curl.se/libcurl/c/easy_setopt_options.html

or is there v3 version options name list?

moteus commented 3 years ago

in the library names map to curl.OPT_FOLLOWLOCATION But as I said you can use several way to set the same options easy:setopt(curl.OPT_FOLLOWLOCATION, 1) easy:setopt{[curl.OPT_FOLLOWLOCATION] = 1} easy:setopt{followlocation = 1} easy:setopt_followlocation(1)

dauuricus commented 3 years ago

Thank you. I found https://github.com/Lua-cURL/Lua-cURLv2/blob/master/examples/ebaylogin.lua

Just now.

dauuricus commented 3 years ago

ok ...( all?? ) Lua-cURLv3 constants are derived from libcurl constants by removing the CURL prefix.

Rf. https://curl.se/libcurl/c/curl_easy_setopt.html https://curl.se/libcurl/c/curl_easy_getinfo.html

Ex: https://curl.se/libcurl/c/CURLINFO_EFFECTIVE_URL.html

CURLcode curl_easy_getinfo(CURL *curl, CURLINFO info, ... );

cURLv3

easy:getinfo_effective_url()
easy:getinfo(curl.INFO_EFFECTIVE_URL)

print(easy:getinfo_effective_url())
print(easy:getinfo(curl.INFO_EFFECTIVE_URL))

so I should write

local curl = require('cURL')

local easy = curl.easy{
  url            = "https://mangabank.org/watch/?tour=/vol",
  --followlocation = true,  -- true: 1
  followlocation = 0, -- false: 0
  maxredirs      = 3,
}

local buffer = {}
easy:setopt_writefunction(table.insert, buffer)

easy:perform()

local res_code = easy:getinfo_response_code()
local res_location = easy:getinfo_redirect_url()

print('-------------------')
print('code: ',     easy:getinfo_response_code())
print('redirect:', easy:getinfo_redirect_url())
print('url: ', easy:getinfo_effective_url())
print('url: ', easy:getinfo(curl.INFO_EFFECTIVE_URL))  -- same as above

if res_code == 302 then
        easy:setopt(curl.OPT_URL,res_location)
        --easy:setopt(curl.OPT_FOLLOWLOCATION,1) -- true: 1 / false: 0                                              
        --easy:setopt{[curl.OPT_FOLLOWLOCATION] = 1}  -- same as above
        --easy:setopt{followlocation = 1}             -- same as above                                              
        --easy:setopt_followlocation(1)               -- same as above
        easy:setopt{followlocation = true} -- true: 1

easy:perform()

print('-------------------')
print('code: ',     easy:getinfo_response_code())
print('redirect: ', easy:getinfo_redirect_url())
print('final')
----print(table.concat(buffer))

print('url: ',easy:getinfo_effective_url())

easy:close()