cant find anything about turbo.socket

[shadylabs]

cant find anything about "turbo.socket" in the API docs

[kernelsauce]

turbo.socket is more like a private/hidden API that can change. Its not supposed to be used directly, except for turbo.socket.new_nonblock_socket(), which basically only does:

local fd = ffi.C.socket(family, socket_type, protocol)
local rc = ffi.C.fcntl(fd, F_SETFL, O_NONBLOCK)
return fd

turbo.socket also contains all the defines for berkeley sockets; turbo.socket.AF_INET turbo.socket.SO_STREAM etc.

All relevant things should be documented. If not in the web documentation http://http://turbolua.org/doc/ it is documented in the source with standard LuaDoc format.

Is there anything in particular you are looking for?

[shadylabs]

i was looking exactly for raw tcp client and i found the implementation in one of the test files hence i know about the socket. I found TCP server useful for my custom communication protocol perfect but i needed same raw tcp client and couldn't find anything until mentioned unittest iostream.lua

[shadylabs]

i am using it exactly as u mentioned : turbo.socket.new_nonblock_socket() then i have the so called fd - a magic number impossible to figure until now

[kernelsauce]

Ok. I can add some details to the docs about creating sockets using both the C functions and simplified Lua ones. :)

[kernelsauce]

To help you a little on the way, it should be pretty simple to create sockets and put them into a IOStream object.

 local sock, msg = turbo.socket.new_nonblock_socket(
     turbo.socket.AF_INET,
     turbo.socket.SOCK_STREAM,
     0)
 if sock == -1 then
     error("Could not create socket. " .. msg)
 end

 local iostream = turbo.iostream.IOStream(sock)

 ...

[shadylabs]

Thanks now its clear, just i probably be wrong to put the loop instance after the sock: local iostream = turbo.iostream.IOStream(sock,loopinstance) this is what i did and i have trouble getting callback on data read but its probably something in my code.

[shadylabs]

I found where my problem is. The server i am connecting with, needs a byte sequence not string and here comes the question is it possible to do it with mentioned code and with Turbo tools or i have to look somewhere else? when i tried to use iostream:write() it sends string of chars stream, then i filled structures.buffer /with the hope of sending bytes/ and got exception when trying to iostream:write_buffer().

[kernelsauce]

What do you mean? The string type of Lua does not do anything with the input, which means it can be binary or whatever you want it to be... What did you fill the turbo.struct.buffer with? Are you refering to Unicode sequences?

If so there are numerous libraries for that...

[shadylabs]

I know the same, about the string type, i am sure my string is byte sequence but no luck so far. The task is to send simple byte sequence 0x0, 0x0, 0x0, 0x1, 0x15 (simple list command, no parameters) and to receive answer. First i tried with iostream:write() and i receive callback for the write finished but no response then i tried to fill the buffer and do the buffer method - same result. The server needs byte sequence -> http://docs.basho.com/riak/latest/dev/references/protocol-buffers/ . What do you think about the best way to send these 5 bytes?

[kernelsauce]

local ffi = require "ffi"

stream:write(ffi.string(ffi.new("char[5]", {0x0, 0x0, 0x0, 0x1, 0x15}), 5))

This will write what you want to the stream. :) Using buffer you just append the char* directly using append_right and pass the buffer as parameter. This will be marginally faster, but I wouldnt stress it...

You can also used a struct that you create a string of for more control of types etc...

local ffi = require "ffi"

ffi.cdef [[
    struct my_array {
        unsigned char b1;
        unsigned char b2;
        unsigned char b3;
        unsigned char b4;
        unsigned char b5;
    }
]]

local bin = ffi.new("struct my_array")
bin.b1 = 0x0
bin.b2 = 0x0
bin.b3 = 0x0
bin.b4 = 0x1
bin.b5 = 0x15

stream:write(ffi.string(bin, 5))

[shadylabs]

Thanks. I tried it as you said , with append_right yesterday . Anyway i prefer :write() method as you suggested. Going to try it now. What do you think about concatenating numbers in string is it possible? like s= 0 .. 0.. 0 .. 1 .. 15 Do we have number or char string at the end

[shadylabs]

Awesome this works as a charm. Thanks a lot.

[kernelsauce]

Well, as you say you need a byte sequence... Concating a string like that would produce a ASCII number sequence which does not correspond to byte values.

Hex dump using my methods would result in this sequence of 5 bytes:

00 00 00 01 15

Using your string method s= "0" .. "0".. "0" .. "1" .. "15" would result in:

30 30 30 31 31 35

Obviously miles apart...

[shadylabs]

well actually i meant s= 0 .. 0.. 0 .. 1 .. 15 but lua console returns "000115"

[shadylabs]

btw this s= "0" .. "0".. "0" .. "1" .. "15" returns same in console and it means lua concatenates only chars not numbers /numbers are converted to chars before concatenation / so if i have a table with number values is there any lua way to serialize those numbers into lua numbers string without using C functions?

[kernelsauce]

No because "0" is not the byte value 0x0, it is 0x30. And of course 0x15 is also not 15 in decimal, its 21, but thats besides the point. You cannot create a string of literal numbers and expect them to be in binary form.

See http://www.klcconsulting.net/ascii.htm

You have to treat these things as binary. Using a char[] array or struct is the only way of getting what you want. You could make a function that returns the correct binary representation of a table like {0x0, 0x0, 0x0, 0x1, 0x15}...

[shadylabs]

Agree, but for me this means string of numbers in Lua are useless because there are no valid concatenation operation, i can substring them, i can separate some parts or compare parts but cant add 2 strings of numbers the way it works with chars. Not to mention if we need exactly bytes not just any number.

[kernelsauce]

Numbers in strings are meant for stuff like this "Kim has " .. number_of_cows .. "cows in his barn.".

Anyway you can concat two byte sequences (inside strings) together:

local ffi = require "ffi"

local str = ffi.string(ffi.new("char[5]", {0x0, 0x0, 0x0, 0x1, 0x15}), 5)
local str2 = ffi.string(ffi.new("char[4]", {0xde, 0xad, 0xbe, 0xef}), 4)
local str3 = str..str2

io.write(str3)

Running "hexdump -C" on output will give you:

00000000  00 00 00 01 15 de ad be  ef                       |.........|

[shadylabs]

I see. I misunderstood the meaning of "string of numbers". Thanks. Now it makes things clear and i have various ways to do the job. I was sure the problem is with all the numbers, bytes, ascii chars conversions in lua, but reading that strings in lua could be numbers , decided i can concatenate. I don't feel comfortable with C code so ill look something else.

[kernelsauce]

It is not a Lua problem, the behaviour will be the same in Python, Java, C, Ruby etc. :)

[shadylabs]

Yeah i knew its my problem of uncertainty what type of data i am sending. Using .Net i always knew the type of data going to the sockets. Its my first Lua network app and i crashed into this. I found lpack quite comfort for the job. Thanks for the invaluable help John.