event hooks for `lua-resty-mysql`

[sinogermany]

First of all, cosocket is cool :) Thanks for the excellent work @agentzh and other contributors. I could never imagine writing an async mysql / postgres driver in lua within 1k lines of codes.

Currently when we fetch a result set, we call db:query, which returns all the rows:

local row = _parse_row_data_packet(packet, cols, compact)
i = i + 1
rows[i] = row
-- ......
return rows

In many (or at lease some) cases however we don't need to keep all records in memory. It would be nice if we could do something like this (take json output as an example):

function row_event_hook(row)
  -- 'row' is returned by '_parse_row_data_packet' function, a lua table
  -- you can do something like ngx.say(cjson.encode(row)) or do whatever you want
  ngx.say(cjson.encode(row))
  -- the above is just an arbitrary example without details like commas etc.
end

ngx.say('[')
-- if row_event_hook is provided, records are not saved in the memory by default
-- if row_event_hook is not provided, return the array of records as usual
db:query('SELECT foo FROM bar', row_event_hook)
ngx.say(']')

I do understand it's probably not worth it in the above example because it was mentioned in another issue that ngx.say doesn't buffer (it would be nice to have ngx.write_to_buffer function :) ). But there are other scenarios where a row_event_hook does make sense to prevent the request cycle from unnecessary memory allocations. Like writing BLOB data (blob_data_event_hook) through some IO stream (or HTTP response) etc.

So to wrap up, it would be nice to have some event hook support since we can potentially save some memory per request cycle, which is very important for a high concurrency web api / app.

Any suggestions / comments welcome.

[agentzh]

@sinogermany I can buy the idea of lazily returning the data rows but I don't buy the event hook API. Basically we can expose a fetch_row function to extract data row by row, kinda similar to PHP's mysql_fetch_row() function. This has indeed been on my TODO list though I haven't found the time to look into this. Will you contribute a patch for that? Thanks!

[agentzh]

@sinogermany BTW, we've been trying hard to avoid asynchronous callback APIs because we know how horrible the control flow can be when looking at native NodeJS code :)

[sinogermany]

Will you contribute a patch for that? Thanks!

Yeah sure. I'll try to do that in the weekend :) I'm just not familiar with Test::Nginx::Socket::Lua though. Let's see what I can contribute. BTW thanks for your fast response @agentzh . You're awesome.

[agentzh]

@sinogermany Great! BTW, you can find documentation for the test scaffold here:

https://metacpan.org/pod/Test::Nginx::Socket::Lua

and here:

https://metacpan.org/pod/Test::Nginx::Socket

Please let me know if you have any further questions :)