Implement box-like API to access spaces

[Khatskevich]

It is very common need to just perform simple lookups, inserts... Those methods should work out of the box. Api suggestion: router:select(bucket_id, select_args) ...

Regarding fullscans (ddl without bucket_id), I am not sure if it should be supported.

[mtrempoltsev]

It will also be useful to perform the query on all nodes.

[Totktonada]

One of our projects uses the following router_proxy module as workaround:

local M = {}

local function _proxy(func_name, space_name, bucket_key, ...)
    local bucket_id = vshard.router.bucket_id(bucket_key)

    local call_func = 'box.space.' .. space_name .. ':' .. func_name
    local res, err = vshard.router.callrw(bucket_id, call_func, { ... })
    if err then
        return err.message
    end

    return res
end

function M.select(space_name, bucket_key, ...)
    return _proxy('select', space_name, bucket_key, ...)
end

function M.get(space_name, bucket_key, ...)
    return _proxy('get', space_name, bucket_key, ...)
end

function M.insert(space_name, bucket_key, ...)
    return _proxy('insert', space_name, bucket_key, ...)
end

function M.replace(space_name, bucket_key, ...)
    return _proxy('replace', space_name, bucket_key, ...)
end

function M.update(space_name, bucket_key, ...)
    return _proxy('update', space_name, bucket_key, ...)
end

function M.delete(space_name, bucket_key, ...)
    return _proxy('delete', space_name, bucket_key, ...)
end

function M.map_reduce_eval(code)
    local cluster = vshard.router.routeall()

    local result = {}
    for _, replicaset in pairs(cluster) do
        local master = replicaset.master

        local _, res = pcall(function()
            return master.conn:eval(code)
        end)

        result[#result + 1] = {
            uuid = master.uuid,
            res = res,
        }
    end

    return result
end

return M

[chetandhembre]

@Totktonada I am not seeing any code in your proxy, especially for insert function where we need to store bucket_id in table. My question is are you sending bucket_id from client while inserting row or your converting bucket_key to bucket_id in server side lua code?

[Totktonada]

We can reduce load of tx thread on a vshard router skipping excessive decoding-encoding of lua call arguments after implementing https://github.com/tarantool/tarantool/issues/3349

[Gerold103]

As a part of cluster box-like API, it was decided vshard should expose an API similar to box.space/netbox.space. It would look exactly like these, but with a few sharding-specific changes. Each non-select operation should get a bucket_id argument. Select() may use a bucket_id argument, or omit it and turn into a map-reduce (not sure about that though). Also each call should have a timeout, and a routing mode - 'prefer replica', 'balanced', and so on, like vshard.router.callbro/callbre/.... The API endpoint is going to be vbox. It not only will provide bucket-safe methods taking into account all bucket referencing stuff, but will also collect, merge, and expose cluster schema. It will fetch schema from each storage, compare them, and expose the set of sharded spaces on routers. It will expose only the spaces having the same schema on all storages, and will maintain that space set up to date. I will probably send an RFC with proper API description later.

[akudiyar]

https://github.com/tarantool/crud has been solving this task for some time already.