Repeated GetBuffer calls by a client instance for a single get operation.

[ysbqiaqia]

Hi vineyard developers,

Recently, we reaserch vineyard flow, I found that when I do one single get operation in python script, client executed multiple GetBuffer requests. I added a print log at command_t::GET_BUFFERS_REQUEST state in the processMessage state mechine （at line 259 in the socket_server.cc）. The printed log show that at least three invocations of the GetBuffer request. Here is the log: from vineyardd:

// log position:
  } else if (cmd == command_t::GET_BUFFERS_REQUEST) {
    printf("[\033[33m%s\033[0m] %s %s doGetBuffers\n", my_time_stamp().c_str(), __FILE__, __FUNCTION__);
    return doGetBuffers(root);
}
...
} else if (cmd == command_t::GET_DATA_REQUEST) {
    printf("[\033[33m%s\033[0m] %s %s doGetData\n", my_time_stamp().c_str(), __FILE__, __FUNCTION__, __LINE__);
    return doGetData(root);
  } 

[2024-05-15 18:03:36.939] v6d/src/server/async/socket_server.cc processMessage doGetData
[2024-05-15 18:03:36.941] v6d/src/server/async/socket_server.cc processMessage doGetBuffers
[2024-05-15 18:03:36.942] v6d/src/server/async/socket_server.cc processMessage doGetData
[2024-05-15 18:03:36.945] v6d/src/server/async/socket_server.cc processMessage doGetBuffers
[2024-05-15 18:03:36.948] v6d/src/server/async/socket_server.cc processMessage doGetData
[2024-05-15 18:03:36.949] v6d/src/server/async/socket_server.cc processMessage doGetBuffers

Test Demo:

#-------------------- ipc_put.py
import vineyard
import numpy as np
import os, sys

if __name__ == '__main__':
    if len(sys.argv) != 2:
        print("using : python main.py size(KB unit)")
        exit(-1)

    client = vineyard.connect('/var/run/vineyard.sock')
    b4 = 2 * np.ones((128,int(sys.argv[1])),dtype=int)
    obj_id = client.put(b4)
    print(obj_id)

#--------------------  ipc_get.py
import vineyard
import numpy as np
import os, sys
if __name__ == '__main__':
    client = vineyard.connect('/var/run/vineyard.sock')
    if len(sys.argv) != 2:
        print("using : python main.py obj-id")
        exit(-1)
    obj = vineyard.ObjectID(str(sys.argv[1]))
    array = client.get(obj)
    print(array)

Tracing the flow of the code as below:

1. In the resover.py(line 235), get function call the obj = client.get_object(object_id) function to retrieve an object (obj).

   
   def get_object(self, object_id: ObjectID) -> Object:
       """
       Fetches the object associated with the given object_id from Vineyard.
       The IPC client is preferred if it's available, otherwise the RPC client
       """
       return self._fetch_object(object_id)  # fetch the object

def _fetch_object(self, object_id: ObjectID) -> Object: meta = self.get_meta(object_id) # get_meta-->GetMetaData-->GetBuffers

    if self.has_ipc_client():
        if meta.instance_id == self._ipc_client.instance_id or meta.isglobal:
            return self._ipc_client.get_object(object_id, fetch=False)  # GetObject-->GetMetaData-->GetBuffers
        else:
            warnings.warn(
                f"Migrating object {object_id} from another vineyard instance "
                f"{meta.instance_id}"
            )
            return self._ipc_client.get_object(object_id, fetch=True)
    if self.has_rpc_client():
        if self._rpc_client.is_fetchable(meta):
            return self._rpc_client.get_object(object_id)
        else:
            return self._locate_and_fetch(meta)

2. Inside the **get_object** function, call the **get_meta** function to obtain the metadata (meta) of the object.

"get_meta", [](Client* self, ObjectIDWrapper const& object_id, bool const sync_remote) -> ObjectMeta { ObjectMeta meta; throw_on_error(self->GetMetaData(object_id, meta, sync_remote)); // Call the GetBuffers internally return meta; }, "object_id"_a, py::arg("sync_remote") = false, doc::IPCClient_get_meta)

3. Inside the **get_meta** function, client calls the **GetBuffer** function to request the buffer of the object from **vineyardd**.
4. When the **get_object** function is called from **client.py +581**, internally (in **client.cc** on line **393**), the GetMetaData function is re-invoked.

Status Client::GetObject(const ObjectID id, std::shared_ptr

[dashanji]

@ysbqiaqia Thanks for the detailed steps. You are right, there are indeed multiple GetBuffer calls.

The current design is to simplify the API configuration. When dealing with multiple vineyard nodes on k8s, we first need to check the metadata to determine which vineyard node the object is stored on. Then, we can create a corresponding IPC/RPC client to obtain the object, as it may exist locally or remotely.

The GetMetaData function consists of two steps: GetData (to get metadata from etcd) and GetBuffer (to get the payload from vineyard). As you mentioned, both get_meta and get_object will invoke the GetMetaData function. One potential solution to address the issue of repeatedly calling GetBuffer and GetData is to split the GetMetaData function.

We will fix it quickly and thanks again!

[ysbqiaqia]

GetMetaData

yes, I think get_meta function just get meta from etcd, and GetBuffer can be called in some where of get_object function.