Closed JaeseungYeom closed 5 months ago
We should not duplicate the reading part. We should just split the functionality and call the same function.
Have consume_data do the I/O consume can do fetch of metadata and call consume_data
I understand it will make it more consistent. However, it will also needlessly allow redundant KVS traffic even with C/C++ when python is not in use. The original form is the best, and addition of the two calls dyad_get_metadata()
and dyad_consume_w_mdata()
is a compromise for python. Do we want to apply that to C and C++? For example, the same file is read locally 10 times, the separate interface will increase the KVS traffic by 10 times.
What is your opinion on the meta_data deallocation?
consume_w_metadata()
.
dyad_consume()
that returns the buffer that has never been assigned.dyad_consume()
that relies on the metadata that has already been pulled from KVSdyad_consume_w_mdata(..., mdata)
deallocates mdata internally. We can change this behavior depending on how we want to write python interface.