The global memory pool and default allocator (#149) allocates memory given just a size. This memory is handled just as a raw pointer. For some workers (e.g. xmodel, tfzendnn and ptzendnn), the inputs and outputs of running an inference must be in custom "tensorbuffer" style classes that require a shape and datatype on construction. Since the batcher is now creating batches in general memory, it requires the worker to copy data into these objects before inference.
This was previously supported with custom buffer classes.
The global memory pool and default allocator (#149) allocates memory given just a size. This memory is handled just as a raw pointer. For some workers (e.g. xmodel, tfzendnn and ptzendnn), the inputs and outputs of running an inference must be in custom "tensorbuffer" style classes that require a shape and datatype on construction. Since the batcher is now creating batches in general memory, it requires the worker to copy data into these objects before inference.
This was previously supported with custom buffer classes.