Because the new file schema must know ahead of time where to start writing tensor data, there must be a certain amount of pre-allocated space for the metadata and headers. Thus you must specify max_tensors to TensorSerializer.init If you plan on callling tensor_write() more than once. This does a wild-ass-guess estimate on a good amount of header space to reserve. In some cases, like really long tensor names, it could still be an underestimate. You will get a RuntimeError() if you call write_tensor() and there isn't enough pre-allocated space.
You do not need to set max_tensors if you just call tensor_write() once.
Write docs on how/when to set the
max_tensorsargCopied from PR:
max_tensors
Because the new file schema must know ahead of time where to start writing tensor data, there must be a certain amount of pre-allocated space for the metadata and headers. Thus you must specify max_tensors to TensorSerializer.init If you plan on callling tensor_write() more than once. This does a wild-ass-guess estimate on a good amount of header space to reserve. In some cases, like really long tensor names, it could still be an underestimate. You will get a RuntimeError() if you call write_tensor() and there isn't enough pre-allocated space.
You do not need to set max_tensors if you just call tensor_write() once.