Closed BlackSamorez closed 1 year ago
Right now if a model contains any meta tensors it will fully be placed on meta device.
Normally all the data will be loaded when dispatching the model but unpersistent buffer will not be loaded and will remain on meta.
This PR fixes this behavior.
Right now if a model contains any meta tensors it will fully be placed on meta device.
Normally all the data will be loaded when dispatching the model but unpersistent buffer will not be loaded and will remain on meta.
This PR fixes this behavior.