More control over external data size

neuralmagic / sparsezoo

Neural network model repository for highly sparse and sparse-quantized models with matching sparsification recipes

Apache License 2.0

371 stars 25 forks source link

More control over external data size #412

Closed dbogunowicz closed 10 months ago

dbogunowicz commented 10 months ago

As discussed recently:

let's fold the splitting of big internal files into onnx_save
split the model.onnx into graph and constant tensors once the model.onnx is larger then DUMP_EXTERNAL_DATA_THRESHOLD (500MB)
let's allow the user to control the size of created external tensor files For usage see: https://github.com/neuralmagic/sparseml/pull/1983

mgoin commented 10 months ago

@bfineran @markurtz this seems quite small, i would recommend at least a few gigabytes, if not the relatively standard 10GB used in HF

rahul-tuli commented 10 months ago

~I thought we were splitting the model into data files if it was >400mb and not restricting the model.data file(s) sizes? The only restriction on model.data file sizes is for easier download/upload from/to sparsezoo for which 400mb seems small? Thoughts?~

Did not see @mgoin comment, can ignore this

dbogunowicz commented 10 months ago

@alexm-nm just giving you the heads up, that splitting the external data is now part of the save_onnx function