Safetensors or to be sure not to load pickled weights

[wllhf]

I'm really grateful for infinity! Thanks a lot! We are thinking about using it in a project with high security demands. Is it possible somehow to only load models via safetensors (or other safe formats) and exclude loading pickled weights? Is there a way knowing which format was used for loading when starting up a model via Infinity?

[michaelfeil]

@wllhf Thanks for the positive feedback.

It inherits the behaviour of "SentenceTransformers", it will automatically filter certain files, and AFAIK not download the jaxformers weights / rustformers etc. Also the default behaviour is --trust_remote_code=True in the infinity cli. I think if you were to modify the safetensors only - you would shrink your attack vector against various other man-in-the-middle-attacks only slightly. Its only useful for slightly increased security when your testing our 10s/100s of models.

Here are 2 security suggestions from a MLE, if the project is a high security one:

• mirror the model you like under huggingface. Private or public - and then pull the model from wllhf. Only copy the files you want. If youre mega paranoid, use a github ci to do that.
• Bake the models into the dockerfile (e.g. also by github ci) - then download the docker including the image and kill all access tohuggingface.co and hf.co. huggingface_hub has also a offline mode env variable.

[michaelfeil]

@wllhf Does this solve the question for you?

[wllhf]

Yes. Thanks!