[BUG] 'trust_remote_code' needs to be set to True or atleast a method to pass this information as required.

[krishnanpooja]

Describe the bug While using HuggingFace model via deepspeed-mii infernece engine. I have issue while loading the model, that the trust_remote_code is not set. I have no params to pass this info here https://github.com/microsoft/DeepSpeed/blob/ade7149db491cec71d49814013f1be2d7041dbdc/deepspeed/inference/v2/checkpoint/huggingface_engine.py#L24

Expected behavior User should be able to set the trust_remote_code value.

ds_report output Please run ds_report to give us details about your setup.

Screenshots ValueError: The repository for /mnt/**/INPUT_model_path contains custom code which must be executed to correctly load the model. You can inspect the repository content at Please pass the argument trust_remote_code=True to allow custom code to be run.

System info (please complete the following information):

• OS: [e.g. Ubuntu 18.04]
• GPU count and types [e.g. two machines with x8 A100s each] - 1
• (if applicable) what DeepSpeed-MII version are you using : 0.2.4
• (if applicable) Hugging Face Transformers/Accelerate/etc. versions
• Python version - 3.10
• Any other relevant info about your setup

Docker context mcr.microsoft.com/azureml/curated/acpt-pytorch-2.2-cuda12.1:12

[jomayeri]

@krishnanpooja Please use this branch for now: https://github.com/microsoft/DeepSpeed/pull/6378