trust_remote_code not honored

[AmazingTurtle]

The bug Passing trust_remote_code=True is not honored for every invocation of the model.

To Reproduce

from guidance import models
lm = models.Transformers('openbmb/MiniCPM-Llama3-V-2_5', trust_remote_code=True)

print(lm + "what is love?")

See image attached

[riedgar-ms]

Can you let us know platform/python version etc., please?

I have just tried this on my laptop, and the error I get is:

Traceback (most recent call last):
  File "C:\Users\riedgar\AppData\Local\miniconda3\envs\guidance-312\Lib\site-packages\transformers\dynamic_module_utils.py", line 609, in resolve_trust_remote_code
    prev_sig_handler = signal.signal(signal.SIGALRM, _raise_timeout_error)
                                     ^^^^^^^^^^^^^^
AttributeError: module 'signal' has no attribute 'SIGALRM'. Did you mean: 'SIGABRT'?

During handling of the above exception, another exception occurred:

<snip>

  File "C:\Users\riedgar\AppData\Local\miniconda3\envs\guidance-312\Lib\site-packages\transformers\dynamic_module_utils.py", line 625, in resolve_trust_remote_code
    raise ValueError(
ValueError: The repository for openbmb/MiniCPM-Llama3-V-2_5 contains custom code which must be executed to correctly load the model. You can inspect the repository content at https://hf.co/openbmb/MiniCPM-Llama3-V-2_5.
Please pass the argument `trust_remote_code=True` to allow custom code to be run.

This looks like the actual issue (for me at least) is in the usage of the signal package, but this is surfaced as a trust_remote_code error. Then again, I'm using Windows, so that is probably affecting things as well.

[AmazingTurtle]

@riedgar-ms i used the latest jupyter notebook devcontainer https://github.com/devcontainers-community/templates-jupyter-datascience-notebooks

and mounted a notebook with the code above does that help you with reproduction?

[hudson-ai]

@AmazingTurtle are you able to load that model from the transformers library directly? This looks like a problem there, not in guidance itself.

Locally, I get a SIGKILL whether I am running via guidance or transformers directly.

[hudson-ai]

Actually, I'm able to instantiate the underlying transformers model if I specify the dtype as float16, but it's still failing from guidance. @riedgar-ms it looks like the tokenizer wants to be instantiated with trust_remote_code=True as well -- thinking that's the issue.

[AmazingTurtle]

@hudson-ai i can load the model with the transformers library directly and talk to it

[Harsha-Nori]

I wouldn't mind special casing trust_remote_code here...it's the only param that I think is really shared in name between the tokenizer and model config on a Transformers auto load (iirc transformers does this too under the hood). Anyone wanting to pass other tokenizer args can just initialize a new obj

@hudson-ai @riedgar-ms

[riedgar-ms]

There are probably other arguments common to HF models and tokenisers and defined on these high level factories. I'll check.

[riedgar-ms]

@Harsha-Nori @hudson-ai just created a new PR which expands a little on Hudson's original one.