When using Binoculars() the following warning is shown:
WARNING: You are currently loading Falcon using legacy code contained in the model repository. Falcon has now been fully ported into the Hugging Face transformers library. For the most up-to-date and high-performance version of the Falcon model code, please update to the latest version of transformers and then load the model without the trust_remote_code=True argument.
Using python 3.9, macOS, freshly downloaded and installed repo from here
This is a warning from upstream and doesn't impact reproducibility or score calculation. The original results also have this because of transformers version used.
When using
Binoculars()
the following warning is shown:Using python 3.9, macOS, freshly downloaded and installed repo from here