ufal / factgenie

Data visualization and span-based annotation of generated texts
MIT License
18 stars 1 forks source link

Robust parsing of int and float model arguments #93

Open kasnerz opened 1 week ago

kasnerz commented 1 week ago

Currently, to correctly send arguments for the model API, we need to do something like this:

for arg in ["temperature", "top_p"]:
    if arg in self.config["model_args"]:
        self.config["model_args"][arg] = float(self.config["model_args"][arg])

for arg in ["max_tokens", "top_k", "seed"]:
    if arg in self.config["model_args"]:
        self.config["model_args"][arg] = int(self.config["model_args"][arg])

It would be better to try guessing the value from its format, e.g. 0.01 is probably a float, 5 is a an int, etc.

That would allow us to send even unfamiliar parameters in a correct format.

oplatek commented 1 week ago

See the argrs https://github.com/ollama/ollama/blob/main/docs/modelfile.md and d32ffc03fa136ec566299e1562f9ecc72da19ffb