How to inference the model?

[Dandelionym]

Hi, is there any code example to call the model? any interface/inference code?

[HanYuanHsu]

I also need a code example. I was trying to run of_hfmodel.py by typing python of_hfmodel.py in the command line prompt. But I believe it needs to read in another argument because I saw the following part in the code (note the sys.argv[1]):

with open(sys.argv[1]) as f: 
        cfg = yaml.safe_load(f)

    hfmodel = cfg["hfmodel"]
    BATCH_SIZE = cfg["batch_size"]
    BEFORE_EXAMPLE = cfg["before_example"]
    AFTER_EXAMPLE = cfg["after_example"]
    IN_KEY = cfg["in_key"]
    OUT_KEY = cfg["out_key"]
    ref_key = cfg["ref_key"]
    save_dir = cfg["save_dir"]
    save_file = cfg["save_file"]
    few_shot_prompt_path = cfg["few_shot_prompt_path"]
    STOP = cfg["stop"]
    max_tokens = cfg["max_tokens"]
    split = cfg["split"]

I believe cfg is some kind of dataset, but I don't know what it is and where to find it.

[zhangir-azerbayev]

cfg is a config file, which are located in eval/configs. So if you're in the eval directory example usage is python of_hfmodel.py configs/pythia-6.9b_6shot_formalize.yaml.