Issue When Trying to Replicate the Result

[thuuyen98]

Hi team,

I recently came across your excellent work on Modular LLM and am excited to build upon it. I found the MBC library you trained at hf://zhan1993/mbc_library_phi2_icml and wanted to give it a try.

However, when running the following command:

python eval_library.py \ -k output_dir=an_expert_eval/ \ library_id=hf://zhan1993/mbc_library_phi2_icml \ pipeline_eval_tasks='bbh' \ merge_or_route='arrow'

I encountered the following error:

Traceback (most recent call last): File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 369, in run_eval(args) File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 251, in run_eval selector_config.selector_data_id = fetch_prototypes(args, library) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 120, in fetch_prototypes add_base_proto=args.add_base_proto, ^^^^^^^^^^^^^^^^^^^ AttributeError: 'EvaluationConfig' object has no attribute 'add_base_proto'`

Could you help me understand what might be causing this error? Is it related to a missing parameter in the configuration or a potential issue with the code or library setup?

[sordonia]

hi, sorry this in on our side, could you remove args.add_base_proto from line 120 in eval_library.py?

Thank you

[sordonia]

should work now if you pull main

[thuuyen98]

Hi, thank you for your response. I tried it, but another issue has come up. It seems like the argument 'add_base_proto' is still being used somewhere else.

Traceback (most recent call last):
  File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 368, in <module>
    run_eval(args)
  File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 250, in run_eval
    selector_config.selector_data_id = fetch_prototypes(args, library)
                                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/uyen/test/mttl/projects/modular_llm/eval_library.py", line 114, in fetch_prototypes
    return compute_arrow_embeddings(
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/workspace/uyen/test/mttl/mttl/models/containers/selectors/arrow_selector.py", line 24, in compute_arrow_embeddings
    cfg = ArrowTransformConfig(
          ^^^^^^^^^^^^^^^^^^^^^
TypeError: ArrowTransformConfig.__init__() got an unexpected keyword argument 'add_base_proto'

[sordonia]

tracked it down and merged, let me know