search

sail-sg / lorahub

[COLM 2024] LoraHub: Efficient Cross-Task Generalization via Dynamic LoRA Composition
MIT License
583 stars 35 forks source link

About LoraHub adapters #7

Closed soroush-tabesh closed 1 year ago

soroush-tabesh commented 1 year ago

Nice work!

I needed clarification about the adapters you released on HF; the base model is flan-t5, and the adapters are trained on some Flan tasks! Am I mistaking something, or they're taught on the seen data? e.g., the ANLI task is a Flan2021 task(which probably flan-t5 has seen) and is included in your hub.

Also, are you planning to release the code to reproduce the numbers in your paper in the near future?

Thanks.

SivilTaram commented 1 year ago

@soroush-tabesh Hello, thanks for your interest on our work!

Yes, in all of our experiments, we use the tasks already seen by FLAN-T5 to compare, since it does not introduce unfair advantage of our method (no more new tasks or new datasets).

Sure, we're still cleaning the code and will release the full script to reproduce our results on BBH, thanks!

Best, Qian

SivilTaram commented 1 year ago

@soroush-tabesh Hello! We have released the reproduce script in reproduce_bbh.py. Please checkout the script! Please let me know if you have any follow-up problems!

soroush-tabesh commented 1 year ago

@SivilTaram Great. thanks!