search

IntelLabs / fastRAG

Efficient Retrieval Augmentation and Generation Framework
Apache License 2.0
1.29k stars 116 forks source link

FiDReader Local Model #6

Closed TuanaCelik closed 1 year ago

TuanaCelik commented 1 year ago

Hey, I noticed that in your examples that you ask for the user to add a local path for a trained fid model but as far as I can tell, we can just provide a HF model name and it works. At least, I got the knowledge graph notebook to work like that. So maybe there's no need for the assertion. This is what I did:

from fastrag.readers import FiDReader

fid_model_path = None  ## change this to the local FID model
#assert fid_model_path is not None, "Please change fid_model_path to the path of your trained FiD model"

reader = FiDReader(
    input_converter_tokenizer_max_len= 256,
    model_name_or_path="Intel/fid_flan_t5_base_nq", 
    num_beams=1, 
    min_length=2, 
    max_length=100, 
    use_gpu=False
)

Let me know if I'm missing something :) - really cool project btw! It's great to see some custom Haystack nodes 👋

danielfleischer commented 1 year ago

Hi! It should work with both, since it uses PreTrainedModel.from_pretrained under the hood. We can definitely improve the docs regarding that. Thanks!