`bpe_encoder` fails when file is not modified because `download()` returns `None`

promptslab / Promptify

Prompt Engineering | Prompt Versioning | Use GPT or other prompt based models to get structured output. Join our discord for Prompt-Engineering, LLMs and other latest research

https://discord.gg/m88xfYMbK6

Apache License 2.0

3.11k stars 232 forks source link

`bpe_encoder` fails when file is not modified because `download()` returns `None` #28

Closed escesare closed 1 year ago

escesare commented 1 year ago

When file is not modified, download() returns None here:

if response.status_code == requests.codes.not_modified:
     return

This poses a problem because then encoder_filename = None when bpe_encoder() tries to read the downloaded file:

encoder_filename = download(encoder_file["link"], encoder_file["filename"]

I believe download() should return the filepath even if file was already downloaded and doesn't need to be updated. Is my understanding correct?