"tfidf_math_arxiv_titles.pkl" returns error when loading

Rubiel1 commented 2 weeks ago

Hi, I keep on getting this error

  File ".../recommender/recommender/initialize_model.py", line 35, in initialize
    math_papers = pd.read_pickle('recommender/arxiv_data/tfidf_math_arxiv_titles.pkl')
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../.conda/envs/py311/lib/python3.11/site-packages/pandas/io/pickle.py", line 206, in read_pickle
    return pickle.load(handles.handle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
_pickle.UnpicklingError: invalid load key, 'v'.

Do you have any suggestion?

Rubiel1 commented 2 weeks ago

So I found this info

blakestad commented 2 weeks ago

I think the most upvoted answer on the stack exchange post you linked to might be your answer. Do you have Git Large File Storage (git lfs) set up on your computer? The models that run the recommender (and the subject classifier) are too big to host on github itself, so they are hosted through its lfs service.

blakestad / recommender

"tfidf_math_arxiv_titles.pkl" returns error when loading #2