search

recommenders-team / recommenders

Best Practices on Recommendation Systems
https://recommenders-team.github.io/recommenders/intro.html
MIT License
19.2k stars 3.1k forks source link

Unable to allocate 6.71 GiB for an array with shape (1800695893,) and data type int32 #1972

Closed shasha920 closed 1 year ago

shasha920 commented 1 year ago

Description

Hi, when I try to use tfidf recommenders

my code:

tf,vectors_tokenized=recommender.tokenize_text(df_clean,text_col=clean_col)

recommender.fit(tf, vectors_tokenized)

top_k_recommendations = recommender.recommend_top_k_items(df_clean, k=5)

Anaconda

MemoryError: Unable to allocate 6.71 GiB for an array with shape (1800695893,) and data type int32

Is anyone have same issue?

Other Comments

miguelgfierro commented 1 year ago

Your RAM probably doesn't support that much data. Try with 10% of the data, and then iterate adding more and more data until you reach your limit.

shasha920 commented 1 year ago

Your RAM probably doesn't support that much data. Try with 10% of the data, and then iterate adding more and more data until you reach your limit.

Thank you! I just want to you know before I ask question about can't run pip install recommenders on google colab and M1 Anaconda, you reply me you are continue working on python 3.10. My solve problem is use windows Anaconda, then downgraded Anaconda to python 3.8, so it is works. :)