als implicit training and scoring slower

benfred / implicit

Fast Python Collaborative Filtering for Implicit Feedback Datasets

https://benfred.github.io/implicit/

MIT License

3.57k stars 612 forks source link

als implicit training and scoring slower #649

Closed Selva163 closed 1 year ago

Selva163 commented 1 year ago

Noticed when we are training the model on 'ml.t3.medium' (vCPU - 2 and Memory - 4 GiB) took only 2 mins on Sagemaker notebook while when running in sagemaker pipeline on larger machine 'ml.t3.2xlarge' (vCPU - 8 and Memory - 32 GiB) taking 20 mins. Also noticed similar runtime when running on containers.

Our idea is to get recommendations refreshed every 2 mins for the active users.

benfred commented 1 year ago

It's a little hard to say exactly what the problem is here - but can you verify that you've disabled internal threading on your BLAS library as suggested here https://github.com/benfred/implicit#optimal-configuration ?

Selva163 commented 1 year ago

Closing this issue as configuration helped.