Stability fixes - Githubissues

runpod-workers / worker-infinity-embedding

MIT License

18 stars 9 forks source link

Closed michaelfeil closed 4 months ago

michaelfeil commented 5 months ago

Adding some suggested fixes:

adding HF_TRANSFER via infinity requirements. It will ship soon as default, is always installed with [all]
lowering the batch_size: Has no significant performance improvments. 32 or 64 is suggested to not OOM for large batch sizes.
HF_HOME is respected to download models.

@alpayariyak

michaelfeil commented 5 months ago

@alpayariyak Please review, thanks!

michaelfeil commented 4 months ago

Hey @alpayariyak, coming by your SF meetup in-person on Friday night, maybe we can fix this together.

michaelfeil commented 4 months ago

Close in favor of #4