Add support for Nvidia Multi-process Service (MPS)

marieai / marie-ai

Integrate AI-powered Document Analysis Pipelines

MIT License

57 stars 3 forks source link

Open gregbugaj opened 7 months ago

gregbugaj commented 7 months ago

Describe the feature Add support for CUDA-MPS to take advantage of new optimizations.

Requesting:

[ ] Setup scripts (Installation / Ansible / Docker )
[ ] Refactor exiting GPU/CPU from initialize_device_settings to device independent mode
[ ] Update models to extend GPU/CPU support for GPU/CPU/MPS/XPU