"not enough memory" for downloaded interrogator

[ooofest]

Hello, I decided to try the

blip2-opt-6.7b-coco

interrogator and it downloaded a large dataset

when it then tried to run on my directory of files, it apparently ran out of memory:

File "C:\Users\user\AppData\Local\Programs\Python\Python310\lib\site-packages\torch\nn\modules\linear.py", line 96, in init self.weight = Parameter(torch.empty((out_features, in_features), **factory_kwargs)) RuntimeError: [enforce fail at ..\c10\core\impl\alloc_cpu.cpp:72] data. DefaultCPUAllocator: not enough memory: you tried to allocate 268435456 bytes.

I have 64 GB of system RAM and over 30GB free, so this made me wonder if my GPU was being used. To that point, I have two RTX-3090 cards (24GB VRAM each), but that obviously wouldn't work for a larger memory need without some sort of active memory manager.

Is there some sort of additional headroom needed for the memory allocation?

Thanks.

[ooofest]

I closed various apps and services, such that 51GB of RAM was available, which enabled loading the blip2-opt-6.7b-coco model without any errors.

Summary: there are no free lunches here, you need enough RAM to load any desired models.