search

neuralmagic / nm-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs
https://nm-vllm.readthedocs.io
Other
251 stars 10 forks source link

[Misc]: Is Magic Wand SparseBitmaskStorageFormat open sourced? #402

Closed dhjoo98 closed 3 months ago

dhjoo98 commented 3 months ago

Anything you want to discuss about vllm.

Hello, I am interested in how the bitmask representation is processed in GPU. By scanning the nm-vllm code, SparseBitmaskStorageFormat seems to be from a pip package called nm-magic-wand.

I was wondering if the source code that pip package is not open sourced.

mgoin commented 3 months ago

Hi @dhjoo98 nm-magic-wand is closed source.