turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.69k stars 283 forks source link

enable module type checking #649

Closed SecretiveShell closed 3 weeks ago

SecretiveShell commented 1 month ago

This PR adds module support for type checking using mypy, so that external projects that depend on exllamav2 can use static type checking.

Previously, when using exllamav2 you would experience the following error

error: Skipping analyzing "exllamav2": module is installed, but missing library stubs or py.typed marker  [import-untyped]