search

PygmalionAI / aphrodite-engine

PygmalionAI's large-scale inference engine
https://pygmalion.chat
GNU Affero General Public License v3.0
722 stars 85 forks source link

fix: remove event and stream, add typing #382

Closed AlpinDale closed 2 months ago

AlpinDale commented 2 months ago

Remove the unused cache_stream and cache_events, since we now have CUDA graphs. Also add some generic typing to LRUCache.