SGLang is a structured generation language designed for large language models (LLMs). It makes your interaction with models faster and more controllable.
In python/sglang/srt/memory_pool.py TokenToKVPool
self.mem_state = torch.zeros((size,), dtype=torch.int16, device="cuda")
but ReqToTokenPool
self.mem_state = torch.ones((size,), dtype=torch.bool, device="cuda")
They are different, can I change all dtype into torch.bool？
In python/sglang/srt/memory_pool.py TokenToKVPool self.mem_state = torch.zeros((size,), dtype=torch.int16, device="cuda") but ReqToTokenPool self.mem_state = torch.ones((size,), dtype=torch.bool, device="cuda") They are different, can I change all dtype into torch.bool？