search

flashinfer-ai / flashinfer

FlashInfer: Kernel Library for LLM Serving
https://flashinfer.ai
Apache License 2.0
1.1k stars 98 forks source link

[TVMWrapper] Support auxiliary DLTensor with byte offset #193

Closed MasterJH5574 closed 5 months ago

MasterJH5574 commented 5 months ago

This PR adds the byte_offset support to the auxiliary DLTensors. This is necessary when all the auxiliary DLTensors are slices of a large storage.