Add `~initial` / `~final` boolean flags to JITted functions to determine if copying from / to host is needed

ahrefs / ocannl

OCANNL: OCaml Compiles Algorithms for Neural Networks Learning

BSD 2-Clause "Simplified" License

67 stars 2 forks source link

Add `~initial` / `~final` boolean flags to JITted functions to determine if copying from / to host is needed #188

Closed lukstafi closed 1 year ago

lukstafi commented 1 year ago

For the GPU backends, with a global on-device memory, both are needed. For CPU backends, maybe only ~final would be used, for cases of "non-recurrent" tensors (without read-before-write).

lukstafi commented 1 year ago

The current approach is that cross-device memory management will be manual.