search

EmergentOrder / onnx-scala

An ONNX (Open Neural Network eXchange) API and backend for typeful, functional deep learning and classical machine learning in Scala 3
GNU Affero General Public License v3.0
137 stars 8 forks source link

Question: why does `Tensor#apply` return a value wrapped in `IO` effect? #377

Closed i10416 closed 1 year ago

i10416 commented 1 year ago

Why does Tensor#apply return a value wrapped in IO effect?

Is there any resource allocation or unsafe invocation to native land?

EmergentOrder commented 1 year ago

Yes, there is native resource allocation, under the hood ONNX Runtime is used and data needs to copied from the JVM into native memory. In addition to this, the transition to using IO here allows for the Scala JVM and Scala JS (JavaScript) APIs to be unified, because onnxruntime-web and onnxruntime-node return Promises, which cannot be awaited because blocking is not allowed in ScalaJS (see: https://github.com/scala-js/scala-js/issues/186#issuecomment-33732043 ).