cnuernber / dtype-next

A Clojure library designed to aid in the implementation of high performance algorithms and systems.
Other
319 stars 18 forks source link

Struct as a return or argument type #87

Closed rutenkolk closed 10 months ago

rutenkolk commented 10 months ago

Hi, I wanted to try out to write a binding for the raylib library.

The library has some functions that take structs directly as arguments (pass by value) or similarly, returns them this way.

Is there a way to use struct definitions from struct/define-datatype! in a ffi/define-library! call? If not, what is the preferred way of handling this with dtype-next?

example:


(dt-struct/define-datatype! :image
  [{:name :data :datatype :pointer} 
   {:name :width :datatype :int32} 
   {:name :height :datatype :int32} 
   {:name :mipmaps :datatype :int32} 
   {:name :format :datatype :int32}])

(ffi/define-library!
  raylib-native
  '{:GenImagePerlinNoise {:rettype :image :argtypes [[width :int32] [height :int32] [offsetX :int32] [offsetY :int32] [scale :float32]]}
    :ExportImageToMemory {:rettype :pointer :argtypes [[image :image] [fileType :pointer] [fileSize :pointer]]}
   }
  nil
  nil)

I currently recieve the following error:

Execution error at tech.v3.datatype.ffi.base/argtype->insn (base.clj:37).
Argument type :image is unrecognized

The error only occurs after a call to ffi/library-singleton-set! though.

Edit:

As an addendum: forgetting about the types for a moment, what is one supposed to do if the return value is of a size that doesn't fit any of the predefined

:int8 :int16 :int32 :int64 :float32 :float64 :size-t :pointer

types? I'm thinking anything from a custom struct, to a fixed size array, to a nondescript buffer of known size?

Is the "canonical" way to handle this currently to basically pretend this is an int8 array and reinterpret the bits later?

UPDATE:

I had a look at tech.v3.datatype.ffi.base and as far as I can see it, while the int8 array approach might work for arguments (albeit being hacky), I don't see a way dtype-next can currently handle any return value larger than 8 byte. Am I correct?

cnuernber commented 10 months ago

So this is partially a duplicate of #69 and I am working on that currently so you will have pass-by-value for structs. For structs that have fixed-size arrays as their members that is supported - but won't be in the initial pass-by-value work. This was required for avclj.

Speaking of which - have you taken a look at that project? If you are mapping a set of complex struct types back and forth then I recommend you use clang to create the struct definition and load structs from those - this process is described at the top of the ffi namespace and is used extensively in the avclj codebase.

I will let you know once I have success with pass-by-value.

rutenkolk commented 10 months ago

I have sadly only read that other issue later, since i thought this was specific to define-library-interface

i have done a cursory read of the parts of avclj that were referenced in the documentation and using clang to record struct layout and parse that. I still didn't understand how one can handle a return value larger than 8 bytes (how would one even use define-library! ? What to use for :rettype ?)

thanks for you work!

cnuernber commented 10 months ago

Right for return types larger than 8 bytes you need to have return-by-value structs -- working on that :-).

Current example - https://github.com/techascent/tmducken/blob/batched_results/src/tmducken/duckdb/ffi.clj#L364

harold commented 10 months ago

@rutenkolk - raylib looks cool, definitely interested to see what you come up with in this area. Anywhere I can follow along with your progress? (blog, repo, etc...)?

rutenkolk commented 10 months ago

@harold

sure: https://github.com/rutenkolk/raylib-clj

not sure if and where I might write about it. not thought about that yet. i should probably resurrect my blog :)

harold commented 10 months ago

Looks good! Let us know how it turns out - games are relevant to our interests (Chris and I used to work on console game dev middleware, and worked at NVIDIA after them acquiring the startup we were at).