Generate bindings via `cl-autowrap`

[nagy]

It seems like the raylib.lisp file is a rather manually written binding file to raylib. Would it make to you to generate these via https://github.com/rpav/cl-autowrap ?

If you want I can draft something out.

[longlene]

Thanks, will try it later.

[compufox]

@nagy could also try doing something similar with https://github.com/borodust/claw

[longlene]

Thanks all for the recommendation. I've read some code of both cl-autowrap and claw. I think the method of the code generation is not the key point of this project now, write manually or generate automatically are both ok to me.

The key issue is that raylib use both struct and pointer to the same struct at the same time, some API can not work at all. I've got stuck here for a long time.

[nagy]

Thank you @longlene for the investigation.

The key issue is that raylib use both struct and pointer to the same struct at the same time, some API can not work at all.

Do you have an example of such a case?

[longlene]

@nagy RLAPI void SetCameraMode(Camera camera, int mode); RLAPI void UpdateCamera(Camera *camera); For these two API, I can't find a way to write binding for them.

[nagy]

I was investigating this a bit more and found the following. When I create a test file raylib-test.c like so:

typedef struct Vector3 {
    float x;
    float y;
    float z;
} Vector3;

typedef struct Camera3D {
    Vector3 position;
    Vector3 target;
    Vector3 up;
    float fovy;
    int projection;
} Camera3D;

typedef Camera3D Camera;

void SetCameraMode(Camera camera, int mode);
void UpdateCamera(Camera *camera);

I can use the tool from https://github.com/rpav/c2ffi to generate something that comes close to actual bindings. In case you are using the nix package manager, you can directly invoke it like so:

$ nix-shell -p c2ffi --run "c2ffi raylib-test.c -D sexp -N cl-raylib"
(in-package :cl-raylib)

;; raylib-test.c:1:16
(struct Vector3
  (x :float)
  (y :float)
  (z :float))

;; raylib-test.c:5:3
(typedef Vector3 (:struct Vector3))

;; raylib-test.c:7:16
(struct Camera3D
  (position Vector3)
  (target Vector3)
  (up Vector3)
  (fovy :float)
  (projection :int))

;; raylib-test.c:13:3
(typedef Camera3D (:struct Camera3D))

;; raylib-test.c:15:18
(typedef Camera Camera3D)

;; raylib-test.c:17:6
(function "SetCameraMode" ((camera Camera) (mode :int)) :void)

;; raylib-test.c:18:6
(function "UpdateCamera" ((camera (:pointer Camera))) :void)

$

As you can see this already generated something that comes close to what is written in raylib.lisp. The :pointer part is taken care of and it looks like we would just have to adapt to the typedef s a bit more.

I have not used it yet, but it looks like asdf has already builtin support for this c2ffi output.

https://common-lisp.net/project/cffi/manual/html_node/Groveller-ASDF-Integration.html

All we would need to do is to prepare a parsed version of raylib, such that asdf can integrate it. But I dont know how yet.

[longlene]

Sorry for the late response. I have created a small dynamic library and cffi binding to test them. Now I'm sure the code does not work. Please have a look at https://common-lisp.net/project/cffi/manual/html_node/Foreign-Structure-Types.html

I think we can not read and write a struct in both lisp and c at the same time.

[kiran-kp]

Hello, cl-autowrap does support return-by-value if that's what you're having trouble with. I'm testing using it now but I wanted to make sure I'm not misunderstanding what the issue being discussed here is.

For what it's worth, switching to autowrap will require users to re-write significant amount of code for a decent sized project since allocation/deallocation needs to be done manually and return values for functions using call-by-value needs to passed in as the first argument to the function.

Here's the relevant quote from the cl-autowrap readme:

Using its own facilities, autowrap now includes autowrap/libffi. This allows functions that pass and return structs to be called using autowrap. To use this, just load or :depends-on cl-autowrap/libffi instead of cl-autowrap:

  :depends-on (... :cl-autowrap/libffi ...)
  ...

Of course, this requires libffi be compiled and available to your lisp.

Usage mostly identical; functions called via libffi look and act the same as any others. The one exception is functions that return a struct by value:

(c-with ((return-value some-struct))
  (some-call-returning-some-struct return-value ...))

Calls returning a struct take the return as their first parameter.

[longlene]

Hi, @kiran-kp return-by-value is not the point, the question is that: There are functions that use the value of the Camera structure and a pointer to the value of the Camera structure as arguments. It is just like that: https://github.com/longlene/cl-raylib/issues/12#issuecomment-927331874

We did can define functions at the same time, but they can not be called at the same time.

[kiran-kp]

Thanks for responding. I'm not sure I follow though. Neither of the functions you listed take a pointer to the camera and it's member at the same time and I'm not sure I understand what could go wrong if it did.

[kiran-kp]

Separately though, I've been trying out cl-autowrap for raylib and there seems to be issues with how typedefs are handled. I'm specifically having issues with Texture/Texture2D and am investigating a fix.

[longlene]

Thanks for responding. I'm not sure I follow though. Neither of the functions you listed take a pointer to the camera and it's member at the same time and I'm not sure I understand what could go wrong if it did.

@kiran-kp The UpdateCamera function need a pointer to a Camera struct and will modify the struct, the SetCameraMode require a Camera struct as parameter, so we can't define a variable in lisp level, please correct me if I'm wrong.

[nagy]

The UpdateCamera function need a pointer to a Camera struct and will modify the struct, the SetCameraMode require a Camera struct as parameter, ... please correct me if I'm wrong.

I think you are correct, I understand it the same way.

so we can't define a variable in lisp level,

You could not define a variable using the #'cl:defstruct function. All this memory would be allocated in the c world. But you still get a lisp value back from cffi which holds a pointer to this C memory. Also, the value would even be garbage-collectable, because these lisp-values could have a "finalizer" attached and then the garbage collector would do the cleanup in the C-world.

I don't think we would loose any functionality. In my experiments, the generated code was almost identical with the one that we are currently having.

[kiran-kp]

If I'm understanding the problem correctly, something like this should work:

(defcfun ("UpdateCamera" update-camera%) :void
 (camera (:pointer (:struct %camera3d))))

(defmacro update-camera (camera)
  (let ((foreign-camera (gensym)))
    `(cffi:with-foreign-object (,foreign-camera (:struct %camera3d))
       (cffi:translate-into-foreign-memory ,camera %camera3d-tclass ,foreign-camera)
       (update-camera% ,foreign-camera)
       (setf (camera3d-position ,camera) (cffi:foreign-slot-value ,foreign-camera '(:struct %camera3d) 'position))
       (setf (camera3d-target ,camera) (cffi:foreign-slot-value ,foreign-camera '(:struct %camera3d) 'target))
       (setf (camera3d-up ,camera) (cffi:foreign-slot-value ,foreign-camera '(:struct %camera3d) 'up))
       (setf (camera3d-fovy ,camera) (cffi:foreign-slot-value ,foreign-camera '(:struct %camera3d) 'fovy))
       (setf (camera3d-projection ,camera) (cffi:foreign-slot-value ,foreign-camera '(:struct %camera3d) 'projection)))))

It would be nice to have the setf forms be generated by a macro and I'm playing around with something for that so I'll update when I have it working.

I've given up on cl-autowrap for what it's worth. It's very buggy and does not look like it's maintained anymore. I'll continue discussion of this in #15 if you decide to close this issue.

[nagy]

I've given up on cl-autowrap for what it's worth. It's very buggy and does not look like it's maintained anymore.

I have come to some of its limits also lately. Namely it hat troubles for me supporting variadic lists. However, it does support a graceful failure mode, meaning that it can try to translate functions and when it cannot, it does not fail. This could allow us to at least use what it was able to figure out correctly and only leave us to keep the implementations that it cant translate. Only of course if we decide to proceed with this.