search

PMunch / futhark

Automatic wrapping of C headers in Nim
MIT License
367 stars 19 forks source link

Trying to wrap OpenWrt library libubus #20

Closed keyro90 closed 2 years ago

keyro90 commented 2 years ago

Hi, as the title, I'm trying to use the OpenWrt libubus.h with futhark as dynamic library. This is the code:

import futhark

importc:
  sysPath "/opt/toolchains/openwrt-build/staging_dir/toolchain-mipsel_24kc_gcc-7.5.0_musl/include"
  path "/opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include"
  define LIBUBUS_IMPLEMENTATION
  "libubus.h"

static:
  writeFile("test.c", """
  #define LIBUBUS_IMPLEMENTATION
  #include "/opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include/libubus.h"
  """)
{.compile: "test.c".}

compiling with:

nim c -d:release --passL="-lubus" -d:useMalloc --checks=off --stackTrace=off --verbosity=1 --cpu=mipsel  main.nim

I get this error:

/home/pippo/Documents/sources/shits/test-wrapper-ubus/main.nim(5, 1) template/generic instantiation of `importc` from here
/home/pippo/.nimble/pkgs/futhark-0.5.0/futhark.nim(456, 14) template/generic instantiation of `importcImpl` from here
/home/pippo/.nimble/pkgs/futhark-0.5.0/futhark.nim(516, 11) Error: Unable to parse output of opir:
Unknown element in structure or union: CXCursor_AlignedAttr (filename: "/opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include/ubusmsg.h", line: 34, column: 3)

And I don't know how to solve it.

In particular, the error is here on ubusmsg.h:

struct ubus_msghdr {
        uint8_t version;
        uint8_t type;
        uint16_t seq;
        uint32_t peer;
} __packetdata;
PMunch commented 2 years ago

Just pushed a new version which should support alignment. Note that I haven't tested this with anything that actually uses alignment, I just copied the relevant snippet from your header file and checked that it was able to generate code with alignment statements.

By the way, since you're building with a dynamic library you don't have to do this:

static:
  writeFile("test.c", """
  #define LIBUBUS_IMPLEMENTATION
  #include "/opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include/libubus.h"
  """)
{.compile: "test.c".}

Simply have the importc that generates undefined symbols, and then when you link to the dynamic library with --passL="-lubus" it should fill out those symbols for you.

sillagiusti commented 2 years ago

Thank you for the support. Now, with 0.5.1 version with this source code:

import futhark

importc:
  sysPath "/opt/toolchains/openwrt-build/staging_dir/toolchain-mipsel_24kc_gcc-7.5.0_musl/include"
  path "/opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include"
  define LIBUBUS_IMPLEMENTATION
  "libubus.h"

this error is shown:

Hint: used config file '/home/agiusti/.choosenim/toolchains/nim-1.6.4/config/nim.cfg' [Conf]
Hint: used config file '/home/agiusti/.choosenim/toolchains/nim-1.6.4/config/config.nims' [Conf]
........................................................................................................
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(509, 12) Hint: Using cached Opir output: /home/pippo/.cache/nim/main_d/opir_66638746.json [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(521, 8) Hint: Parsing Opir output [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(535, 8) Hint: Generating Futhark output [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "ptr" to "ptrarg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "method" to "methodarg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "ptr" to "ptrarg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typefield" in structubusobject [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "method" to "methodarg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "object" to "objectfield" in structubusaclkey [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typearg" [User]
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(66, 10) Hint: Renaming "type" to "typefield" in structblobmsgpolicy [User]
stack trace: (most recent call last)
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(616, 21) importcImpl
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(315, 59) createStruct
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(185, 24) toNimType
/home/pippo/.choosenim/toolchains/nim-1.6.4/lib/pure/collections/tables.nim(246, 18) []
/home/pippo/.choosenim/toolchains/nim-1.6.4/lib/pure/collections/tables.nim(234, 5) raiseKeyError
/home/pippo/Documents/sources/prism/test-ubus-wrapper/main.nim(3, 1) template/generic instantiation of `importc` from here
/home/pippo/.nimble/pkgs/futhark-0.5.1/futhark.nim(469, 14) template/generic instantiation of `importcImpl` from here
/home/pippo/.choosenim/toolchains/nim-1.6.4/lib/pure/collections/tables.nim(234, 5) Error: unhandled exception: key not found: blobmsg_type [KeyError]

blobmsg_type is in /opt/toolchains/openwrt-build/staging_dir/target-mipsel_24kc_musl/usr/include/libubox/blobmsg.h. So, How can I specify path to reach this library?

PMunch commented 2 years ago

Hmm, seems like another edge-case. Do you have a link to the sources you are trying to wrap so I can run it myself?

PMunch commented 2 years ago

Do you have a less sketchy link where I don't have to accept terms and services to some service I've never heard of to use it? Like a GitHub repository for example?

sillagiusti commented 2 years ago

No problem, but I had to split tar.gz file. Repo here: https://github.com/sillagiusti/openwrt-staging-dir-mipsel

PMunch commented 2 years ago

Why are you uploading a tar.gz file to GitHub? That's not how GitHub is supposed to be used, just take the stuff you have in the tar.gz and check that into your repo.

sillagiusti commented 2 years ago

Why are you uploading a tar.gz file to GitHub? That's not how GitHub is supposed to be used, just take the stuff you have in the tar.gz and check that into your repo.

I know, but openwrt SDK generates a lot of file with >= 100MB size. So, I can't push them and they're mandatory to compile. If you have any other solution to upload it, I'll do it.

PMunch commented 2 years ago

Hmm, that sounds weird. How do you set up the toolchain?

sillagiusti commented 2 years ago

Hmm, that sounds weird. How do you set up the toolchain?

You can download the default SDK here from official website.

You can find everything inside staging_dir.

PMunch commented 2 years ago

That doesn't contain the libubus.h file though, how do I get that?

sillagiusti commented 2 years ago

That doesn't contain the libubus.h file though, how do I get that?

I can give you the /usr folder inside staging_dir/target-mipsel_24kc_musl. Here

PMunch commented 2 years ago

Aha, the issue is that there is both a procedure blobmsg_type and an enum blobmsg_type declared in that file. Futhark doesn't take into account the different types of symbols for some of the steps in the generation. I've worked around this for structs and unions by prepending struct_ and union_ to their names, and I've now done something similar for enums. But I can't push right now because GitHub seems to have some trouble.. This isn't a great way to fix the problem though, it should rather take the type into account and then later just stick with the renaming scheme if there is a collision. This way you don't have to specify enum_blobmsg_type if there is no collision. I'm going to fix this ASAP

sillagiusti commented 2 years ago

Ok, many thanks for your time! I hope to contribute very soon to this project. It's very useful.

PMunch commented 2 years ago

I have managed to push the fix now. So it should be able to wrap your this code. I'm trying to design a good system for this, but I'm not entirely sure how I want it to work yet, but this patch will be good enough for now.

sillagiusti commented 2 years ago

Ok I'm going to test very soon.

sillagiusti commented 2 years ago

I have managed to push the fix now. So it should be able to wrap your this code. I'm trying to design a good system for this, but I'm not entirely sure how I want it to work yet, but this patch will be good enough for now.

It works! :)