search

google / autocxx

Tool for safe ergonomic Rust/C++ interop driven from existing C++ headers
https://docs.rs/autocxx
Apache License 2.0
2.09k stars 133 forks source link

The FFI for the desired type is not generated for some reason #317

Closed Boscop closed 3 years ago

Boscop commented 3 years ago

Thanks for this crate, it seems very useful :) I'm trying to use it to build a Rust lib with an extern function that returns a Model, which represents a VCV Rack audio module (so that I can call this module creation function from a cpp glue dll to enable writing VCV Rack modules in Rust): https://vcvrack.com/manual/Building#setting-up-your-development-environment https://vcvrack.com/manual/PluginDevelopmentTutorial I downloaded the Rack SDK and built the demo C++ plugin successfully, but the Rust build fails. Any idea how to make it work? :)

Or any idea where I can inspect the generated code for ffi? I only found this D:\VCV-Rack\vcvrust\target\debug\build\vcvrust-c3303f75cfa90a31\out\autocxx-build-dir\rs\14020848728261679440.rs but not the full code after macro expansion:

# [allow (non_snake_case)] # [allow (dead_code)] # [allow (non_upper_case_globals)] # [allow (non_camel_case_types)] mod ffi { pub trait ToCppString { fn to_cpp (& self) -> cxx :: UniquePtr < cxx :: CxxString > ; } impl ToCppString for str { fn to_cpp (& self) -> cxx :: UniquePtr < cxx :: CxxString > { cxxbridge :: make_string (self) } } mod bindgen { pub mod root { # [allow (unused_imports)] use self :: super :: super :: cxxbridge ; # [allow (unused_imports)] use self :: super :: root ; } } # [cxx :: bridge] pub mod cxxbridge { unsafe extern "C++" { fn make_string (str_ : & str) -> UniquePtr < CxxString > ; include ! ("header.hpp") ; include ! ("rack.hpp") ; include ! ("autocxxgen.h") ; } } }

Expected Behavior

It compiles.

Actual Behavior

image

Steps to Reproduce the Problem

lib.rs:

use autocxx::include_cpp;

include_cpp!(
    #include "header.hpp"
    #include "rack.hpp"
    generate!("Model")
    safety!(unsafe)
);

#[no_mangle]
pub extern "C" fn model() -> *mut ffi::Model {
    unimplemented!()
}

build.rs:

use std::path::PathBuf;

fn main() {
    let mut b = autocxx_build::build(
        "src/lib.rs",
        &[
            &PathBuf::from(r"D:\VCV-Rack\vcvrust\src"),
            &PathBuf::from(r"D:\VCV-Rack\Rack-SDK\include"),
            &PathBuf::from(r"D:\VCV-Rack\Rack-SDK\dep\include"),
        ],
    )
    .unwrap();
    b.flag_if_supported("-std=c++11")
        .file(r"D:\VCV-Rack\Rack\dep\nanovg\src\nanovg.c")
        .compile("autocxx-vcv");

    println!("cargo:rerun-if-changed=src/main.rs");
    println!("cargo:rerun-if-changed=src/header.hpp");
}

Cargo.toml:

[package]
name = "vcvrust"
version = "0.1.0"
authors = []
edition = "2018"

[lib]
crate-type = ["staticlib"]

[dependencies]
autocxx = "0.5.3"
cxx = "0.3.4"

[build-dependencies]
autocxx-build = "0.5.3"

header.hpp:

#pragma once
#ifndef M_PI
    #define M_PI 3.14159265358979323846264338327
#endif

Btw

Model is a struct, defined in Rack-SDK\include\plugin\Model.hpp as:

struct Model {
    Plugin* plugin = NULL;
    std::vector<std::string> presetPaths;

    /** Must be unique. Used for saving patches. Never change this after releasing your module.
    The model slug must be unique within your plugin, but it doesn't need to be unique among different plugins.
    */
    std::string slug;
    /** Human readable name for your model, e.g. "Voltage Controlled Oscillator" */
    std::string name;
    /** List of tag IDs representing the function(s) of the module.
    Tag IDs are not part of the ABI and may change at any time.
    */
    std::vector<int> tags;
    /** A one-line summary of the module's purpose */
    std::string description;

    virtual ~Model() {}
    /** Creates a headless Module */
    virtual engine::Module* createModule() {
        return NULL;
    }
    /** Creates a ModuleWidget with a Module attached */
    virtual app::ModuleWidget* createModuleWidget() {
        return NULL;
    }
    /** Creates a ModuleWidget with no Module, useful for previews */
    virtual app::ModuleWidget* createModuleWidgetNull() {
        return NULL;
    }

    void fromJson(json_t* rootJ);
};

Specifications

adetaylor commented 3 years ago

Thanks for the details!

I wanted to make sure that either rack.hpp or header.hpp does #include Model.hpp? It does look rather like autocxx just isn't seeing Model.

You found the right place for the generated code, but it looks like it includes only the code for the built-in string operations, and nothing that would be generated from Model.

I'd be interested in logs using the instructions listed here but they can be a bit temperamental to get out from the cargo build process.

A few other points:

More generally I should warn you that this crate is still very immature and experimental, and so far every new codebase we run it against yields about 10 new bugs which it takes me about 2 weeks to fix. I am still very grateful for the real-world testing though so please don't let that put you off.

Boscop commented 3 years ago

Thanks for the quick reply :) Yes, rack.hpp has #include <plugin/Model.hpp>. But I get the same error when including it directly like this:

include_cpp! {
    #include "header.hpp"
    #include "plugin/Model.hpp"
    generate!("Model")
    safety!(unsafe)
}

Not sure why I had the old cxx version (I had added it via cargo add cxx, which chose the old version for some reason). I'm now using the latest version, but getting the same result/error. And when I set RUST_LOG to info and run cargo build, I get no bindgen output. Do I need to have bindgen.exe installed via cargo install?

adetaylor commented 3 years ago

I think I figured it out. Is this the codebase? If so - you'll want generate!("rack::plugin::Model") because it's in a C++ namespace.

And then we'll get to the real problems which this codebase produces, which are bound to be numerous.

Meanwhile I raised #322 and #323 for issues you've already encountered.

adetaylor commented 3 years ago

@Boscop I'm going to close this because I think the previous comment gives the explanation. Thanks for the report. I expect you to find more problems, please let me know how you get on.

Boscop commented 3 years ago

Is this the codebase?

Yes. The full Rack SDK can be downloaded here: https://vcvrack.com/downloads/ See also: https://vcvrack.com/manual/PluginDevelopmentTutorial

If so - you'll want generate!("rack::plugin::Model") because it's in a C++ namespace.

Ah right. It's been a long time since I used C++ ..

Now that works but I ran into a new problem. The plugin dll must have an init entry point that adds all the modules that this plugin provides:

use crate::ffi::rack::plugin::Model;
use crate::ffi::rack::plugin::Plugin;
use autocxx::include_cpp;
use std::pin::Pin;

include_cpp! {
    #include "plugin/Model.hpp"

    safety!(unsafe)

    generate!("rack::plugin::Model")
    generate!("rack::plugin::Plugin")
    generate!("createModel")
}

#[no_mangle]
pub extern "C" fn init(p: *mut Plugin) {
    Pin::new(&mut (*p)).addModel(todo!());
}
error: src\lib.rs:25: `PhantomPinned` cannot be unpinned
error: src\lib.rs:25: within `root::rack::plugin::Plugin`, the trait `Unpin` is not implemented for `PhantomPinned`
note: src\lib.rs:25: consider using `Box::pin`
note: src\lib.rs:25: required by `Pin::<P>::new`
note: src\lib.rs:1: required because it appears within the type `PhantomData<PhantomPinned>`
note: target\debug\build\vcvrust-2e355000dc4171e5\out\autocxx-build-dir\rs\260199486107533567.rs:1: required because it appears within the type `root::rack::plugin::Plugin`

The referenced rs file looks like this if I format it:

#[allow(non_snake_case)]
#[allow(dead_code)]
#[allow(non_upper_case_globals)]
#[allow(non_camel_case_types)]
mod ffi {
    pub trait ToCppString {
        fn to_cpp(&self) -> cxx::UniquePtr<cxx::CxxString>;
    }
    impl ToCppString for str {
        fn to_cpp(&self) -> cxx::UniquePtr<cxx::CxxString> {
            cxxbridge::make_string(self)
        }
    }
    unsafe impl cxx::ExternType for bindgen::root::rack::plugin::Plugin {
        type Id = cxx::type_id!("rack::plugin::Plugin");
        type Kind = cxx::kind::Opaque;
    }
    unsafe impl cxx::ExternType for bindgen::root::rack::plugin::Model {
        type Id = cxx::type_id!("rack::plugin::Model");
        type Kind = cxx::kind::Opaque;
    }
    unsafe impl cxx::ExternType for bindgen::root::json_t {
        type Id = cxx::type_id!("json_t");
        type Kind = cxx::kind::Opaque;
    }
    mod bindgen {
        pub mod root {
            #[repr(C, packed)]
            pub struct json_t {
                do_not_attempt_to_allocate_nonpod_types: [*const u8; 0],
                _pinned: core::marker::PhantomData<core::marker::PhantomPinned>,
            }
            pub mod rack {
                pub mod plugin {
                    #[repr(C, packed)]
                    pub struct Plugin {
                        do_not_attempt_to_allocate_nonpod_types: [*const u8; 0],
                        _pinned: core::marker::PhantomData<core::marker::PhantomPinned>,
                    }
                    #[repr(C, packed)]
                    pub struct Model {
                        do_not_attempt_to_allocate_nonpod_types: [*const u8; 0],
                        _pinned: core::marker::PhantomData<core::marker::PhantomPinned>,
                    }
                    impl Plugin {
                        pub fn getModel(
                            self: std::pin::Pin<&mut root::rack::plugin::Plugin>,
                            slug: cxx::UniquePtr<cxx::CxxString>,
                        ) -> *mut root::rack::plugin::Model {
                            cxxbridge::getModel_autocxx_wrapper(self, slug)
                        }
                    }
                    impl Model {
                        pub unsafe fn fromJson(
                            self: std::pin::Pin<&mut root::rack::plugin::Model>,
                            rootJ: *mut root::json_t,
                        ) {
                            cxxbridge::rack_plugin_Model_fromJson_autocxx_wrapper(self, rootJ)
                        }
                    }
                    #[allow(unused_imports)]
                    use self::super::super::super::super::cxxbridge;
                    #[allow(unused_imports)]
                    use self::super::super::super::root;
                }
                #[allow(unused_imports)]
                use self::super::super::super::cxxbridge;
                #[allow(unused_imports)]
                use self::super::super::root;
            }
            #[allow(unused_imports)]
            use self::super::super::cxxbridge;
            #[allow(unused_imports)]
            use self::super::root;
        }
    }
    #[cxx::bridge]
    pub mod cxxbridge {
        impl UniquePtr<Plugin> {}
        impl CxxVector<Plugin> {}
        impl UniquePtr<Model> {}
        impl CxxVector<Model> {}
        impl UniquePtr<json_t> {}
        impl CxxVector<json_t> {}
        unsafe extern "C++" {
            fn make_string(str_: &str) -> UniquePtr<CxxString>;
            #[namespace = "rack::plugin"]
            type Plugin = super::bindgen::root::rack::plugin::Plugin;
            #[namespace = "rack::plugin"]
            type Model = super::bindgen::root::rack::plugin::Model;
            #[namespace = "rack::plugin"]
            pub unsafe fn addModel(self: Pin<&mut Plugin>, model: *mut Model);
            pub fn getModel_autocxx_wrapper(
                autocxx_gen_this: Pin<&mut Plugin>,
                slug: UniquePtr<CxxString>,
            ) -> *mut Model;
            #[namespace = "rack::plugin"]
            pub unsafe fn fromJson(self: Pin<&mut Plugin>, rootJ: *mut json_t);
            pub unsafe fn rack_plugin_Model_fromJson_autocxx_wrapper(
                autocxx_gen_this: Pin<&mut Model>,
                rootJ: *mut json_t,
            );
            type json_t = super::bindgen::root::json_t;
            include!("plugin/Model.hpp");
            include!("autocxxgen.h");
        }
    }
    pub use cxxbridge::json_t;
    pub mod rack {
        #[allow(unused_imports)]
        use super::cxxbridge;
        pub mod plugin {
            #[allow(unused_imports)]
            use super::cxxbridge;
            pub use cxxbridge::Model;
            pub use cxxbridge::Plugin;
        }
    }
}

The C++ Plugin type is defined as:

#pragma once
#include <common.hpp>
#include <jansson.h>
#include <vector>

namespace rack {
namespace plugin {

struct Model;

// Subclass this and return a pointer to a new one when init() is called
struct Plugin {
    /** A list of the models available by this plugin, add with addModel() */
    std::vector<Model*> models;
    /** The file path to the plugin's directory */
    std::string path;
    /** OS-dependent library handle */
    void* handle = NULL;

    /** Must be unique. Used for saving patches. Never change this after releasing your plugin.
    To guarantee uniqueness, it is a good idea to prefix the slug by your "company name" if available, e.g. "MyCompany-MyPlugin"
    */
    std::string slug;
    /** Your plugin's latest version, using the guidelines at https://github.com/VCVRack/Rack/issues/266. Do not include the "v" prefix.
    */
    std::string version;
    /** The license type of your plugin. Use "proprietary" if all rights are reserved. If your license is in the [SPDX license list](https://spdx.org/licenses/), use its abbreviation in the "Identifier" column.
    */
    std::string license;
    /** Human-readable display name for your plugin. You can change this on a whim, unlike slugs.
    */
    std::string name;
    /** Prefix of each module name in the Module Browser.
    If blank, `name` is used.
    */
    std::string brand;
    /** Your name, company, alias, or GitHub username.
    */
    std::string author;
    /** Your email address for support inquiries.
    */
    std::string authorEmail;
    /** Homepage of the author.
    */
    std::string authorUrl;
    /** Homepage featuring the plugin itself.
    */
    std::string pluginUrl;
    /** The manual of your plugin. HTML, PDF, or GitHub readme/wiki are fine.
    */
    std::string manualUrl;
    /** The source code homepage. E.g. GitHub repo.
    */
    std::string sourceUrl;
    /** Link to donation page for users who wish to donate. E.g. PayPal URL.
    */
    std::string donateUrl;
    /** Last modified timestamp of the plugin directory.
    */
    double modifiedTimestamp = -INFINITY;

    ~Plugin();
    void addModel(Model* model);
    Model* getModel(std::string slug);
    void fromJson(json_t* rootJ);
};

} // namespace plugin
} // namespace rack

I'm wondering why getModel is generated to be inside the Plugin impl but addModel is outside. And what's the right way to call addModel on the given *mut Plugin? :)

Btw, if I try to call it like addModel(Pin::new(&mut (*p)), todo!());, I get an error if I try to import it: I tried with use crate::ffi::rack::plugin::addModel; and use crate::ffi::cxxbridge::addModel;, with both it couldn't find that symbol. What's the right way to import it?

adetaylor commented 3 years ago

Hi,

pub extern "C" fn init(p: *mut Plugin) {
    Pin::new(&mut (*p)).addModel(todo!());
}

This feels not quite right. I must admit all this pinning stuff is a part of cxx and I'm not fully confident of how to use it in practice. But I'm pretty sure that you should be converting the existing pointer into a pinned mutable reference, instead of creating a new pinned mutable reference to the dereferenced plugin.

I would be trying something like:

pub extern "C" fn init(p: *mut Plugin) {
    unsafe { Pin::new_unchecked(p) }.addModel(todo!());
}

The unsafe is required because you, the human, are having to guarantee that the p will not change or move subsequently.

I'm wondering why getModel is generated to be inside the Plugin impl but addModel is outside.

In fact they're both generated as methods of Plugin. Where possible, we ask cxx to generate such methods (as we have in the case of addModel) but for others cxx isn't flexible enough, so autocxx needs to do it (as is the case for getModel).