Involve RUST as an alternative for low level signal processing - fix starting problems

[jvxgit]

With Rust, I intend to realize all signal processing on the signal level as an alternative where currently, C is in use. However, the coupling is kind of complicated and demands the view of a Rust expert.

The first project is in the sub project ayfstarter. There, a C library exists that shall be replaced and be suitable as a reference for whatever is required for future projects.

The following problems are there with Rust at the moment:

ISSUE A

• In the project, we define a Rust core signal processing lib in folder "AudYoFlo/sources/sub-projects/ayfstarter/sources/Libraries/ayfstarterlib-rust". The project here is compiled as a lib and relies on some project specific struct, e.g. ayf_starter.
• There is another version of the lib which is extended by debug features which will be used in the Matlab implementation phase lateron. This project is located at "AudYoFlo/sources/sub-projects/ayfstarter/sources/Libraries/ayfstarterlib_debug-rust". It uses the same structs as the lib 1).

PROBLEM DESCRIPTION

The common structs for the lib 2) could be taken from the definition in 1). However, this yields an error message and fails. It seems to be related to the fact that the crate is of type "lib" which seems to not allow any "use" statement for parts of the definitions in the lib.

CURRENT WORKAROUND

Currently, I have created a 3rd project located at "AudYoFlo/sources/sub-projects/ayfstarter/sources/Libraries/ayfstarterlib-rust-common". In that project, the structs for both projects are defined. This project is not of type "lib" and therefore can be referenced using "use".

ISSUE B

• I am using cbindgen to generate the connecting header files to integrate Rust into AudYoFlo. I am using this code generation to not rely on an unsafe binary interfacing using existing header files.

PROBLEM DESCRIPTION

When using the cbindgen code generation in the project ayfstarterlib-rust-common, nothing is exported. That is since the structs are not used there locally but will be used in the other projects. We might use the option in cbindgen to export headers for all dependencies. However, this is not very clever since lots of other referenced lib files are generated as well and neglect all local compile options.

CURRENT WORKAROUND

I am declaring a dummy function

#[no_mangle]
pub extern "C" fn ayf_generate(_hdl: *mut ayf_starter) -> JvxDspBaseErrorType 
{
    return JvxDspBaseErrorType::JVX_NO_ERROR;
}

This lets the code generator output the code for the structs in C. And then, I add an additional header to translate the unknown typedefs, denoted as "jvx_rust_audyoflo_api.h".

ISSUE C

• I have implemented the debug functions in ayfstarterlib_debug-rust. Part of it is a call to global extern "C" functions jvx_profiler_allocate_single_entry and jvx_profiler_deallocate_single_entry which are declared in C as

void jvx_profiler_allocate_single_entry(struct jvx_profiler_data_entry* entry, jvxSize szFld, jvxCBool cplxFld)
{
...
}

and

void jvx_profiler_deallocate_single_entry(struct jvx_profiler_data_entry* entry)
{
...
}

and declared in rust as

extern "C" {
    pub fn jvx_profiler_allocate_single_entry(
        entry: *mut jvx_profiler_data_entry, 
        szFld: JvxSize, 
        cplxFld: JvxCBool);

    pub fn jvx_profiler_deallocate_single_entry(
        entry: *mut jvx_profiler_data_entry);
}

PROBLEM DESCRIPTION

When using cbindgen to generate the code, the "struct" keyword is missing. That leads to compile errors.

CURRENT WORKAROUND C

I have "hidden" the Rust declaration in the jvx_dsp lib - which is unsatisfying. The issue seems to be reported but not really fixed: https://github.com/mozilla/cbindgen/issues/621 The described workaround is added to the cbindgen.toml file but does not solve the issue.

ISSUE D

• In the call of the debug functions, there is a c callback being passed as a pointer,

jvxDspBaseErrorType ayf_starter_data_debug_prepare(
    struct ayf_starter_data_debug* hdlDbg, struct ayf_starter* hdlStarter, jvxCBitField spec, 
    jvx_register_entry_profiling_data_c cb, jvxHandle* inst)
{
...
}

The callback is jvx_register_entry_profiling_data_c. This callback can even be a nullptr. Therefore, I need to check the value which I found to be realized as an Option in Rust,

#[no_mangle]
pub extern "C" fn ayf_starter_data_debug_prepare(
     hdl_dbg: *mut ayf_starter_data_debug, 
     hdl_starter: *mut ayf_starter, spec_data: JvxCBitField, 
     cb_register: Option<jvx_register_entry_profiling_data_c>, inst: *mut std::ffi::c_void) -> JvxDspBaseErrorType 

This will be translated in cbindgen to

struct Option_jvx_register_entry_profiling_data_c;

struct Option_jvx_unregister_entry_profiling_data_c;

JvxDspBaseErrorType ayf_starter_data_debug_prepare(ayf_starter_data_debug *hdl_dbg,
                                                   ayf_starter *hdl_starter,
                                                   JvxCBitField spec_data,
                                                   struct Option_jvx_register_entry_profiling_data_c cb_register,
                                                   void *inst);

JvxDspBaseErrorType ayf_starter_data_debug_postprocess(ayf_starter_data_debug *hdl_dbg,
                                                       struct Option_jvx_unregister_entry_profiling_data_c cb_unregister,
                                                       void *inst);

How can I map the struct Option_jvx_unregister_entry_profiling_data_c to a function pointer?

CURRENT WORKAROUND

None yet, this is a show stopper!

[jvxgit]

For ISSUE D: Good idea, let us run the c-code generator to find out. The code generator is denoted as c2rust. Installing this tool yields

REALLY?

[jvxgit]

Ok, I found a workaround for ISSUE D:

We can follow the hint here:

https://github.com/mozilla/cbindgen/issues/326

Hence, I declare

pub type jvx_register_entry_profiling_data_c = extern fn (dat: *mut jvx_profiler_data_entry, name: *const std::os::raw::c_char, inst: *mut std::ffi::c_void);
pub type RegisterCallbackFunction = Option<jvx_register_entry_profiling_data_c>;

and then, declare the function to use the callback as

#[no_mangle]
pub extern "C" fn ayf_starter_data_debug_prepare(hdl_dbg: *mut ayf_starter_data_debug, hdl_starter: *mut ayf_starter, spec_data: JvxCBitField, 
                                            cb_register: RegisterCallbackFunction, inst: *mut std::ffi::c_void) -> JvxDspBaseErrorType 
{
...
}

And then, we can override the typedef for the callback in C with

typedef jvx_register_entry_profiling_data_c  RegisterCallbackFunction;

The latter to be done in file AudYoFlo/sources/jvxLibraries/jvx-system-rust/include/jvx_rust_audyoflo_api.h.

I can build the project now but I have not tested the code yet.

Extra hint:

The generation of elements in cbindgen can be suppressed as described here,

https://github.com/mozilla/cbindgen/blob/master/docs.md

by using the syntax:

[flo-at]

For issue C I changed the rs file like this:

#[repr(C)]
pub struct ayf_test_test {         
    pub test: i16,              
}   

#[no_mangle]                                                                                                                                   
pub extern "C" fn ayf_starter_data_debug_prepare(hdl_dbg: *mut ayf_test_test, hdl_starter: *mut ayf_starter, spec_data: JvxCBitField,         
                                            cb_register: RegisterCallbackFunction, inst: *mut std::ffi::c_void) -> JvxDspBaseErrorType
{

just to see what happens. In that case cbindgen generates the right function declaration for the test struct:

struct ayf_test_test {
  int16_t test;
};

JvxDspBaseErrorType ayf_starter_data_debug_prepare(struct ayf_test_test *hdl_dbg,
// [...]

It not the same problem as in the cbindgen ticket bzw similar. Still investigating..

Edit1: Works with

[parse]                                
# Whether to parse dependent crates and include their types in the output
# default: false                                                        
parse_deps = true

in the cbindgen config but that generates bindings for all parsed types which is also a problem.