Help with generating PInvoke for modsecurity

[vasicvuk]

Brief Description

My idea was to use CppSharp to generate PInvoke classes for ModSecurity c++ library.

OS: Windows / OS X / Linux (include version and/or distro)

Ubuntu 20.04

My setup looks like this:

            driver.ParserOptions.AddArguments("-fexceptions");
            driver.ParserOptions.AddArguments("-fcxx-exceptions");
            driver.ParserOptions.EnableRTTI = true;
            // driver.ParserOptions.SetupLinux("");
            var parserOptions = driver.ParserOptions;
            parserOptions.MicrosoftMode = false;
            parserOptions.LanguageVersion = CppSharp.Parser.LanguageVersion.CPP11;
            parserOptions.NoBuiltinIncludes = true;
            var headersPath = string.Empty;
            var versions = Directory.EnumerateDirectories(Path.Combine(headersPath, "/usr/include/c++"));

            if (versions.Count() == 0)
                throw new Exception("No valid GCC version found on system include paths");

            string gccVersionPath = versions.First();
            string gccVersion = gccVersionPath.Substring(gccVersionPath.LastIndexOf(Path.DirectorySeparatorChar) + 1);

            string[] systemIncludeDirs = {
                Path.Combine("/usr", "include", "c++", gccVersion),
                Path.Combine("/usr", "include", "x86_64-linux-gnu", "c++", gccVersion),
                Path.Combine("/usr", "include", "c++", gccVersion, "backward"),
                Path.Combine("/usr", "lib", "gcc", "x86_64-linux-gnu", gccVersion, "include"),
                Path.Combine("/usr", "include", "x86_64-linux-gnu"),
                Path.Combine("/usr", "include")
            };
            parserOptions.TargetTriple = "x86_64-linux-gnu-cxx11abi";
            parserOptions.AddDefines("_GLIBCXX_USE_CXX11_ABI=1");

            foreach (var dir in systemIncludeDirs)
                parserOptions.AddSystemIncludeDirs(Path.Combine(headersPath, dir));

            var options = driver.Options;

            options.GeneratorKind = GeneratorKind.CSharp;
            var module = options.AddModule("ModSecurity");
            module.IncludeDirs.Add(@"/opt/ModSecurity/headers");
            module.Headers.Add("modsecurity/intervention.h");
            module.Headers.Add("modsecurity/collection/collection.h");
            module.Headers.Add("modsecurity/collection/collections.h");
            module.Headers.Add("modsecurity/transaction.h");
            module.Headers.Add("modsecurity/debug_log.h");
            module.Headers.Add("modsecurity/modsecurity.h");
            module.Headers.Add("modsecurity/rules_properties.h");
            module.Headers.Add("modsecurity/rules.h");
            module.Headers.Add("modsecurity/rules_set.h");
            module.LibraryDirs.Add("/usr/local/modsecurity/lib/");
            module.LibraryDirs.Add("/usr/lib/");
            module.LibraryDirs.Add("/usr/lib/x86_64-linux-gnu/");
            module.Libraries.Add("libmodsecurity.so.3.0.6");
            module.Libraries.Add("libmodsecurity.a");
            module.Libraries.Add("libmodsecurity.so");
            module.Libraries.Add("libxml2.so");
            module.Libraries.Add("libcurl.so");
            module.Libraries.Add("libpcre.so.3");
            module.Libraries.Add("libyajl.so");
            module.Libraries.Add("libGeoIP.so");
            module.Libraries.Add("liblmdb.so");

            module.LibraryName = "libmodsecurity.so.3.0.6";

But I get a lot of empty Std.cs classes. Here is the generated code

Stack trace or incompilable generated code

// ----------------------------------------------------------------------------
// <auto-generated>
// This is autogenerated code by CppSharp.
// Do not edit this file or all your changes will be lost after re-generation.
// </auto-generated>
// ----------------------------------------------------------------------------
using System;
using System.Runtime.InteropServices;
using System.Security;
using __CallingConvention = global::System.Runtime.InteropServices.CallingConvention;
using __IntPtr = global::System.IntPtr;

namespace Std
{
}

namespace Std
{
    namespace CharTraits
    {
        [StructLayout(LayoutKind.Sequential, Size = 1)]
        public unsafe partial struct __Internal
        {
        }
    }

    public unsafe partial class CharTraits<_CharT> : IDisposable
    {
        public __IntPtr __Instance { get; protected set; }

        internal static readonly global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.CharTraits<_CharT>> NativeToManagedMap = new global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.CharTraits<_CharT>>();

        protected bool __ownsNativeInstance;

        internal static CharTraits<_CharT> __CreateInstance(__IntPtr native, bool skipVTables = false)
        {
            return new CharTraits<_CharT>(native.ToPointer(), skipVTables);
        }

        internal static CharTraits<_CharT> __GetOrCreateInstance(__IntPtr native, bool saveInstance = false, bool skipVTables = false)
        {
            if (native == __IntPtr.Zero)
                return null;
            if (NativeToManagedMap.TryGetValue(native, out var managed))
                return (CharTraits<_CharT>)managed;
            var result = __CreateInstance(native, skipVTables);
            if (saveInstance)
                NativeToManagedMap[native] = result;
            return result;
        }

        internal static CharTraits<_CharT> __CreateInstance(global::Std.CharTraits.__Internal native, bool skipVTables = false)
        {
            return new CharTraits<_CharT>(native, skipVTables);
        }

        private static void* __CopyValue(global::Std.CharTraits.__Internal native)
        {
            var ret = Marshal.AllocHGlobal(sizeof(global::Std.CharTraits.__Internal));
            *(global::Std.CharTraits.__Internal*) ret = native;
            return ret.ToPointer();
        }

        private CharTraits(global::Std.CharTraits.__Internal native, bool skipVTables = false)
            : this(__CopyValue(native), skipVTables)
        {
            __ownsNativeInstance = true;
            NativeToManagedMap[__Instance] = this;
        }

        protected CharTraits(void* native, bool skipVTables = false)
        {
            if (native == null)
                return;
            __Instance = new __IntPtr(native);
        }

        public void Dispose()
        {
            Dispose(disposing: true, callNativeDtor : __ownsNativeInstance );
        }

        partial void DisposePartial(bool disposing);

        internal protected virtual void Dispose(bool disposing, bool callNativeDtor )
        {
            if (__Instance == IntPtr.Zero)
                return;
            NativeToManagedMap.TryRemove(__Instance, out _);
            DisposePartial(disposing);
            if (__ownsNativeInstance)
                Marshal.FreeHGlobal(__Instance);
            __Instance = IntPtr.Zero;
        }
    }
}

namespace Std
{
    namespace Allocator
    {
        [StructLayout(LayoutKind.Sequential, Size = 1)]
        public unsafe partial struct __Internal
        {
            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNSaIcEC2Ev", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern void ctorc__N_std_S_allocator__C(__IntPtr __instance);

            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNSaIcED2Ev", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern void dtorc__N_std_S_allocator__C(__IntPtr __instance);
        }
    }

    public unsafe partial class Allocator<_Tp> : IDisposable
    {
        public __IntPtr __Instance { get; protected set; }

        internal static readonly global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.Allocator<_Tp>> NativeToManagedMap = new global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.Allocator<_Tp>>();

        protected bool __ownsNativeInstance;

        internal static Allocator<_Tp> __CreateInstance(__IntPtr native, bool skipVTables = false)
        {
            return new Allocator<_Tp>(native.ToPointer(), skipVTables);
        }

        internal static Allocator<_Tp> __GetOrCreateInstance(__IntPtr native, bool saveInstance = false, bool skipVTables = false)
        {
            if (native == __IntPtr.Zero)
                return null;
            if (NativeToManagedMap.TryGetValue(native, out var managed))
                return (Allocator<_Tp>)managed;
            var result = __CreateInstance(native, skipVTables);
            if (saveInstance)
                NativeToManagedMap[native] = result;
            return result;
        }

        internal static Allocator<_Tp> __CreateInstance(global::Std.Allocator.__Internal native, bool skipVTables = false)
        {
            return new Allocator<_Tp>(native, skipVTables);
        }

        private static void* __CopyValue(global::Std.Allocator.__Internal native)
        {
            var ret = Marshal.AllocHGlobal(sizeof(global::Std.Allocator.__Internal));
            *(global::Std.Allocator.__Internal*) ret = native;
            return ret.ToPointer();
        }

        private Allocator(global::Std.Allocator.__Internal native, bool skipVTables = false)
            : this(__CopyValue(native), skipVTables)
        {
            __ownsNativeInstance = true;
            NativeToManagedMap[__Instance] = this;
        }

        protected Allocator(void* native, bool skipVTables = false)
        {
            if (native == null)
                return;
            __Instance = new __IntPtr(native);
        }

        public Allocator()
        {
            var ___Tp = typeof(_Tp);
            if (___Tp.IsAssignableFrom(typeof(sbyte)))
            {
                __Instance = Marshal.AllocHGlobal(sizeof(global::Std.Allocator.__Internal));
                __ownsNativeInstance = true;
                NativeToManagedMap[__Instance] = this;
                global::Std.Allocator.__Internal.ctorc__N_std_S_allocator__C(__Instance);
                return;
            }
            throw new ArgumentOutOfRangeException("_Tp", string.Join(", ", new[] { typeof(_Tp).FullName }), "global::Std.Allocator<_Tp> maps a C++ template class and therefore it only supports a limited set of types and their subclasses: <sbyte>.");
        }

        public void Dispose()
        {
            Dispose(disposing: true, callNativeDtor : __ownsNativeInstance );
        }

        partial void DisposePartial(bool disposing);

        internal protected virtual void Dispose(bool disposing, bool callNativeDtor )
        {
            if (__Instance == IntPtr.Zero)
                return;
            NativeToManagedMap.TryRemove(__Instance, out _);
            DisposePartial(disposing);
            if (callNativeDtor)
            {
                var ___Tp = typeof(_Tp);
                if (___Tp.IsAssignableFrom(typeof(sbyte)))
                {
                    global::Std.Allocator.__Internal.dtorc__N_std_S_allocator__C(__Instance);
                    return;
                }
                throw new ArgumentOutOfRangeException("_Tp", string.Join(", ", new[] { typeof(_Tp).FullName }), "global::Std.Allocator<_Tp> maps a C++ template class and therefore it only supports a limited set of types and their subclasses: <sbyte>.");
            }
            if (__ownsNativeInstance)
                Marshal.FreeHGlobal(__Instance);
            __Instance = IntPtr.Zero;
        }
    }
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
    namespace BasicString
    {
        [StructLayout(LayoutKind.Explicit, Size = 32)]
        public unsafe partial struct __Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C
        {
            [FieldOffset(0)]
            internal global::Std.BasicString.AllocHider.__Internal _M_dataplus;

            [FieldOffset(8)]
            internal ulong _M_string_length;

            [FieldOffset(16)]
            internal fixed sbyte _M_local_buf[16];

            [FieldOffset(16)]
            internal ulong _M_allocated_capacity;

            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEC2Ev", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern void ctorc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C(__IntPtr __instance);

            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEED2Ev", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern void dtorc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C(__IntPtr __instance);
        }

        namespace AllocHider
        {
            [StructLayout(LayoutKind.Sequential, Size = 8)]
            public unsafe partial struct __Internal
            {
                internal __IntPtr _M_p;
            }

        }

        namespace _0
        {
            [StructLayout(LayoutKind.Explicit, Size = 16)]
            public unsafe partial struct __Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C
            {
                [FieldOffset(0)]
                internal fixed sbyte _M_local_buf[16];

                [FieldOffset(0)]
                internal ulong _M_allocated_capacity;
            }
        }

    }

    public unsafe partial class BasicString<_CharT, _Traits, _Alloc> : IDisposable
    {
        public __IntPtr __Instance { get; protected set; }

        internal static readonly global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.BasicString<_CharT, _Traits, _Alloc>> NativeToManagedMap = new global::System.Collections.Concurrent.ConcurrentDictionary<IntPtr, global::Std.BasicString<_CharT, _Traits, _Alloc>>();

        protected bool __ownsNativeInstance;

        internal static BasicString<_CharT, _Traits, _Alloc> __CreateInstance(__IntPtr native, bool skipVTables = false)
        {
            return new BasicString<_CharT, _Traits, _Alloc>(native.ToPointer(), skipVTables);
        }

        internal static BasicString<_CharT, _Traits, _Alloc> __GetOrCreateInstance(__IntPtr native, bool saveInstance = false, bool skipVTables = false)
        {
            if (native == __IntPtr.Zero)
                return null;
            if (NativeToManagedMap.TryGetValue(native, out var managed))
                return (BasicString<_CharT, _Traits, _Alloc>)managed;
            var result = __CreateInstance(native, skipVTables);
            if (saveInstance)
                NativeToManagedMap[native] = result;
            return result;
        }

        internal static BasicString<_CharT, _Traits, _Alloc> __CreateInstance(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C native, bool skipVTables = false)
        {
            return new BasicString<_CharT, _Traits, _Alloc>(native, skipVTables);
        }

        private static void* __CopyValue(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C native)
        {
            var ret = Marshal.AllocHGlobal(sizeof(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C));
            *(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C*) ret = native;
            return ret.ToPointer();
        }

        private BasicString(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C native, bool skipVTables = false)
            : this(__CopyValue(native), skipVTables)
        {
            __ownsNativeInstance = true;
            NativeToManagedMap[__Instance] = this;
        }

        protected BasicString(void* native, bool skipVTables = false)
        {
            if (native == null)
                return;
            __Instance = new __IntPtr(native);
        }

        public BasicString()
        {
            var ___CharT = typeof(_CharT);
            var ___Traits = typeof(_Traits);
            var ___Alloc = typeof(_Alloc);
            if (___CharT.IsAssignableFrom(typeof(sbyte)) && ___Traits.IsAssignableFrom(typeof(global::Std.CharTraits<sbyte>)) && ___Alloc.IsAssignableFrom(typeof(global::Std.Allocator<sbyte>)))
            {
                __Instance = Marshal.AllocHGlobal(sizeof(global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C));
                __ownsNativeInstance = true;
                NativeToManagedMap[__Instance] = this;
                global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C.ctorc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C(__Instance);
                return;
            }
            throw new ArgumentOutOfRangeException("_CharT, _Traits, _Alloc", string.Join(", ", new[] { typeof(_CharT).FullName, typeof(_Traits).FullName, typeof(_Alloc).FullName }), "global::Std.BasicString<_CharT, _Traits, _Alloc> maps a C++ template class and therefore it only supports a limited set of types and their subclasses: <sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>>.");
        }

        public void Dispose()
        {
            Dispose(disposing: true, callNativeDtor : __ownsNativeInstance );
        }

        partial void DisposePartial(bool disposing);

        internal protected virtual void Dispose(bool disposing, bool callNativeDtor )
        {
            if (__Instance == IntPtr.Zero)
                return;
            NativeToManagedMap.TryRemove(__Instance, out _);
            DisposePartial(disposing);
            if (callNativeDtor)
            {
                var ___CharT = typeof(_CharT);
                var ___Traits = typeof(_Traits);
                var ___Alloc = typeof(_Alloc);
                if (___CharT.IsAssignableFrom(typeof(sbyte)) && ___Traits.IsAssignableFrom(typeof(global::Std.CharTraits<sbyte>)) && ___Alloc.IsAssignableFrom(typeof(global::Std.Allocator<sbyte>)))
                {
                    global::Std.BasicString.__Internalc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C.dtorc__N_std_N___cxx11_S_basic_string__C___N_std_S_char_traits__C___N_std_S_allocator__C(__Instance);
                    return;
                }
                throw new ArgumentOutOfRangeException("_CharT, _Traits, _Alloc", string.Join(", ", new[] { typeof(_CharT).FullName, typeof(_Traits).FullName, typeof(_Alloc).FullName }), "global::Std.BasicString<_CharT, _Traits, _Alloc> maps a C++ template class and therefore it only supports a limited set of types and their subclasses: <sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>>.");
            }
            if (__ownsNativeInstance)
                Marshal.FreeHGlobal(__Instance);
            __Instance = IntPtr.Zero;
        }
    }

    public unsafe static partial class BasicStringExtensions
    {
        public partial struct __Internal
        {
            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE6assignEPKc", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern __IntPtr Assign(__IntPtr __instance, [MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(CppSharp.Runtime.UTF8Marshaller))] string __s);

            [SuppressUnmanagedCodeSecurity, DllImport("Std-symbols", EntryPoint = "_ZNKSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEE4dataEv", CallingConvention = __CallingConvention.Cdecl)]
            internal static extern __IntPtr Data(__IntPtr __instance);
        }

        public static global::Std.BasicString<sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>> Assign(this global::Std.BasicString<sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>> @this, string __s)
        {
            var __arg0 = @this is null ? __IntPtr.Zero : @this.__Instance;
            var __ret = __Internal.Assign(__arg0, __s);
            var __result0 = global::Std.BasicString<sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>>.__GetOrCreateInstance(__ret, false);
            return __result0;
        }

        public static string Data(this global::Std.BasicString<sbyte, global::Std.CharTraits<sbyte>, global::Std.Allocator<sbyte>> @this)
        {
            var __arg0 = @this is null ? __IntPtr.Zero : @this.__Instance;
            var __ret = __Internal.Data(__arg0);
            return CppSharp.Runtime.MarshalUtil.GetString(global::System.Text.Encoding.UTF8, __ret);
        }
    }
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace GnuCxx
{
}

namespace Std
{
}

namespace Std
{
    namespace Detail
    {
    }
}

namespace Std
{
}

namespace Std
{
    namespace Detail
    {
    }
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

namespace Std
{
}

Is there something I am doing wrong?

Thanks

[ddobrev]

No, it's a bug we have. We haven't got to it mostly because It's harmless - unless you lack any actually important classes. Do you?

[vasicvuk]

@ddobrev actually I don't see any class generated related to modsecurity directly. So I guessed that this empty classes are for Modsecurity but for some reason empty.

[ddobrev]

@vasicvuk they wouldn't be in Std. Your problem is actually different, I can see now. Our headers in the settings only take file names. All of your paths must go to include dirs instead: that is, as if you were writing C++.

[vasicvuk]

@ddobrev Thanks, this helped in terms that the new .cs file is generated, but unfortunately, I still have a few issues.

First in Std.cs now I have UnorderedMultimap namespace which uses Std.Hashtable which is not generated.

Second in the LibModSecurity.cs I have 235 errors, duplicate methods, nonexisting namespaces and a lot of https://docs.microsoft.com/en-us/dotnet/csharp/misc/cs0208?f1url=%3FappId%3Droslyn%26k%3Dk(CS0208)

I have also tried to generate C wrapper instead but then I am not getting any DllImport statement.

[ddobrev]

@vasicvuk you can't generate a C wrapper for C++. This setting is for the source language. Please report bugs about the rest. We'll take a look whenever we can. If you need faster resolution, please contact our support or try fixing them yourself with our help.

[vasicvuk]

I switched target language. Thanks I will contact support