search

mono / CppSharp

Tools and libraries to glue C/C++ APIs to high-level languages
MIT License
3.12k stars 512 forks source link

C# code generation fails with System.NullReferenceException #1721

Open imaras opened 1 year ago

imaras commented 1 year ago
Brief Description

C# code generation for header file my_header.h

#ifndef MY_HEADER_H
#define MY_HEADER_H

#include <cstdint>
#include <variant>

class my_variant_wrapper
{
public:;
      my_variant_wrapper(float result) :
          m_Result(result)
      {
      }

      my_variant_wrapper(uint8_t errorCode) :
          m_Result(errorCode)
      {
      }

private:
    std::variant<float, uint8_t> m_Result;
};

#endif

fails with System.NullReferenceException.

Console output is:

Parsing libraries...
Parsed 'my_library.dll'
Parsing code...
Parsed 'my_header.h'
Processing code...
Pass 'CppSharp.Passes.ResolveIncompleteDeclsPass'
Pass 'CppSharp.Passes.IgnoreSystemDeclarationsPass'
Pass 'CppSharp.Passes.MatchParamNamesWithInstantiatedFromPass'
Pass 'CppSharp.Passes.EqualiseAccessOfOverrideAndBasePass'
Pass 'CppSharp.Passes.FlattenAnonymousTypesToFields'
Pass 'CppSharp.Passes.CheckIgnoredDeclsPass'
    Decl 'm_Result' was ignored due to invalid access
    Field 'my_variant_wrapper::m_Result' was ignored due to ignored type '::std::variant<>'
    Field '::_Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Field '_Variant_storage_::' was ignored due to ignored type '::std::_Variant_storage_<1>::'
    Field '::_Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Field '_Variant_storage_::' was ignored due to ignored type '::std::_Variant_storage_<1>::'
Pass 'CppSharp.Passes.MarkUsedClassInternalsPass'
Pass 'CppSharp.Passes.TrimSpecializationsPass'
Pass 'CppSharp.Passes.CheckAmbiguousFunctions'
    Found ambiguous overload: my_variant_wrapper::operator=
Pass 'CppSharp.Passes.GenerateSymbolsPass'
Pass 'CppSharp.Passes.CheckIgnoredDeclsPass'
    Field 'my_variant_wrapper::m_Result' was ignored due to internal type '::std::variant<>'
    Function 'operator=' was ignored due to ignored param
    Field '::_Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Field '_Variant_storage_::' was ignored due to internal type '::std::_Variant_storage_<1>::'
    Field '::_Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Field '_Variant_storage_::' was ignored due to ignored type '::std::_Variant_storage_<1>::'
Pass 'CppSharp.Passes.MoveFunctionToClassPass'
Pass 'CppSharp.Passes.ValidateOperatorsPass'
    Invalid operator overload my_variant_wrapper::Equal
Pass 'CppSharp.Passes.FindSymbolsPass'
Pass 'CppSharp.Passes.CheckMacroPass'
Pass 'CppSharp.Passes.CheckStaticClass'
Pass 'CppSharp.Passes.CheckAmbiguousFunctions'
Pass 'CppSharp.Passes.ConstructorToConversionOperatorPass'
Pass 'CppSharp.Passes.MarshalPrimitivePointersAsRefTypePass'
Pass 'CppSharp.Passes.CheckOperatorsOverloadsPass'
Pass 'CppSharp.Passes.CheckVirtualOverrideReturnCovariance'
Pass 'CppSharp.Passes.CleanCommentsPass'
Pass 'CppSharp.Passes.CheckAbiParameters'
Pass 'CppSharp.Passes.CleanInvalidDeclNamesPass'
Pass 'CppSharp.FastDelegateToDelegatesPass'
Pass 'CppSharp.Passes.FieldToPropertyPass'
    Property created from field: std::_Variant_storage_<bool, >::_0::___Head
    Property created from field: std::_Variant_storage_<bool, >::_0::___Tail
Pass 'CppSharp.Passes.CheckIgnoredDeclsPass'
    Field 'my_variant_wrapper::m_Result' was ignored due to internal type '::std::variant<>'
    Function 'operator=' was ignored due to ignored param
    Function 'operator=' was ignored due to ignored param
    Field '_0::___Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Property '_Head' was ignored due to ignored type
    Property '_Tail' was ignored due to ignored type
    Field '_Variant_storage_::_0' was ignored due to internal type '::std::_Variant_storage_<1>::'
    Field '_0::_Tail' was ignored due to ignored type '::std::_Variant_storage_<1>'
    Field '_Variant_storage_::_0' was ignored due to ignored type '::std::_Variant_storage_<1>::'
Pass 'CppSharp.Passes.CheckFlagEnumsPass'
Pass 'CppSharp.Passes.MakeProtectedNestedTypesPublicPass'
Pass 'CppSharp.Passes.GenerateAbstractImplementationsPass'
Pass 'CppSharp.Passes.MultipleInheritancePass'
Pass 'CppSharp.Passes.DelegatesPass'
Pass 'CppSharp.Passes.GetterSetterToPropertyPass'
Pass 'CppSharp.Passes.StripUnusedSystemTypesPass'
Pass 'CppSharp.Passes.SpecializationMethodsWithDependentPointersPass'
Pass 'CppSharp.Passes.ParamTypeToInterfacePass'
Pass 'CppSharp.Passes.CheckDuplicatedNamesPass'
Pass 'CppSharp.Passes.CaseRenamePass'
Pass 'CppSharp.Passes.CheckKeywordNamesPass'
Pass 'CppSharp.Passes.HandleVariableInitializerPass'
Pass 'CppSharp.MarkEventsWithUniqueIdPass'
Generating code...
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at CppSharp.AST.ClassExtensions.HasDependentValueFieldInLayout(Class class, IEnumerable`1 specializations)
   at CppSharp.Generators.Helpers.GetSuffixForInternal(Class class)
   at CppSharp.Generators.CSharp.CSharpTypePrinter.VisitClassDecl(Class class)
   at CppSharp.AST.Class.Visit[T](IDeclVisitor`1 visitor)
   at CppSharp.Generators.TypePrinter.VisitTagType(TagType tag, TypeQualifiers quals)
   at CppSharp.Generators.CSharp.CSharpTypePrinter.VisitTagType(TagType tag, TypeQualifiers quals)
   at CppSharp.AST.TagType.Visit[T](ITypeVisitor`1 visitor, TypeQualifiers quals)
   at CppSharp.Generators.TypePrinter.VisitQualifiedType(QualifiedType type)
   at CppSharp.AST.QualifiedType.Visit[T](ITypeVisitor`1 visitor)
   at CppSharp.Generators.CSharp.CSharpTypePrinter.VisitFieldDecl(Field field)
   at CppSharp.Generators.CSharp.CSharpSources.GenerateClassInternalsFields(Class class, Boolean sequentialLayout)
   at CppSharp.Generators.CSharp.CSharpSources.GenerateClassInternals(Class class)
   at CppSharp.Generators.CSharp.CSharpSources.GenerateClassTemplateSpecializationsInternals(Class template, IList`1 specializations)
   at CppSharp.Generators.CSharp.CSharpSources.GenerateClassTemplateSpecializationInternal(Class classTemplate)
   at CppSharp.Generators.CSharp.CSharpSources.VisitClassDecl(Class class)
   at CppSharp.AST.Class.Visit[T](IDeclVisitor`1 visitor)
   at CppSharp.Generators.CSharp.CSharpSources.VisitDeclContext(DeclarationContext context)
   at CppSharp.Generators.CodeGenerator.VisitNamespace(Namespace namespace)
   at CppSharp.Generators.CSharp.CSharpSources.VisitNamespace(Namespace namespace)
   at CppSharp.AST.Namespace.Visit[T](IDeclVisitor`1 visitor)
   at CppSharp.Generators.CSharp.CSharpSources.VisitDeclContext(DeclarationContext context)
   at CppSharp.Generators.CodeGenerator.VisitNamespace(Namespace namespace)
   at CppSharp.Generators.CSharp.CSharpSources.VisitNamespace(Namespace namespace)
   at CppSharp.Generators.CodeGenerator.VisitTranslationUnit(TranslationUnit unit)
   at CppSharp.AST.TranslationUnit.Visit[T](IDeclVisitor`1 visitor)
   at CppSharp.Generators.CSharp.CSharpSources.Process()
   at CppSharp.Generators.Generator.GenerateModule(Module module)
   at CppSharp.Generators.Generator.Generate()
   at CppSharp.Driver.GenerateCode()
   at CppSharp.ConsoleDriver.Run(ILibrary library)
   at Program.<Main>$(String[] args)

OS: Windows

imaras commented 1 year ago

If I modify my header file to define CS_IGNORE and I apply it on a class level generation still fails.

#ifndef MY_HEADER_H
#define MY_HEADER_H

#define CS_IGNORE

#include <cstdint>
#include <variant>

class CS_IGNORE my_variant_wrapper
{
public:;
      my_variant_wrapper(float result) :
          m_Result(result)
      {
      }

      my_variant_wrapper(uint8_t errorCode) :
          m_Result(errorCode)
      {
      }

private:
    std::variant<float, uint8_t> m_Result;
};

#endif
zhiyangyou commented 8 months ago

I also encountered the same error, also about std::variant

dsudic commented 7 months ago

Is there any feedback for this?

angryzor commented 2 months ago

Running into this one now, so yeah... Guess I'll fix this one too haha.

Minimal repro:

template<typename X>
class SpecializedTemplateReferencingChildStruct;

template<typename R>
class SpecializedTemplateReferencingChildStruct<R*> {
    struct Bar {
        R xyzzy;
    };

    Bar baz;
};

class C {
    SpecializedTemplateReferencingChildStruct<int*> a;
};

Root cause of the error is here.

The code tries to look for Bar in the template, but because Bar is dependent on R the declaration is instead in the specialization itself.

angryzor commented 2 months ago

This code is in general also not very resilient: it only checks the direct parent for specializations.

If I modify the template a bit to look like this instead:

template<typename X>
class SpecializedTemplateReferencingDeepChildStruct;

template<typename R>
class SpecializedTemplateReferencingDeepChildStruct<R*> {
    struct Goo {
        struct Bar {
            bool xyzzy;
        };
    };

    typename Goo::Bar baz;
};

class D {
    SpecializedTemplateReferencingDeepChildStruct<int*> a;
};

Then the GetSuffixForInternal function will immediately quit through the specialization == null shortcut, because the specialization is actually 2 levels up. It should walk the whole list of parents instead.

angryzor commented 2 months ago

Ok, this issue seems quite a bit more complex than that other one, so I'm going to post a small update here:

Ideally, I wanted to solve this issue by calling getTemplateInstantiationPattern in Clang during the initial parse and storing the result on the Class node, so that the actual correct original template as known by Clang can be referenced directly. However, I had the native code set up and then realized I have no idea how to regenerate the parser bindings. Tried running Cpp.Parser.Bootstrap.exe in the bin directory but that crashes when it can't find llvm/Support/TrailingObjects.h.

So at the moment I'm using a hack that does "solve" the issue:

        public static string GetSuffixForInternal(Class @class)
        {
            var specialization = @class.GetParentSpecialization();

            if (specialization == null)
                return string.Empty;

            Class template = @class == specialization
                ? specialization.TemplatedDecl.TemplatedClass
                : specialization.FindNestedClass(@class.Name) ?? specialization.TemplatedDecl.TemplatedClass.FindNestedClass(@class.Name);

            if (template.HasDependentValueFieldInLayout())
            {
                if (specialization.Arguments.All(
                    a => a.Type.Type?.IsAddress() == true))
                    return "_Ptr";
                return GetSuffixFor(specialization);
            }
            return string.Empty;
        }

but is obviously really fragile as well. If the search back down into the AST tree finds another class with the same name first it will incorrectly take that one.

However, even though it doesn't crash anymore now, the code it generates is still faulty:

namespace test
{
    namespace SpecializedTemplateReferencingChildStruct
    {
        [StructLayout(LayoutKind.Sequential, Size = 4)]
        public unsafe partial struct __Internal
        {
            internal global::test.SpecializedTemplateReferencingChildStruct.Bar.__Internal baz;
        }
    }
    // implementation of C...
}

It doesn't generate anything for Bar at all, so it seems like CppSharp in general has trouble with this kind of construct?

tritao commented 2 months ago

To re-generate the parser bindings you should use https://github.com/mono/CppSharp/blob/main/src/CppParser/ParserGen/ParserGen.cs.

https://github.com/mono/CppSharp/releases/download/CppSharp/headers.zip is also necessary to generate bindings for all platforms.

It doesn't generate anything for Bar at all, so it seems like CppSharp in general has trouble with this kind of construct?

Yep, template support in general still has some issues.

angryzor commented 2 weeks ago

To re-generate the parser bindings you should use https://github.com/mono/CppSharp/blob/main/src/CppParser/ParserGen/ParserGen.cs.

https://github.com/mono/CppSharp/releases/download/CppSharp/headers.zip is also necessary to generate bindings for all platforms.

It doesn't generate anything for Bar at all, so it seems like CppSharp in general has trouble with this kind of construct?

Yep, template support in general still has some issues.

Having another look at this. @tritao Could you please explain to me the rationale behind collapsing all internal structs for specializations that only have pointers as template parameters? It seems logical since IntPtr is used for all pointers, but it also seems to only handle one very specific case and it's hard to calculate the postfixes for complex templates.

tritao commented 2 weeks ago

To re-generate the parser bindings you should use https://github.com/mono/CppSharp/blob/main/src/CppParser/ParserGen/ParserGen.cs. https://github.com/mono/CppSharp/releases/download/CppSharp/headers.zip is also necessary to generate bindings for all platforms.

It doesn't generate anything for Bar at all, so it seems like CppSharp in general has trouble with this kind of construct?

Yep, template support in general still has some issues.

Having another look at this. @tritao Could you please explain to me the rationale behind collapsing all internal structs for specializations that only have pointers as template parameters? It seems logical since IntPtr is used for all pointers, but it also seems to only handle one very specific case and it's hard to calculate the postfixes for complex templates.

Pretty much all of template support was done by @ddobrev, who unfortunately is no longer with us, so I really cannot tell what the rationale was for that decision. Feel free to propose changes to it to fix any issues and we can get them merged.