Trying to support clang 15

[you74674]

I see there are some commits about vs2022, so I wonder if there will be support for clang 15? Recently I found that the latest vs2022 seems to require clang 15, which doesn't work with current CppSharp. (_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH works but it looks unsafe) I have built llvm 15.0.7 and so far the result CppSharp works with my own usecase, but I don't know if my change make sense since I don't understand the internal part of CppSharp. Here is my fork with the changes, and I have some questions:

1. I update premake to official latest version which support vs2022 action. Is there anything required from the InteropAlliance fork?
2. Latest llvm changes some fields and functions, is my change correct?
3. There seems to be some class that mirrors class in llvm, for example, clang::CoroutineBodyStmt doesn't have getResultDecl anymore, so I guess AST::CoroutineBodyStmt should also be updated. But it also appears a lot in src/CppParser/Bindings/CSharp, and those files are auto generated. How to update them?
4. These changes are not compatible with clang 14, how should we deal with this part?(if we are going to merge this or something)

[netcorefan1]

I'm in the same boat! You can find more details here. Did you also got the same error on vs2022? Did you tried to rebuild CppSharp with MSVC v142? In my case failed to compile, but could be some mistake from my side and I just suspended my attempts to study better documentation. From what I have read, vs2022 should work, as long as msvc 142 is used. I would like to try your fork. ./build.sh clone_llvm build_llvm package_llvm is the way to go, right?

[you74674]

I'm in the same boat! You can find more details here. Did you also got the same error on vs2022? Did you tried to rebuild CppSharp with MSVC v142? In my case failed to compile, but could be some mistake from my side and I just suspended my attempts to study better documentation. From what I have read, vs2022 should work, as long as msvc 142 is used. I would like to try your fork. ./build.sh clone_llvm build_llvm package_llvm is the way to go, right?

I did get the Unexpected compiler version error. I'm using v143, so I'm not sure about v142, but before I updated my visual studio 2022 from around 17.3.x to the latest version, I was using cppsharp 1.0.1 and it worked fine.

yes, I followed the original instruction about building llvm.

[netcorefan1]

The line that emit the error is CLang version check (if __clang_major__ < 15 error ). I probably get a different error string due to a different revision of the header (MSVC 14.35.32215). I cloned your repo and compilation went smoothly until the time to compile the generated solution:

 ParseStmt.cpp
  Parser.cpp
  Sources.cpp
  Stmt.cpp
  Target.cpp
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\Comments.cpp) [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\Parser.cpp) [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\AST.cpp(11,10): fatal error C1083: Cannot open include file: 'llvm/ADT/SmallString.h': No such file or directory [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\ParseStmt.cpp) [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\CppParser.cpp) [C:\CppSharpEx\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\ParseExpr.cpp) [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
..\CppSharp\src\CppParser\Parser.h(10,10): fatal error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\Link.cpp) [..\CppSharp\build\vs2019\projects\CppSharp.CppParser.vcxproj]
  Std-symbols.cpp

It seems a problem with a generator which still point to the old vs2019 folder. Let me see if I can fix this... UPDATE: may be this was because I was compiling from Developer command prompt and generated solution was still targeting MSVC 142. I opened the whole CppSharp.sln, retargeted to 143, but still fails with what seems to be the same errors: CppSharp.CppParser.vcxproj

fatal  error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\Comments.cpp)
fatal  error C1083: Cannot open include file: 'llvm/ADT/SmallString.h': No such file or directory
fatal  error C1083: Cannot open include file: 'llvm/Object/Archive.h': No such file or directory (compiling source file ..\..\..\src\CppParser\ParseExpr.cpp)
etc etc

What am I missing?

[you74674]

I think you need to run build.sh generate in vs2022 developer command prompt so the include directories are correct

[netcorefan1]

Yes, this what I have done, but still generates vs2019 projects:

**********************************************************************
** Visual Studio 2022 Developer Command Prompt v17.5.2
** Copyright (c) 2022 Microsoft Corporation
**********************************************************************
C:\CppSharpEx\CppSharp\build>"C:\Program Files\Git\bin\sh.exe" build.sh generate -configuration Release -platform x64
Archive llvm-791523-windows-vs2022-x64-RelWithDebInfo.7z already exists.
Using LLVM build: C:/CppSharpEx/CppSharp/build/llvm/llvm-project
Generating CppSharp build configuration file 'BuildConfig.cs'
Building configurations...
Running action 'vs2019'...
Generated ../CppSharp.sln...
Generated vs2019/projects/CppSharp.CppParser.vcxproj...
Generated vs2019/projects/CppSharp.CppParser.vcxproj.user...
Generated vs2019/projects/Std-symbols.vcxproj...
Generated vs2019/projects/Std-symbols.vcxproj.user...
Generated vs2019/projects/Std-symbols.vcxproj.filters...
Generated vs2019/projects/CppSharp.Parser.CLI.vcxproj...
Generated vs2019/projects/CppSharp.Parser.CLI.vcxproj.user...
Generated vs2019/projects/CLI.Native.vcxproj...
Generated vs2019/projects/CLI.Native.vcxproj.user...
Generated vs2019/projects/CLI.Native.vcxproj.filters...
Generated vs2019/projects/CLI.CLI.vcxproj...
Generated vs2019/projects/CLI.CLI.vcxproj.user...
Generated vs2019/projects/CLI.CLI.vcxproj.filters...
Generated vs2019/projects/Common.Native.vcxproj...
Generated vs2019/projects/Common.Native.vcxproj.user...
Generated vs2019/projects/Common.CLI.vcxproj...
Generated vs2019/projects/Common.CLI.vcxproj.user...
Generated vs2019/projects/Common.CLI.vcxproj.filters...
Generated vs2019/projects/CSharp.Native.vcxproj...
Generated vs2019/projects/CSharp.Native.vcxproj.user...
Generated vs2019/projects/Encodings.Native.vcxproj...
Generated vs2019/projects/Encodings.Native.vcxproj.user...
Generated vs2019/projects/NamespacesBase.Native.vcxproj...
Generated vs2019/projects/NamespacesBase.Native.vcxproj.user...
Generated vs2019/projects/NamespacesDerived.Native.vcxproj...
Generated vs2019/projects/NamespacesDerived.Native.vcxproj.user...
Generated vs2019/projects/StandardLib.Native.vcxproj...
Generated vs2019/projects/StandardLib.Native.vcxproj.user...
Generated vs2019/projects/StandardLib.CLI.vcxproj...
Generated vs2019/projects/StandardLib.CLI.vcxproj.user...
Generated vs2019/projects/StandardLib.CLI.vcxproj.filters...
Generated vs2019/projects/VTables.Native.vcxproj...
Generated vs2019/projects/VTables.Native.vcxproj.user...
Done (265ms).

I must miss something obvious. Not sure, but it could be some flaw in the VS version extraction code. My VS version is 17.5.2. Were you on this version when you compiled?

[you74674]

Why is your llvm at commit 791523? If you're using my fork then it should be 8dfdcc. And also check the vs variable in build.sh.

[tritao]

I see there are some commits about vs2022, so I wonder if there will be support for clang 15? Recently I found that the latest vs2022 seems to require clang 15, which doesn't work with current CppSharp. (_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH works but it looks unsafe) I have built llvm 15.0.7 and so far the result CppSharp works with my own usecase, but I don't know if my change make sense since I don't understand the internal part of CppSharp. Here is my fork with the changes, and I have some questions:

1. I update premake to official latest version which support vs2022 action. Is there anything required from the InteropAlliance fork?

It had some bug fixes and improvements (which are on https://github.com/tritao/premake-core/branches). But it's been a while so maybe upstream works now.

2. Latest llvm changes some fields and functions, is my change correct?

They look OK to me.

3. There seems to be some class that mirrors class in llvm, for example, clang::CoroutineBodyStmt doesn't have getResultDecl anymore, so I guess AST::CoroutineBodyStmt should also be updated. But it also appears a lot in src/CppParser/Bindings/CSharp, and those files are auto generated. How to update them?

These are auto-generated by the Bootstrap tool: https://github.com/mono/CppSharp/blob/main/src/CppParser/Bootstrap/Bootstrap.cs

4. These changes are not compatible with clang 14, how should we deal with this part?(if we are going to merge this or something)

Ideally we would keep supporting LLVM 14 by using conditional compilation. But in this case I am also fine with just upgrading to the latest LLVM.

For the upgrade, it's also necessary to have prebuilt packages for all platforms. We have two actions that do this: https://github.com/mono/CppSharp/actions/workflows/llvm.yml https://github.com/mono/CppSharp/actions/workflows/llvm-win.yml

I have issued two builds with your branch to see if we get the artifacts out of it:

https://github.com/mono/CppSharp/actions/runs/4467255519 https://github.com/mono/CppSharp/actions/runs/4467257403

[you74674]

Somehow the prebuilt binaries in premake official release use zip for windows but use tar.gz for linux and macosx...

Also I just found that my fork cannot run the bootstrap, still trying to debug it.

[tritao]

Somehow the prebuilt binaries in premake official release use zip for windows but use tar.gz for linux and macosx...

Also I just found that my fork cannot run the bootstrap, still trying to debug it.

https://github.com/mono/CppSharp/commit/a357ea2248203e0ccaa8c88f9e7155ad91b6d276

I noticed the same and fixed to see if LLVM builds can complete now.

[you74674]

Something must be wrong about the resultDecl part, but I have no idea how to fix it. I saw the CI failing NamespacesDerived.Tests and look into the detail, and found that the content of generated NamespacesDerived.xml is wrong about the \<returns> part. The correct one should have \<returns> tag, but the output here is mixed into next tag.

EDIT: According to commit dddd5d, the CommentCommandKind in AST/Comment.cs need to be updated. Updating it fix the test error. It doesn't help the bootstrap error, though.

[you74674]

Is there any special requirement when running bootstrap? Since I cannot run bootstrap in my fork, I'm trying to run bootstrap of Cppsharp 1.0.1, but both give the same error. The class body of all these output files(Stmt.h, Stmt.cpp, ...) are empty.

Generating parser bootstrap code...
Parsing libraries...
Parsing code...
Parsed 'clang/AST/Stmt.h, clang/AST/StmtCXX.h, clang/AST/Expr.h, clang/AST/ExprCXX.h'
Processing code...
Writing 'Stmt.h'.
Writing 'Stmt.cpp'.
Writing 'ParseStmt.cpp'.
Writing 'Stmt.cs'.
Unhandled exception. System.NullReferenceException: Object reference not set to an instance of an object.
   at CppSharp.ManagedVisitorCodeGenerator.GenerateVisitor() in D:\lib\CppSharp2\src\CppParser\Bootstrap\Bootstrap.cs:line 677
   at CppSharp.ManagedVisitorCodeGenerator.Process() in D:\lib\CppSharp2\src\CppParser\Bootstrap\Bootstrap.cs:line 654
   at CppSharp.Bootstrap.GenerateStmt(BindingContext ctx) in D:\lib\CppSharp2\src\CppParser\Bootstrap\Bootstrap.cs:line 204
   at CppSharp.Bootstrap.Preprocess(Driver driver, ASTContext ctx) in D:\lib\CppSharp2\src\CppParser\Bootstrap\Bootstrap.cs:line 115
   at CppSharp.ConsoleDriver.Run(ILibrary library) in D:\lib\CppSharp2\src\Generator\Driver.cs:line 449
   at CppSharp.Bootstrap.Main(String[] args) in D:\lib\CppSharp2\src\CppParser\Bootstrap\Bootstrap.cs:line 273
Segmentation fault

EDIT: I think the problem might be about GenerationKind. Now it's almost always None and so nothing is written to the output file.

EDIT2: clang/include/clang/AST needs to be in IncludeDirs or the whole module is marked as ignored. Now bootstrap can run, but the output result cannot be compile yet.

[netcorefan1]

Why is your llvm at commit 791523? If you're using my fork then it should be 8dfdcc. And also check the vs variable in build.sh.

Ohh! My stupid mistake! I forgot to checkout! The solution compiles and I have been able to generate the cs files. Unfortunately, during the process I got lots Virtual method xxx was ignored, Function xxx was ignored due to ignored return decl, Typedef xxx was ignored due to incomplete type and the list continue. The resulting csproj is uncompilable due to missing type or namespace. I'll retry again as soon as you will be able to apply the new changes (I would like to help, but I'm new to CppSharp and I don't have the right knowledge).

[you74674]

I'm trying to fix bootstrap, but there are too many details that I don't really understand. The fix I'm trying:

1. add new AbstractStmt: ValueStmt in CodeGeneratorHelpers.IsAbstractStmt. this should be correct?
2. A lot of Expr and Stmt should be Expr* and Stmt* in output files. Should it be handled in CodeGeneratorHelpers.GetDeclTypeName?
3. CastIterator is not deduced to Expr*. Maybe change ExprIterator to Expr in CodeGeneratorHelpers.GetIteratorTypeName?
4. There are new fields in output file without definition. I think part of them should be added in Types.h, and part of them should be ignored, but I don't know how to distinguish them.
5. Some fields are missing compare to past version. For example, there's no singleDecl field in DeclStmt, which looks strange, because the getter is still in original clang Stmt.h.

Any idea about how can I tackle these problem?

[tritao]

I'm trying to fix bootstrap, but there are too many details that I don't really understand. The fix I'm trying:

1. add new AbstractStmt: ValueStmt in CodeGeneratorHelpers.IsAbstractStmt. this should be correct?
2. A lot of Expr and Stmt should be Expr* and Stmt* in output files. Should it be handled in CodeGeneratorHelpers.GetDeclTypeName?
3. CastIterator is not deduced to Expr*. Maybe change ExprIterator to Expr in CodeGeneratorHelpers.GetIteratorTypeName?
4. There are new fields in output file without definition. I think part of them should be added in Types.h, and part of them should be ignored, but I don't know how to distinguish them.
5. Some fields are missing compare to past version. For example, there's no singleDecl field in DeclStmt, which looks strange, because the getter is still in original clang Stmt.h.

Any idea about how can I tackle these problem?

Sorry for the delay in getting back to you, do you still need help with this?

[you74674]

Personally my problem is solved with the workaround(_ALLOW_COMPILER_AND_STL_VERSION_MISMATCH), and so far it seems to be stable. I do still want to get it work with clang 15, though. Now a problem is that I can't get the bootstrap script work, even for older version. Maybe it's just that I'm not running it using the right way? Can you provide an example for running the bootstrap process? Like, is it possible to reproduce the output(Expr.cpp/cs, etc.) for current working version?

[tritao]

I've been trying to get Clang 15 to work, latest revision is here: https://github.com/mono/CppSharp/actions/runs/5125269224/jobs/9222274884

However as you can see, it's still failing on all platforms :sob:, all seemingly due to different issues, still trying to figure out what's going on.

About the bootstrap, I think it's normal it doesn't work atm since I haven't been maintaining it for a while, as for running it, all you needed was to run the executable but seems like it got broken now. Maybe I can look into fixing it, why do you have interest in it btw?

[you74674]

Because manual modifications need to be done on auto generated files, which I think is not the correct way to do it. But I don't know how exactly CppSharp work, maybe it's just fine as long as it can launch?

I haven't been working on it for months, but I think I got the same error in tests(for windows-2022 task). Not sure about the error in other platform, though.