Add IntelliSense for C++20 modules importing

sean-mcmanus commented 4 years ago

See the previous Windows/cl.exe-only issue at https://github.com/microsoft/vscode-cpptools/issues/6290 (UPDATE: the issue was deleted, see https://github.com/microsoft/vscode-cpptools/issues/8256#issuecomment-941809096 ).

NOTE: C++20 modules importing works for cl.exe if you set /ifcSearchDir (and possibly other modules args if necessary); however the IntelliSense support is still in active development so there are lots of known issues/bugs still.

This is dependent on IntelliSense parser changes from VS (i.e. open file Linux solutions in VS don't have module importing support for IntelliSense either).

EDIT: example here: https://github.com/microsoft/vscode-cpptools/issues/6302#issuecomment-1930791901

redmms commented 10 months ago

C++ modules: exist 4 years IntelliSense:

ddelconte commented 10 months ago

C++ modules: exist 4 years IntelliSense:

Also C++ modules: supported by all major compilers, CMake, and build systems IntelliSense:

bobbrow commented 10 months ago

C++ modules: exist 4 years IntelliSense:

Also C++ modules: supported by all major compilers, CMake, and build systems IntelliSense:

We continue to work with EDG (our front end compiler that powers IntelliSense) to ensure that this is a priority and we'll get it to the C++ extension as soon as we can. The initial implementation was based on msvc's IFC format as the other compilers were not as quick to add modules support. Official CMake support only arrived last year.

Because this extension supports more compilers than Visual Studio does, we have asked EDG for a more general-purpose implementation. IntelliSense can be made to work with modules for projects built with the msvc compiler - set the switches and keep your build up to date - but we have not officially supported modules yet because we need support for all compilers.

cjwijtmans commented 10 months ago

C++ modules: exist 4 years IntelliSense:

Also C++ modules: supported by all major compilers, CMake, and build systems IntelliSense:

We continue to work with EDG (our front end compiler that powers IntelliSense) to ensure that this is a priority and we'll get it to the C++ extension as soon as we can. The initial implementation was based on msvc's IFC format as the other compilers were not as quick to add modules support. Official CMake support only arrived last year.

Because this extension supports more compilers than Visual Studio does, we have asked EDG for a more general-purpose implementation. IntelliSense can be made to work with modules for projects built with the msvc compiler - set the switches and keep your build up to date - but we have not officially supported modules yet because we need support for all compilers.

Is there an example project anywhere, where it works in vs code, cmake and msvc?

bobbrow commented 9 months ago

Is there an example project anywhere, where it works in vs code, cmake and msvc?

I'm sorry, I just tried it and it doesn't appear to be working reliably. I found a bug in the processing of "/ifcSearchDir" and "/reference" which must have regressed at some point. After fixing that, I'm still seeing issues in the language server which I haven't tracked down.

bobbrow commented 9 months ago

After stepping through the language server, I see how it's finding and loading the compiled modules. I'm blocked by one last thing which is that the version of cl.exe I'm using outputs an IFC with a version greater than our language server knows about. The update for this version was recently merged into VS and won't make it to the C++ extension until 1.20, so IntelliSense won't work with the latest version of the MSVC compiler.

That aside, here is what I did (which will depend on a small fix which will be added to 1.19).

CMakeLists.txt

cmake_minimum_required(VERSION 3.28)

project(modules)

set(CMAKE_CXX_STANDARD 20)

add_executable(test)
target_sources(test PUBLIC main.cpp)
target_sources(test PUBLIC
    FILE_SET CXX_MODULES FILES
        foo.ixx)

# Copy ifc files for IntelliSense
install(
    DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/test.dir/
    DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/ifc
    FILES_MATCHING PATTERN "*.ifc"
)

main.cpp

import foo;

int main()
{
    Foo();
}

foo.ixx

module;
#include <iostream>

export module foo;

export void Foo()
{
    std::cout << "hello modules!" << std::endl;
}

c_cpp_properties.json

{
    "configurations": [
        {
            "name": "Win32",
            "includePath": [
                "${workspaceFolder}/**"
            ],
            "cStandard": "c17",
            "cppStandard": "c++20",
            "intelliSenseMode": "windows-msvc-x64",
            "compilerPath": "cl.exe",
            "compilerArgs": [
                "/ifcSearchDir", "${workspaceFolder}/build/ifc"
            ]
        }
    ],
    "version": 4
}

settings.json

{
    "cmake.generator": "Ninja"
}

It is important to note that you should use Ninja to build your CMake project because it compiles the modules into foo.ifc instead of foo.ixx.ifc which the Visual Studio 2022 generator does. The language server does not know how to combine /ifcSearchDir with /reference so you would have to map all of your modules as full paths to make it work if you use the latter. If you use Ninja you only have to set the search dir and it will find the compiled modules.

When my fix goes in (1.19.3), IntelliSense should start working for .ixx and .cppm files, but you will get an error in main.cpp until we get the IFC version update in 1.20.

cjwijtmans commented 9 months ago

Nobody will name their cross platform code in .ixx Although i believe cmake will tag the files as modules even when not ixx

ddelconte commented 9 months ago

Nice work, Bob. Looking forward to the fix.

nobody will name their cross platform code in .ixx seems like C++ modules will never take off at this rate.

MSVC allows you to use any extension you want with module files, provided it's properly designated as a module in CMakeLists. This adds the "/interface" argument to each of these files, bypassing the .ixx forced naming convention. But I agree that it should not be forced by default and left up to the developer, similar to GCC and Clang. But as for whether or not a module file not ending in .ixx will work with IntelliSense is another matter that the folks here will have to solve.

bobbrow commented 9 months ago

I failed to mention that .cppm will also be recognized. You don't have to name your files .ixx according to the msvc preference.

cjwijtmans commented 9 months ago

then the only complaint i would have is that

"compilerArgs": [
                "/ifcSearchDir", "${workspaceFolder}/build/CMakeFiles/test.dir"
            ]

would not work for multiple cmake projects, in my case. I have a dozen dependencies and 3-5 projects in my workspace. I can try it tomorow since now it is bed time here. One solution is to specify this in cmake for each project, but that still means non cross platform jargon in cmake.

bobbrow commented 9 months ago

You should be able to specify multiple search dirs (just like includePath has multiple search paths). So this would be supported:

"compilerArgs": [
                "/ifcSearchDir", "${workspaceFolder}/build/CMakeFiles/target1.dir",
                "/ifcSearchDir", "${workspaceFolder}/build/CMakeFiles/target2.dir"
            ]

Challanger524 commented 8 months ago

https://github.com/microsoft/vscode-cpptools/issues/6302#issuecomment-1930791901

When my fix goes in (1.19.3), IntelliSense should start working for .ixx and .cppm files, but you will get an error in main.cpp until we get the IFC version update in 1.20.

IFC file ".../<module>.ifc" has unsupported version 0.43 C/C++(3373)

C/C++ v1.19.8, looking forward 1.20, cheer you all up 🥇

loopedice commented 8 months ago

C++ modules: exist 4 years IntelliSense:

I moved to CLion Nova. It just works over there 🤷 Maybe when Microsoft invests more resources into this issue, I will see again

bobbrow commented 8 months ago

1.20.0 was just published. The IFC unsupported version 0.43 error should be gone now. Though now I'm seeing that the server is holding a reference to the built .ifc files. 😞 Closing those files in the editor will release the file lock so that you can build again, but since it's a bit disruptive, I added an INSTALL step to my CMakeLists.txt to copy the .ifc's somewhere else so that rebuilding doesn't get blocked on IntelliSense. Something like this:

install(
    DIRECTORY ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles/test.dir/
    DESTINATION ${CMAKE_CURRENT_BINARY_DIR}/ifc
    FILES_MATCHING PATTERN "*.ifc"
)

I'll update my example above. When you need to update your ifc's for IntelliSense, you can either close your source files and run the "CMake: Install" command once all of the servers have been shut down (there's about a 5 second delay), or run "Reload Window" and then "CMake: Install".

Challanger524 commented 8 months ago

Switched to pre-release v1.20 version and everything works! Little lifehack with CMake - running install after build can be automated with:

add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD COMMAND
  ${CMAKE_COMMAND} --install ${CMAKE_BINARY_DIR} --config $<CONFIG>)

Sad that it is still only a workaround for msvc compiler since clang does not generates any .ifcs for Intellisense :(

ddelconte commented 8 months ago

Shouldn't the IntelliSense be based on the source files as opposed to the compiled .ifc's? It's a bit impractical to have to first compile the project in order for it to work. Is this a feature planned in the future?

bobbrow commented 8 months ago

Shouldn't the IntelliSense be based on the source files as opposed to the compiled .ifc's? It's a bit impractical to have to first compile the project in order for it to work. Is this a feature planned in the future?

Yes, and that's why we don't have any sort of modules support for gcc/clang yet. We are waiting anxiously for EDG (company that provides our compiler front-end) to support modules more generally.

gwankyun commented 8 months ago

It is important to note that you should use Ninja to build your CMake project because it compiles the modules into foo.ifc instead of foo.ixx.ifc which the Visual Studio 2022 generator does. The language server does not know how to combine /ifcSearchDir with /reference so you would have to map all of your modules as full paths to make it work if you use the latter. If you use Ninja you only have to set the search dir and it will find the compiled modules.

The other way is updated vcxproj file with script.

$vcxproj = Get-ChildItem "./build/*.vcxproj"

foreach ($i in vcxproj) {
    (Get-Conntent $i) -replace ".ixx.ifc", ".ifc" | Set-Content $i
}

gwankyun commented 8 months ago

Switched to pre-release v1.20 version and everything works! Little lifehack with CMake - running install after build can be automated with:
add_custom_command(TARGET ${PROJECT_NAME} POST_BUILD COMMAND
  ${CMAKE_COMMAND} --install ${CMAKE_BINARY_DIR} --config $<CONFIG>)
Sad that it is still only a workaround for msvc compiler since clang does not generates any .ifcs for Intellisense :(

I found a way that CMake & VSCode can use with Visual Studio 2022 generator.

# CMakeLists.txt
install(SCRIPT install.cmake)

add_custom_command(
  TARGET ${PROJECT_NAME} 
  POST_BUILD COMMAND
    ${CMAKE_COMMAND} --install ${CMAKE_BINARY_DIR} --config $<CONFIG>
    )

# install.cmake
include(CMakePrintHelpers) # cmake_print_variables
# cmake_print_variables(CMAKE_CURRENT_BINARY_DIR)

file(GLOB_RECURSE ifc_src ${CMAKE_CURRENT_BINARY_DIR}/*.ixx.ifc)

set(ifc_dest ${CMAKE_CURRENT_BINARY_DIR}/ifc)

if(NOT EXISTS ${ifc_dest})
  file(MAKE_DIRECTORY ${ifc_dest})
endif()

foreach(item ${ifc_src})
  cmake_path(GET item EXTENSION LAST_ONLY last_ext) # src/m.ixx.ifc -> .ifc
  cmake_path(GET item STEM name_we)                 # src/m.ixx.ifc -> m
  set(newname ${ifc_dest}/${name_we}${last_ext})    # -> dest/m.ifc
  # cmake_print_variables(newname)
  file(COPY_FILE ${item} ${newname})                # src/m.ixx.ifc -> dest/m.ifc
endforeach()

AR-devJ commented 7 months ago

I am learning C++ using VSCode and MSVC Build Tools. I read about MSVC's module support and how to consume the standard library using modules and also using individual header units. I read this thread entirely, installed the C/C++ extension v1.20.1 (pre-release), and successfully configured IntelliSense support for modules. It's great that IntelliSense is now able to import declarations from the compiled std.ifc interface file too, when it's included in the /ifcSearchDir path.

However, I am facing issues making it work with compiled standard library header units like <print>, <string>, <vector>, etc. Having read the MSVC compiler reference in detail, I am able to get the build done successfully. However, the red squiggles from IntelliSense are what I'd really want to get rid of. I am aware that importing std is a more recommended practice than using header units, but I was just experimenting and successfully compiled a source that imports the latter. But the IntelliSense support is now lost.

My (minimal) source code:

import <print>;

int main() {
    std::println("Hello World!");
    return 0;
}

Contents of c_cpp_properties.json file:

{
    "configurations": [
        {
            "name": "Win32",
            "includePath": [
                "${workspaceFolder}\\**"
            ],
            "defines": [
                "_DEBUG",
                "UNICODE",
                "_UNICODE"
            ],
            "compilerPath": "cl.exe",
            "cStandard": "c17",
            "cppStandard": "c++23",
            "compilerArgs": [
                "/ifcSearchDir",
                "${workspaceFolder}\\ifc"
            ],
            "intelliSenseMode": "windows-msvc-x64",
            "windowsSdkVersion": "10.0.22621.0"
        }
    ],

    "version": 4
}

tasks.json

{
    "tasks": [
        {
            "type": "shell",
            "label": "C++23: Build project from active workspace",
            "options": {
                "shell": {
                    "executable": "cmd.exe",
                    "args": [
                        "/C",
                        "if not exist \"${workspaceFolder}\"\\output\\ ( mkdir \"${workspaceFolder}\"\\output )",
                        "else ( del /q \"${workspaceFolder}\"\\output )",
                        "&&",
                        "if not exist \"${workspaceFolder}\"\\ifc\\ ( mkdir \"${workspaceFolder}\"\\ifc )",
                        "else (del /q \"${workspaceFolder}\"\\ifc )",
                        "&&"
                    ]
                },
                "cwd": "${workspaceFolder}"
            },
            "command": "cmd.exe",
            "args": [
                "/C",

                "cl.exe",
                "/EHsc",
                "/std:c++latest",
                "/exportHeader",
                "/headerName:angle",
                "/Fooutput\\",
                "/ifcOutput",
                "ifc\\",
                "print",
                "&&",

                "cl.exe",
                "/Zi",
                "/EHsc",
                "/c",
                "/std:c++latest",
                "/nologo",
                "/headerUnit:angle",
                "print=ifc\\print.ifc",
                "/Fooutput\\",
                "${workspaceFolder}\\*.cpp",
                "&&",

                "link.exe",
                "/out:output\\${workspaceFolderBasename}.exe",
                "output\\*.obj",
                "&&",

                "cd",
                "output",
                "&&",
                "del",
                "*.obj"
            ],
            "problemMatcher": [
                "$msCompile"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            },
            "detail": "Builds header unit dependency, compiles source file and links them together"
        }
    ],

    "version": "2.0.0"
}

Even after the build succeeds and print.ifc gets generated in the ifc\ subdirectory, IntelliSense is unable associate it with the import statement and reports "print" is not an importable header C/C++(3270) error. The following screenshot shows this situation:

IntelliSenseError

How to resolve this issue? I can conveniently use import std; in all my learning projects and get good IntelliSense support, but i really want to know what's going wrong here despite the build success. Am I missing any compilerArgs besides /ifcSearchDir?

Also there's a second issue. While IntelliSense is able to import declarations from std.ifc (derived by compiling std.ixx standard library module), it isn't able to do the same for std.compat, even when std.compat.ifc lies in the /ifcSearchDir path. Strangely enough, this occurs only when the code uses members from the std namespace wherein IntelliSense encounters an internal error C/C++(1504). When only C runtime functions from the global namespace (like printf()) are used, there's no issue at all. If required I can share the relevant configuration and source files from that project too.

For now, I would really like to know how I can work around the first issue of IntelliSense not recognizing compiled header units.

Challanger524 commented 7 months ago

@AR-devJ, C++20 modules feature is still not full-scale implemented and somewhat raw. And even being not perfectly designed due to macros handling (omitting) and other flaws.

You have 2 options:

stick to Visual Studio 22 (Community) - is a best free C++ compiler for Windows with fully implemented c++ modules feature
skip c++ modules for now (it will take several more years for the feature to get mature and tested, maybe even more)

I picked variant №2 while starting pet-project in 2024 coz I want to have ability to develop with all 3 major compilers (msvc, gcc, clang) in VS Code IDE, which is not possible right now. Modules is not a core c++ feature, so feel free to skip for now since you can adopt/learn them later.

bobbrow commented 7 months ago

@AR-devJ, The experience with header units is still sub-optimal. You might find that the red squiggles you're seeing around types defined in the standard headers will exist in Visual Studio as well (since we use largely the same tech for IntelliSense). We are tracking a number of these issues and are reporting them to EDG as we become aware of them. If there are any that are particularly annoying for you, you may report them in a new issue and we'll check them against the ones we are tracking to make sure they all get reported.

LiAuTraver commented 7 months ago

@AR-devJ same issue here. whenimport std or some <header>, I always use

#ifndef my_module
#include <print>
#else
import <print>
#endif

to cheat the vscode intellisense, and add /D my_module into compiler args. if you insist on using import/export in vscode, this might be a cost-effective trick.

p.s.: It's so disgusting that C++ module is designed to improve compile speed, code safety and maintainability but it turns out we spent lots of time to achieve it. Really sad.

mitsukuri commented 7 months ago

@LiAuTraver Utterly sad indeed. It looks like all the resources are thrown somewhere else and someone on top doesn't deem C++ anything but a legacy maintenance-mode language anymore.

With this AI bullshit creeping out of every corner of recent VSCode builds and elsewhere, they might be as well deeming the olden programming with human brain and hands not worth the fuss itself.

sean-mcmanus commented 7 months ago

@mitsukuri Microsoft is still investing in C++, lots of Microsoft products are still built on C++, all the resources aren't being thrown somewhere else, and it's not in legacy maintenance-mode. You can read about the work that is being done at https://devblogs.microsoft.com/cppblog/ .

Our IntelliSense compiler is based on the EDG compiler and they haven't finished implementing modules support yet. See https://www.edg.com/c/features and click on the C++ 20 link:

Other compilers haven't even finished implemented modules 100% to spec: see https://en.cppreference.com/w/cpp/compiler_support

I've personally been very impressed by the AI tools like GitHub Copilot -- I highly recommend trying them out yourself. They can do code reviews (and actually find bugs, unideal code, etc.), rewrite/improve comments, generate code, complete lines of code, answer programming questions, etc.

Longhanks commented 6 months ago

@AR-devJ same issue here. whenimport std or some <header>, I always use
#ifndef my_module
#include <print>
#else
import <print>
#endif
to cheat the vscode intellisense, and add /D my_module into compiler args. if you insist on using import/export in vscode, this might be a cost-effective trick.

p.s.: It's so disgusting that C++ module is designed to improve compile speed, code safety and maintainability but it turns out we spent lots of time to achieve it. Really sad.

I do the same, and by the way, you can use #ifdef __INTELLISENSE__, which is defined for when the IntelliSense engine parses files.

MathiasMagnus commented 5 months ago

This is the best I could muster (for MSVC)

Use CMake 3.30-rc3 (or other 3.30+ derivate)
Be sure to set CMAKE_EXPERIMENTAL_CXX_IMPORT_STD in your [preset file | command-line | script] to "0e5b6991-d74f-4b3d-a41c-cf096e0b2508"
Use the Ninja (Multi-Config) generator. MSBuild isn't supported by CMake 3.30 yet for built-in import std handling
Have a CMakeLists.txt that looks morally equivalent to

cmake_minimum_required(VERSION 3.30)
project(Test-C++Modules LANGUAGES CXX)
add_executable(Test-C++Modules Main.cpp)
target_compile_features(Test-C++Modules PRIVATE cxx_std_23)
set_target_properties(Test-C++Modules PROPERTIES CXX_MODULE_STD ON)
target_compile_options(Test-C++Modules PRIVATE $<$<CXX_COMPILER_ID:MSVC>:/ifcSearchDir$<TARGET_PROPERTY:__cmake_cxx23,BINARY_DIR>\\CMakeFiles\\__cmake_cxx23.dir\\$<CONFIG>>)

Do note that you need to compile your code for IntelliSense to light up, as the target CMake target __cmake_cxx23 is only created when you trigger your first build.

However, it would be lovely if CppTools would learn to react to modmap files generated by CMake. I don't really care about which extension bears the bulk of the load, CppTools or CMakeTools, but it does generate a module map for every object file. In this simplest of cases, it generates <build_root>\CMakeFiles\Test-C++Modules.dir\Debug\Main.cpp.obj.modmap:

-reference std=CMakeFiles\__cmake_cxx23.dir\Debug\std.ifc

It would be lovely if ifcSearchDir need not be set in scripts on targets with this fairly horrendous syntax only to light up external tooling features when the project otherwise compiles just fine.

@bobbrow @sean-mcmanus

bobbrow commented 5 months ago

@MathiasMagnus we understand that there are several pain points with modules right now. From the C++ extension's perspective, the changes you mention should go into CMake Tools because the C++ extension does not attempt to understand the build system. I would recommend opening an issue on the CMake Tools repo if you haven't already.

1vanK commented 4 months ago

It looks like Microsoft is forcing everyone to Rust.

LiAuTraver commented 4 months ago

It looks like Microsoft is forcing everyone to Rust.

although it's off-topic I think, to be fair, not the fault of Microsoft, the C++ ecosystem itself has so many historical problems and the language just has no owners but a committee. Every slight change needs several companies to work together for it. not an implementer myself, I was also wondering why module feature was so hard to implement in C++ than other languages.

luca-script commented 2 months ago

Seeing as there hasn't been many updates for a while, I would suggest a solution similar to below:

For support with GCC and Clang, I would personally take the approach adding support for the C++ module mapper API used by GCC, as documented partially here in the GCC documentation (And its references) and implemented in libcody. And extending the protocol with many more commands for various purposes (i.e. source file translation)

With that support, an implementation of a smart build system (Which does not include CMake, as it currently generates module dependency files, which are incredibly inefficient) can create a service which talks to the C++ language server and tells it where modules are. This also allows the build system to apply any transformations or create virtual files if generated files are needed.

This would be preferrable to requiring a build system to generate lots of files or a module dependency tree or resolution file, which is not desirable and should not be done.

We could also with the ability to talk to a module mapper/build service, do things like file-specific compilers or configuration queries via more protocol extensions, as the protocol implemented by libcody can be easily extended with new commands if need be. i.e. our language service can ask the build tool important questions for proper language service availability and functionality:

sequenceDiagram
    Language Service->>Build Tool: MODULE-SOURCE foo.bar
    Build Tool->>Language Service: PATHNAME /workspace/dependencies/foo/src/bar.cpp
    Language Service->>Build Tool: FILE-CONFIGURATION /workspace/dependencies/foo/src/bar.cpp
    Build Tool->>Language Service: PATHNAME /tmp/configuration-7aee052f0c19772a6e436d184816ad7d
    Language Service->>Build Tool: FILE-UPDATE /workspace/src/kaz.cpp <start offset>:<end offset> <start offset>:<end offset> <start offset>:<end offset>
    Note right of Build Tool: The build tool rebuilds the module
    Build Tool->>Language Service: ERRORS 182:"Variable 'foo' not found" 173:"Undefined reference to function 'bar'"
    Language Service->>Build Tool: RUN-TESTS
    Build Tool->>Language Service: BOOL TRUE

The advantage of such a system is that a build system like CMake can easily add new features without waiting for LSP support, and allowing the build tool to be aware of the language service and rebuild files in the background, which would greatly accelerate the developer experience. The disadvantage would be requiring an implementation of this system, and proper maintenance. In addition, I'm not sure if Visual Studio will implement such features, or if a seperate system will be required (Seeing as Visual Studio likes to do its own thing regarding C++ feature support) In addition, we'd probably need to gate it behind an option in .vscode/settings.json:

{
  "C_Cpp.backend": "|foobarservice $(ProjectDir)", // Run foobarservice and talk the module mapper/build tool service protocol using stdin/stdout

A fallback implementation for file list-style module dependency resolution files would need to implemented as a seperate service for support with current versions of CMake, and this would likely require further integration between CMake and its build system (Most of the time Makefiles or Ninja) for any real kind of effect.

Some answers I want to put out there so there won't be needless useless discussion (In Q&A format): Q: Why not make a full language service at this point? A: It is incredibly difficult to make a robust language service, and piggying off of an existing language service and only implementing module translation and build integration would significantly reduce implementation cost and complexity. Q: Why add build integration? A: Build integration is an important part of the modern IDE experience, and real-time rebuilding of edited modules will GREATLY speed up build times. Q: Why extend the existing C++ module mapper protocol? A: The existing protocol has similar enough goals that extending it makes sense. It also avoids requiring a brand new proprietary/largely undocumented protocol to be made. Q: Why even care with this? A: I deeply care about IDE integration, and the current state of C++ modules in IDEs is extremely poor. By adopting the ability for your build tool to provide C++ module translation and build acceleration, we can make the experience better. Q: This is pointless, I don't want it. A: If you don't want it, don't use it. I do not want endless "But this isn't how I want it to work ideally!!!!" comments here, as these are not productive to the current problem. Q: How long do you think this would take to get implemented and adopted into tools like CMake? A: I forsee it taking around 5-8 years from seeing how long it has taken for C++ modules to get proper support and considering that this is quite a bit more complex and off-topic. Q: Why should I care? (For anyone who doesn't directly work on the language service) A: That's the neat part, you don't. The only people who need to care about these things are the people who make compilers, make the build tools and the language services. Q: Will this affect me as a Visual Studio and MSVC user? A: Probably not, as Visual Studio has their own integration for this and their own (proprietary) language service which already fully supports C++ modules and build integration with MSBuild. Q: Why this early? When GCC and Clang barely support C++ modules. A: C++ modules will likely take a long while to be fully supported, this is just a step to C++ module adoption.

Take this not as a "we must implement this now!", but as an idea to throw out there into the void to see what people who actually work on the LSP think of this. Also please do not reply with hate, this is unproductive and I do not want the same situation as often happens with Wayland where 30 people scream "I DON'T THINK THIS IS GOOD" and the entire conversation devolves into hatespeech against developers and implementers.

vspefs commented 1 month ago

Seeing as there hasn't been many updates for a while, I would suggest a solution similar to below:

For support with GCC and Clang, I would personally take the approach adding support for the C++ module mapper API used by GCC, as documented partially here in the GCC documentation (And its references) and implemented in libcody. And extending the protocol with many more commands for various purposes (i.e. source file translation)

With that support, an implementation of a smart build system (Which does not include CMake, as it currently generates module dependency files, which are incredibly inefficient) can create a service which talks to the C++ language server and tells it where modules are. This also allows the build system to apply any transformations or create virtual files if generated files are needed.

This would be preferrable to requiring a build system to generate lots of files or a module dependency tree or resolution file, which is not desirable and should not be done.

We could also with the ability to talk to a module mapper/build service, do things like file-specific compilers or configuration queries via more protocol extensions, as the protocol implemented by libcody can be easily extended with new commands if need be. i.e. our language service can ask the build tool important questions for proper language service availability and functionality:
sequenceDiagram
    Language Service->>Build Tool: MODULE-SOURCE foo.bar
    Build Tool->>Language Service: PATHNAME /workspace/dependencies/foo/src/bar.cpp
    Language Service->>Build Tool: FILE-CONFIGURATION /workspace/dependencies/foo/src/bar.cpp
    Build Tool->>Language Service: PATHNAME /tmp/configuration-7aee052f0c19772a6e436d184816ad7d
    Language Service->>Build Tool: FILE-UPDATE /workspace/src/kaz.cpp <start offset>:<end offset> <start offset>:<end offset> <start offset>:<end offset>
    Note right of Build Tool: The build tool rebuilds the module
    Build Tool->>Language Service: ERRORS 182:"Variable 'foo' not found" 173:"Undefined reference to function 'bar'"
    Language Service->>Build Tool: RUN-TESTS
    Build Tool->>Language Service: BOOL TRUE
The advantage of such a system is that a build system like CMake can easily add new features without waiting for LSP support, and allowing the build tool to be aware of the language service and rebuild files in the background, which would greatly accelerate the developer experience. The disadvantage would be requiring an implementation of this system, and proper maintenance. In addition, I'm not sure if Visual Studio will implement such features, or if a seperate system will be required (Seeing as Visual Studio likes to do its own thing regarding C++ feature support) In addition, we'd probably need to gate it behind an option in .vscode/settings.json:
{
  "C_Cpp.backend": "|foobarservice $(ProjectDir)", // Run foobarservice and talk the module mapper/build tool service protocol using stdin/stdout
A fallback implementation for file list-style module dependency resolution files would need to implemented as a seperate service for support with current versions of CMake, and this would likely require further integration between CMake and its build system (Most of the time Makefiles or Ninja) for any real kind of effect.

Some answers I want to put out there so there won't be needless useless discussion (In Q&A format): Q: Why not make a full language service at this point? A: It is incredibly difficult to make a robust language service, and piggying off of an existing language service and only implementing module translation and build integration would significantly reduce implementation cost and complexity. Q: Why add build integration? A: Build integration is an important part of the modern IDE experience, and real-time rebuilding of edited modules will GREATLY speed up build times. Q: Why extend the existing C++ module mapper protocol? A: The existing protocol has similar enough goals that extending it makes sense. It also avoids requiring a brand new proprietary/largely undocumented protocol to be made. Q: Why even care with this? A: I deeply care about IDE integration, and the current state of C++ modules in IDEs is extremely poor. By adopting the ability for your build tool to provide C++ module translation and build acceleration, we can make the experience better. Q: This is pointless, I don't want it. A: If you don't want it, don't use it. I do not want endless "But this isn't how I want it to work ideally!!!!" comments here, as these are not productive to the current problem. Q: How long do you think this would take to get implemented and adopted into tools like CMake? A: I forsee it taking around 5-8 years from seeing how long it has taken for C++ modules to get proper support and considering that this is quite a bit more complex and off-topic. Q: Why should I care? (For anyone who doesn't directly work on the language service) A: That's the neat part, you don't. The only people who need to care about these things are the people who make compilers, make the build tools and the language services. Q: Will this affect me as a Visual Studio and MSVC user? A: Probably not, as Visual Studio has their own integration for this and their own (proprietary) language service which already fully supports C++ modules and build integration with MSBuild. Q: Why this early? When GCC and Clang barely support C++ modules. A: C++ modules will likely take a long while to be fully supported, this is just a step to C++ module adoption.

Take this not as a "we must implement this now!", but as an idea to throw out there into the void to see what people who actually work on the LSP think of this. Also please do not reply with hate, this is unproductive and I do not want the same situation as often happens with Wayland where 30 people scream "I DON'T THINK THIS IS GOOD" and the entire conversation devolves into hatespeech against developers and implementers.

There's no need to extend existing facilities. CMI/BMI files are enough for a language front end to do IntelliSense. Cpptools can't provide it, simply because the vendor of its front end hasn't finished this feature. It doesn't only concern reading a new file format, but also involves quite a few problems on C++ itself, and some parts of the front end's basic structure. Clangd has finished its support for modules a while ago, which you can definitely try. Back to CMI/BMI files. They are not designed for IntelliSense, but as a kind of standalone utility that caches the final interface and logic of a module unit. It works as a cache that helps avoid repeated work for compilers, and offers a new way to distribute your module. But just so it happens that it contains the final interface of a module unit, so front ends use it to support IntelliSense for modules. So no, though it helps front ends do IntelliSense, it's a utility for compiling your program and should not hold information that a build system/IntelliSense extension would additionally know, nor any information that makes it less portable. And you recommended that we store source file locations and 'offsets' in CMI/BMI files, so that we can read a module's source file directly for IntelliSense. Well, now that you know a CMI/BMI file already contains the final interface of a module unit, there's no longer need for anyone to 'trace back' to the source file. Besides, what we just discussed clarifies that a CMI/BMI file shouldn't contain such information. And reading directly from source files is time-and-performance-costing. If we don't have CMI/BMI files, yes, we would HAVE to do that, and maybe do some heavy caching to get ourselves a enjoyable writing experience. But now we have them, right? So we mentioned 'standalone' and 'portable.' Like CMI/BMI files, those dependency files are generated for this purpose, too. You should know that C++ doesn't have a default build system (or say, a standard build structure) and everything is based on little, individual translation units. So yes, we need those silly dependency files. More on CMI/BMI files and dep files here: P1838

I don't feel this is the place for some long, arrogant, cocky, and error-prone (I hope it's not) lectures. But you don't have an email or something like that, so I'll have to reply here. If my reply disturbs/annoys anybody, or it's wrong (God, I hope not), please do inform me. And luca, if you're interested in how the language and helper facilities interact and pass information between each other, this recent paper might give you a brief view: P3335.

Challanger524 commented 1 month ago

Move on guys, it takes too long and waiting won't speed up anything. I am starting to suspect that this feature may even never come to vscode.

kamrann commented 1 month ago

It appears that the processing of /reference is not taking into account the compile_commands.json directory entry for resolving relative paths.

The following gives a "cannot find module file" error:

"arguments": ["/reference", "mod=./foo.ifc"], "directory": "C:/somewhere"

whereas this does not:

"arguments": ["/reference", "mod=C:/somewhere/foo.ifc"]

Also rather annoyingly, it seems if I point the extension to a compile_commands.json file that's outside of the workspace root it will just silently behave as if it doesn't exist.

1vanK commented 1 month ago

Move on guys, it takes too long and waiting won't speed up anything. I am starting to suspect that this feature may even never come to vscode.

Problems with modules are not only in IDEs, but also in compilers, everything is heading towards modules being "deprecated" in the next C++ standards

Challanger524 commented 1 month ago

Move on guys, it takes too long and waiting won't speed up anything. I am starting to suspect that this feature may even never come to vscode.

Problems with modules are not only in IDEs, but also in compilers, everything is heading towards modules being "deprecated" in the next C++ standards

Eah, modules being poorly designed by committee and overtimed by some compiler/parser vendors. Sad that C++ rushing for new useless features leaving old structural weaknesses behind.

microsoft / vscode-cpptools

Add IntelliSense for C++20 modules importing #6302

This is the best I could muster (for MSVC)