--editable does not work with manifest mode

[autoantwort]

Describe the bug ./vcpkg install --editable

... 
-- Cleaning sources at /Users/leanderSchulten/git_projekte/Lichtsteuerung/vcpkg/buildtrees/aubio/src/0.4.9-f1b891adc9.clean. Use --editable to skip cleaning for the packages you specify.

Even if you add the --editable argument, it cleans the sources.

To Reproduce Steps to reproduce the behavior: See description

Expected behavior Respect the --editable argument

[strega-nil]

@autoantwort --editable doesn't really work with manifest mode; we'd need to have real design work put in to support that, since it wasn't designed into the original.

[JonLiu1993]

@autoantwort, Currently, the option --editable is not supported in manifest mode

PS F:\Feature-test\test\vcpkg> ./vcpkg install --editable
Error: The option --editable is not supported in manifest mode.
Using manifest file at F:\Feature-test\test\vcpkg\vcpkg.json.

[Edi61]

Hello, is there maybe some update or some workaround possibility expect not to use manifest mode? Maybe by using a vcpkg-configuration.json file? Here you can select an overlay-port where you can reference an edited fork commit -> but this would force you to commit every little code change which is not sensefull.

[autoantwort]

No there is no update and it still requires classic mode.

[abroun]

Hi all,

Can you please advise a good workflow for when you're working in manifest mode and need to make a package editable?

My project is setup so that I have vcpkg imported as a git submodule, a vcpkg.json manifest in the project root and then I configure with cmake using the following

cmake -DCMAKE_TOOLCHAIN_FILE=../3rd_party/vcpkg/scripts/buildsystems/vcpkg.cmake ../

This all works well until I need to make a quick patch to a large dependency (OpenCV) which I'd like to do in vcpkg/buildtrees rather than go for the long winded alternative which I think would be 'patch opencv repo, update port file, repull all source and rebuild from scratch'. (On my machine OpenCV takes almost an hour to build from scratch).

In this situation is the best solution currently to do the following?

• Run 'vcpkg install PACKAGE --editable' (in a separate root folder so that vcpkg.json doesn't interfere)
• Edit the source code in the new dir that lacks the .clean suffix
• Run 'vcpkg remove PACKAGE' followed by 'vcpkg install PACKAGE --editable'
• Manually copy the new files from vcpkg/installed to the cmake build/vcpkg_installed folder (or hack your CMakeLists.txt file somehow)

Any advice here would be much appreciated. It all seems a bit clunky so if I've got the wrong end of the stick here or have missed something obvious please let me know. :)

Many thanks

Alan

[msvetkin]

As a workaround you can set _VCPKG_EDITABLE property in a triplet file like this:

if (PORT MATCHES "<port-name>")
  set(_VCPKG_EDITABLE ON)
endif()

It will build a particular port as editable, but unfortunately it will rebuild all other ports.

[Joker2387089590]

I have serveral projects that depend on gRPC:

1. gRPC is built with native triplet, which doesn't need to be edited, but required to keep the source code so that I can step into when debugging. Without --editable, however, sometimes the source code will be cleaned.
2. In these projects, I have to use manifest mode for specifing versions of gRPC and its dependencies, otherwise they will be compiled again after updating vcpkg, which takes a lot of time.

I think the lack of option --keep causes the problem:

• --editable means that the source code in buildtrees is necessary, and may be "dirty". It is dangerous to be able to edit the source of a package in manifest mode, because the modifications from different projects may have conflicts.
• clean means users doesn't take care of the source code of packages, so vcpkg have permission to remove sources when rebuilding for a new project.
• So we can add a option for users to tell vcpkg that never removing sources, and reverts all modifications when rebuilding.

[Joker2387089590]

In addition, I have a cross-compiling project that also depends on gRPC, and I need to edit the gRPC source code to fix some compile errors.

My solution is:

1. create a new git branch
2. modify the portfile.cmake of gRPC to append a patch
3. commit and then a new baseline is generated
4. use the new baseline in manifest file

It is a bit complicated when having many revisions.

[sharadhr]

+1 to this.

Having --editable under manifest mode will drastically speed up project bring-ups and migrations. Current methods to patch dependencies make rapid iteration impossible, especially given that manifest mode is the recommended way to manage packages.

Possible current methods are all sub-optimal in some way:

• Clone the dependency repo; make changes, commit; push to some repository; create overlay portfile for the dependency with the new commit (slowest)
• Clone the dependency repo; make changes, make patchfile; create overlay port for the dependency; specify patchfile under PATCHES (less slow)
• Clone the dependency repo; make changes, compile; manually copy binaries/headers to VCPKG_INSTALLED_DIR (not scalable for dependencies with lots of build artifacts)
• Set _VCPKG_EDITABLE in an overlay triplet (unnecessarily rebuilds every other dependency; may not work because binaries could be cached and buildtrees changes will be ignored)

Perhaps it's worth sending vcpkg into an 'unsafe' mode and warn the user that ABIs don't match when a package under manifest mode is specified as --editable. Workflow-wise this doesn't seem very complex:

1. Every saved edit to a dependency source tree specified as --editable should change its build ABI.
2. Under --editable, the source under buildtrees is always rebuilt when the build ABI changes. This should be similar behaviour to modifying a dependency's feature set in vcpkg.json, or specifying a different version, or even removing a dependency, etc.

[silverqx]

editable and head should be able to be set in the vcpkg.json manifest for particular ports because eg. currently I'm hitting bugs that I can't reproduce in classic mode but the bug exists in manifest mode only.

In my case, a library is linked statically against MSVC CRT even if I'm using x64-windows triplet and I can't reproduce it in classic mode. Something like: cmake cl : Command line warning D9025 : overriding '/MDd' with '/MTd'

It would be very helpful to set head and editable in the manifest vcpkg.json so I can add debug messages to the CMakeLists.txt to find out why is this happening, it would save a lot of commits and git reset and similar things.

[gblikas]

Having --editable under manifest mode will drastically speed up project bring-ups and migrations. Current methods to patch dependencies make rapid iteration impossible, especially given that manifest mode is the recommended way to manage packages.

Expanding on @sharadhr 's answer, I think one of the most advantageous things about something like

{
  "dependencies": [
   {
    "name": "foo",
    "default-features": false,
    "features": ["bar"] ,
    "editable": true
   }
  ]
}

is the ability to pseudo-cache.

I agree with what others have suggested for a single-project workflow; just use the CLI to develop with --editable. However, some projects which use CMake or CMakePresets might be in development themselves, using Manifest Mode to include many many dependencies. If something in the root-level CMakeLists.txt file breaks, a rebuild will always happen in the current implementation.

[rokups]

I am also interested in potential workflow for manifest mode. We are looking to migrate our project to vcpkg and our project uses a fair bit of dependencies. For us it is important to maintain reasonable iteration times when working on our dependencies, because we do active debugging and development on them as required. Naturally, safety is not a priority here - fast iteration times and ability to debug from IDE are. If something breaks we can always clear files of dependency in question and rebuild it. Such a feature should have no impact on our users who consume our library.