[bug] `conan export-pkg` doesn't remove `.dirty` marks on packages

[datalogics-kam]

Environment Details (include every applicable attribute)

• Operating System+version: macOS
• Compiler+version: apple-clang 10.0
• Conan version: 1.21.1
• Python version: 2.7.13

Steps to reproduce (Include if Applicable)

We have a project that can't be built in the local cache. It's huge and for other reasons, we're just building it and exporting the package from our local build.

1. conan install project A, which depends on B.
2. Oops, there isn't a B package for this configuration. Forgot to export it to the local cache.
3. The conan install of A tries to build the B package, which fails because there are no sources. At this point, the dirty file is created.
4. Go to project B.
5. conan export . kam/testing -bf mac_debug
6. Go back to the project A that's using B
7. conan install

I get:

B/<version>@kam/testing: WARN: Package is corrupted, removing folder: /Users/kam/.conan/B/<version>/kam/testing/package/408f7dd2c505e67efb1913cddaea764d8dfa6c73

Because the conan export-pkg didn't clear the dirty file, Conan thinks the package is corrupted.

This gets into a cycle that isn't obvious to fix. The only solution is to manually find the 408f7dd2c505e67efb1913cddaea764d8dfa6c73.dirty file in the cache and remove it. Or, conan remove the whole package and make all the configs.

Logs (Executed commands with output) (Include/Attach if Applicable)

[memsharded]

Hi @datalogics-kam

I am trying to reproduce this issue, but so far not possible, I am missing something. I guess in:

The conan install of A tries to build the B package, which fails because there are no sources. At this point, the dirty file is created.

If there is no B package, then it raises a "package not found" error, but nothing is created in the cache regarding package B. Not a folder, not a .dirty file. If there is a B recipe, but not a B binary, than it throws a "binary not existing" error. But still no .dirty file there

What do you mean with There are no sources? Could you please explain a bit better how to get the cache a .dirty file with that process?

[datalogics-kam]

Thanks for responding.

What do you mean with There are no sources?

Basically, the package can't be built from the recipe. We don't export the sources with the recipe.

f there is a B recipe, but not a B binary, than it throws a "binary not existing" error.

Oh, I know why, we use --build missing as a matter of course here.

But there is a build() method, only used when building from a Git checkout. It tries to build, but fails:

CMake Error: The source directory "/Users/kam/.conan/data/B/<version>/kam/testing/build/408f7dd2c505e67efb1913cddaea764d8dfa6c73" does not appear to contain CMakeLists.txt.

Even if I add something to the B conanfile.py:

    def build(self):
        if self.in_local_cache:
            raise Exception("Can't build while installing")

        cmake = self.configure_cmake()
...

The failure leaves the .dirty files.

[datalogics-kam]

I think the culprit is that the conan export-pkg command with just a build folder calls package() method pretty directly, and doesn't set/clear the dirty folder the way conan create works. The following patch seems to clear the dirty file for me, but I'm not claiming I really know what I'm doing, or that I know all the places to fix.

diff --git a/conans/client/cmd/export_pkg.py b/conans/client/cmd/export_pkg.py
index a4fc16f2..5583ecab 100644
--- a/conans/client/cmd/export_pkg.py
+++ b/conans/client/cmd/export_pkg.py
@@ -5,7 +5,7 @@ from conans.client.graph.graph_manager import load_deps_info
 from conans.errors import ConanException
 from conans.model.conan_file import get_env_context_manager
 from conans.model.ref import PackageReference
-from conans.util.files import rmdir
+from conans.util.files import rmdir, set_dirty_context_manager

 def export_pkg(app, recorder, full_ref, source_folder, build_folder, package_folder, install_folder,
@@ -50,9 +50,10 @@ def export_pkg(app, recorder, full_ref, source_folder, build_folder, package_fol
                                    hook_manager, conan_file_path, ref)
     else:
         with get_env_context_manager(conanfile):
-            prev = packager.run_package_method(conanfile, package_id, source_folder, build_folder,
-                                               dest_package_folder, install_folder, hook_manager,
-                                               conan_file_path, ref, local=True)
+            with set_dirty_context_manager(dest_package_folder):
+                prev = packager.run_package_method(conanfile, package_id, source_folder, build_folder,
+                                                dest_package_folder, install_folder, hook_manager,
+                                                conan_file_path, ref, local=True)

     packager.update_package_metadata(prev, layout, package_id, full_ref.revision)
     pref = PackageReference(pref.ref, pref.id, prev)

[memsharded]

Thanks, I have reproduced the error, and proposed a fix as suggested in https://github.com/conan-io/conan/pull/6498.

Planned for Conan 1.22.1 to be released soon. Thanks for all the feedback and help!

[memsharded]

Merged, to be released in 1.22.1