cannot include X as it contains subpackage X

[patmaddox]

I have a metarepo, and some sub-repos include BUILD files. I want to use please to build from these sources. I am not able to add please to these repos - I need to write a higher-level build task using the source as a dependency, and I want it to use my local source so I can edit it. A good example is building the FreeBSD source code.

Right now I am not able to write this build task, because please refuses to copy the source directory.

It was suggested to me that I might change BuildFileName in my metarepo. This works as a short-term solution, but I think it kicks the can down the road / forces me to chase a moving target. I would need to change my BuildFileName any time a new repo causes a conflict. More importantly, all repos that I have access to would need to use a different name, so now I get into a funky naming convention, e.g. BUILD.myorg.proj1 BUILD.myorg.client_a.secret-project. I may not be permitted to do that, even if I wanted to.

proposal: respect blacklistdirs when copying src

Assuming that there's currently no simple way for me to copy a dir that has BuildFileName in it...

I believe that blacklistdirs already expresses the idea that a subdir is independent of the repo, and is not a subpackage. Therefore I propose that when defining srcs, it should respect the blacklistdirs setting and permit any matching dirs that include a BuildFileName.

example script

#!/bin/sh
set -eu

# Example demonstrating how please will refuse to include a src dir if
# it has a BUILD file in it

root=$(mktemp -d -t plz-subdir)

main() {
    init-repo
    init-subrepo
    plz-build
}

init-repo() {
    cd ${root}
    plz init

    cat <<EOF >> .plzconfig

[parse]
blacklistdirs = subrepo
EOF

    cat <<EOF > BUILD
filegroup(
    name = "subrepo",
    srcs = ["subrepo"],
)
EOF
}

init-subrepo() {
    cd ${root}
    mkdir subrepo
    cd subrepo
    yes | plz init
    touch BUILD
}

plz-build() {
    cd ${root}
    plz build :subrepo
}

main

output

$ sh subdir_with_build_file.sh
Wrote config template to /tmp/plz-subdir.XhdGNGZNaE/.plzconfig, you're now ready to go!

<SNIP>

You already seem to be in a plz repo (found /tmp/plz-subdir.XhdGNGZNaE/.plzconfig). Continue: y
Wrote config template to /tmp/plz-subdir.XhdGNGZNaE/subrepo/.plzconfig, you're now ready to go!

<SNIP>

Build stopped after 20ms. 1 target failed:
    //:subrepo
cannot include subrepo as it contains subpackage subrepo

[patmaddox]

After experimenting a bit more, it clicked that I'm basically doing the same thing as github_repo - only instead of fetching code via HTTP, I want to fetch it from disk. This led to another insight: if I use my SCM's built-in cloning rather than cp, then I'll only get committed files, similar to if I were to download a tarball from GitHub. So with new_http_archive as a reference, I came up with:

def jj_repo(name:str, version:str=None):
    target_name = name
    version_flag = ""

    if version:
        target_name = f"{name}-{version}"
        version_flag = f"-r {version}"

    cmd = " && ".join([
        f"jj -R $(plz query reporoot)/repos/{name}.jj workspace add --name __plz {version_flag} {target_name}",
        f"jj -R $(plz query reporoot)/repos/{name}.jj workspace forget __plz",
        f"rm -rf {target_name}/.jj"
    ])

    workspace_rule = build_rule(
        name = target_name,
        cmd = cmd,
        outs = [target_name],
        _subrepo = True
    )

    return subrepo(
        name = target_name,
        dep = workspace_rule
    )