Using .sl/store/git for transparent git interop?

[adamh-oai]

Like a lot of orgs we have all sorts of tools that expect you're working in a git repo (for pre-commit, deploy etc), which is a blocker for sapling adoption. I've played around some with hooks and scripts to transparently sync a git repo from sapling though it's all sort of slow and fragile.

I noticed .sl/store/git has an existing git repo that (?) mirrors the sapling contents. Is there any existing support for this I'm missing? Any suggestions for a good way to implement it?

Some specific git operations I run into:

• What Is HEAD?
• Is the working directory dirty?
• List modified files, run a linter on them

[adamh-oai]

I've ended up at a solution with these two hooks:

[hooks]
txnclose = sl-git
update = sl-git

And sl-git looks like:

# point git HEAD to sapling HEAD
case $HG_HOOKTYPE in
    "txnclose")
        git update-ref HEAD `sl whereami`
        ;;
    "update")
        git update-ref HEAD $HG_PARENT1
        ;;
esac

# update the index without changing the working directory
git read-tree HEAD

This seems to keep git in sync with sapling in all the cases I tried, but I'm a little uneasy there might be cases I'm not aware of.

[ahupp]

Now I'm wondering about pre-commit hooks (eg formatters), and how to make them work across all the paths that update a commit (amend, rebase, etc)

[quark-zju]

The .sl/store/git is an implementation detail that we plan to change. While you can ln -s .sl/store/git .git and get something working, it's not an officially supported setup and will likely break in the future.

See #182 for more discussions. We don't want to support .git/ file format compatibility, but are happy to accept patches that provide a command-line git shim even if it's imperfect. If your use-case is automation (ex. the GitLens extension), they might just need a subset of git features and a shim could be a feasible solution.

[skevy]

@ahupp @adamh-oai just out of curiosity...are you using Sapling with a large repo? When I use your approach on a large repo it's quite slow to run git status after changing a commit in sapling as it has to refresh the index...

[adamh-oai]

Not sure what you count as "large", its 33k files. The first git status does take a bit but this isn't on my critical path.

[skevy]

Yah fair enough. Mine is > 10 times that, so it's more noticeable. I really like your solution -- was previously doing this with a separate worktree from .sl/git/store and then would rsync on demand + various other cmds to get the repo in the correct state. The problem with that is similar I think to what you experienced -- it can be finicky.

We have a lot of scripts that use git status/git diff to determine changed files, and so this is kind of on the critical path for me. But it's not necessarily a deal breaker as it's only a one-time thing after a ref change.

[oleg-codaio]

Wanted to share some (hacky) Makefile commands I've put together to get Sapling set up in a clone of an existing GitHub repo (thanks @adamh-oai and @adzenith!). This handles a few requirements:

• Transition to use Sapling in existing directory to preserve local untracked/ignored files (moving .sl seems to work...)
• Execute pre-push/pre-commit GitHub hooks using their HG counterparts (mostly for lint)
• Keep some git commands working (for existing tooling like lint-staged) by linking .git, setting the origin/main ref
• Using the single-commit PR workflow for cleaner reviews/diffs on GitHub (due to running into issues with ReviewStack)
• Enable reverting back to Git when needed

.PHONY: experimental-sapling-install
experimental-sapling-install:
    # Hack: https://github.com/facebook/sapling/issues/182 and https://github.com/facebook/sapling/issues/745
    if [ -d .sl ]; then \
        echo "Sapling is already installed"; \
        exit 1; \
    fi; \
    if ! command -v sl; then \
        echo "Installing Sapling..."; \
        brew install sapling; \
    fi; \
    if [ -d .git/.sl.bak ]; then \
        echo "Found backup; restoring"; \
        mv .git/.sl.bak .sl; \
    else \
        DIR=$$(basename $$(pwd)); \
        pushd ..; \
        echo "Cloning repo (this will take a while)..."; \
        sl clone https://github.com/path-to/repo "$${DIR}.temp"; \
        mv "$${DIR}.temp/.sl" "$${DIR}/"; \
        rm -rf "$${DIR}.temp"; \
        popd; \
        sl config --local hooks.precommit.githooks 'for f in .githooks/pre-commit/*; do sh "$$f"; done'; \
        sl config --local hooks.preoutgoing.githooks 'for f in .githooks/pre-push/*; do sh "$$f"; done'; \
        sl config --local hooks.update.githooks 'for f in .githooks/post-rewrite/*; do sh "$$f"; done'; \
        sl config --local hooks.update.syncgit 'git update-ref HEAD $$HG_PARENT1; git read-tree HEAD'; \
        sl config --local hooks.txnclose.syncgit 'git update-ref HEAD `sl whereami`; git read-tree HEAD'; \
        sl config --local github.pr_workflow single; \
    fi; \
    mv .git .sl/.git.bak; \
    git init --separate-git-dir .sl/store/git .; \
    git config --local core.bare false; \
    git update-ref origin/main remotes/remote/main; \
    echo "Sapling installed."

.PHONY: experimental-sapling-uninstall
experimental-sapling-uninstall:
    if [ ! -d .sl ]; then \
        echo "Sapling is not installed"; \
        exit 1; \
    fi; \
    rm .git; \
    mv .sl/.git.bak .git; \
    mv .sl .git/.sl.bak; \
    git config --local --unset core.bare; \
    git reset --hard; \
    echo "Sapling uninstalled. To purge, run: rm -rf .git/.sl.bak"

So far this seems to work. The only issue I've found is that updating the ref keeps showing some _unused_branch branch in sl sl, but it's mostly harmless. I've also needed to hide / turn off existing Git tooling in my IDE so I don't see statuses for two SCMs simultaneously.

This is obviously pretty brittle, so use at your own risk. But this setup enables experimenting with Sapling in existing repos. Personally I think interoperability is something this project should invest in to be successful, since sl is most useful in large repositories and those are 1) already likely using Git with lots of existing automation/tooling and 2) need to support a gradual transition or a subset of developers using sl while everyone else still uses Git. The features make it worth it!