GIT CLONE command not working with private repos

[alexcb]

Consider an Earthfile like:

WORKDIR /environ

private:
    GIT CLONE \
        --branch=master \
        gitlab.com/alexcb/privproj .

public:
    GIT CLONE \
        --branch=main \
        github.com/earthly/earthly .

I can run earthly +public without any issues; however when I try to run it against my private repo with earthly +private, it fails with:

$ earthly-v0.4.4 --verbose --debug +private
           buildkitd | Found buildkit daemon as docker container (earthly-buildkitd)
            +private | gitlab.com/alexcb/privproj
            +private | --> GIT CLONE (--branch master) gitlab.com/alexcb/privproj
               +base | --> WORKDIR /environ
               +base | Completed in 31.858µs
            +private | fatal: could not read Username for 'https://gitlab.com': terminal prompts disabled
            +private | WARN: (GIT CLONE (--branch master) gitlab.com/alexcb/privproj) failed to fetch remote https://gitlab.com/alexcb/privproj: exit status 128
            +private | Completed in 798.444207ms
Summary of timing information
Note that the times do not include the expansion of commands like BUILD, FROM, COPY (artifact).
            +private | (gitlab.com/alexcb/privproj) 798.444207ms
               +base | () 31.858µs
===============================================================
Total           798.476065ms
Total (real)    929.851393ms
Error: build target: build main: bkClient.Build: failed to solve: rpc error: code = Unknown desc = failed to load cache key: failed to fetch remote https://gitlab.com/alexcb/privproj: exit status 128
Check your git auth settings.
Did you ssh-add today? Need to configure ~/.earthly/config.yml?
For more information see https://docs.earthly.dev/guides/auth

This is with ~/.earthly/config.yml:

global:
  cache_size_mb: 40000
git:
  gitlab.com:
    auth: https
    user: alexcb
    password: "secret"

however the auth works when I reference the git repo on the command line via: earthly gitlab.com/alexcb/privproj+docker

[vladaionescu]

I see two issues here:

• gitlab.com/alexcb/privproj doesn't look like a valid git URL
• I don't think we should be using the git auth configured in Earthly for the GIT CLONE command. But perhaps there are good reasons to support this.

[alexcb]

I changed my earthfile to:

WORKDIR /environ

private:
    GIT CLONE \
        --branch=master \
        https://gitlab.com/alexcb/privproj.git .

public:
    GIT CLONE \
        --branch=main \
        github.com/earthly/earthly .

and it's still not working.

[alexcb]

here's a potential test-case showing this issue: https://github.com/earthly/earthly/pull/709

[vladaionescu]

I don't think we should be using the git auth configured in Earthly for the GIT CLONE command. But perhaps there are good reasons to support this.

BTW, I realized that our docs say that we support this.

[baptiste0928]

Same issue with a private bitbucket repo : fatal: could not read Username for 'https://bitbucket.org': terminal prompts disabled.

[jazzdan]

I'm also experiencing this on GitHub with GitHub actions.

Link to Slack discussion https://earthlycommunity.slack.com/archives/C01DL2928RM/p1615570638014000

[stk0vrfl0w]

I've been experiencing the same issue as well.

Initially, I had used the workaround suggested here: https://earthlycommunity.slack.com/archives/C01DL2928RM/p1610568364141300?thread_ts=1610505194.098400&cid=C01DL2928RM

Lately, though, I've found that a combination of secrets in .env and a small git credential helper made my workflow a little smoother when requiring multiple sets of credentials.

.env

GITHUB_USER_FOO=foo
GITHUB_TOKEN_FOO=footoken
GITHUB_USER_BAR=bar
GITHUB_TOKEN_BAR=bartoken

Earthfile

FROM goboring/golang:1.15.8b5

WORKDIR /app

RUN git config --global credential.helper '!p() { printf "username=${GIT_USER}\npassword=${GIT_PASS}"; }; p'

clone:
    RUN --secret GIT_PASS=+secrets/GITHUB_TOKEN_FOO --secret GIT_USER=+secrets/GITHUB_USER_FOO \
        git clone https://github.com/foo/foo-proto.git
    RUN --secret GIT_PASS=+secrets/GITHUB_TOKEN_BAR --secret GIT_USER=+secrets/GITHUB_USER_BAR \
        git clone https://github.com/bar/bar-app.git