Closed ruXlab closed 3 years ago
Sorry, but I don't have any Linux and Docker experience.
I'm developing this on Windows and my only "contact" with Docker so far was setting up some CI on Bitbucket Pipelines, which happens to use Docker containers under the hood.
So I don't have the knowledge to directly help you with any permission issues.
And SCM Backup just calls the Git executable, so there's nothing special happening in SCM Backup that I could "fix".
That error message sounds like something deep from the bowels of Git.
Also,
git clone
inside container works fine as expected
SCM Backup uses git fetch
, not git clone
. Here are the exact parameters.
So for your example, it would execute this:
git -C "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" fetch --force --prune https://bitbucket.org/xxxx/proj/ refs/heads/*:refs/heads/* refs/tags/*:refs/tags/*
Does this work in your Docker container?
@christianspecht thanks for prompt response
I tried to reproduce what your code does but in Docker:
1) Create folder mkdir "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" -p
2) I noted git init
is used, so I did git init --bare
in that folder
3) Run command above and it failed!
git -C "/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/" fetch --force --prune https://github.com/christianspecht/scm-backup.git refs/heads/*:refs/heads/* refs/tags/*:refs/tags/*
remote: Enumerating objects: 324, done.
remote: Counting objects: 100% (324/324), done.
remote: Compressing objects: 100% (194/194), done.
fatal: Unable to create temporary file '/scm-backup-out/xxx-bitbucket/xxxx#proj/repo/objects/pack/tmp_pack_XXXXXX': Permission denied
fatal: index-pack failed
So, you were right about issue - it was caused by git and/or docker themself. Will look into this later
Thank you!
It's very interesting - git clone
works as expected
git clone https://github.com/christianspecht/scm-backup.git
Cloning into 'scm-backup'...
remote: Enumerating objects: 324, done.
remote: Counting objects: 100% (324/324), done.
remote: Compressing objects: 100% (194/194), done.
remote: Total 3884 (delta 205), reused 212 (delta 128), pack-reused 3560
Receiving objects: 100% (3884/3884), 486.04 KiB | 1.36 MiB/s, done.
Resolving deltas: 100% (2756/2756), done.
So I'd assume it has to do with git init
and the way it creates git index files
git init --bare
produces valid permissions (uid 1000)
root@fa3ca74f5265:/scm-backup-out/xxx-bitbucket# tree -u
.
`-- [1000 ] xxxx#proj
`-- [1000 ] repo
|-- [1000 ] FETCH_HEAD
|-- [1000 ] HEAD
|-- [1000 ] branches
|-- [1000 ] config
|-- [1000 ] description
|-- [1000 ] hooks
| |-- [1000 ] applypatch-msg.sample
| |-- [1000 ] commit-msg.sample
| |-- [1000 ] fsmonitor-watchman.sample
| |-- [1000 ] post-update.sample
| |-- [1000 ] pre-applypatch.sample
| |-- [1000 ] pre-commit.sample
| |-- [1000 ] pre-push.sample
| |-- [1000 ] pre-rebase.sample
| |-- [1000 ] pre-receive.sample
| |-- [1000 ] prepare-commit-msg.sample
| `-- [1000 ] update.sample
|-- [1000 ] info
| `-- [1000 ] exclude
|-- [1000 ] objects
| |-- [1000 ] info
| `-- [1000 ] pack
| `-- [1000 ] eeeeeeee
`-- [1000 ] refs
|-- [1000 ] heads
`-- [1000 ] tags
11 directories, 17 files
FYI: someone else got it to work in Docker, see #51
Hi there, here is a sample of working Dockerfile
:
FROM alpine:3.13
ARG YQ_VERSION=4.7.1
ARG SCM_FILE=scm-backup-1.5.3.7c34699.zip
ARG DOTNET_FILE=dotnet-runtime-3.1.14-linux-musl-x64.tar.gz
ARG DOTNET_FILE_SHA512=77856f6183ef7ee561fe8510e89148566972ae21e3915242ae4ac7ef987b1aa78cda09bb06fdae96cd03758975dc5eb0e8652dea79b96db327ac5de2a4d41961
ENV SCM_ROOT=/opt/scm-backup
ENV DOTNET_ROOT=/opt/dotnet
ENV PATH=${PATH}:${DOTNET_ROOT}
WORKDIR ${SCM_ROOT}
RUN apk add --no-cache \
# Install deps
bash icu-libs krb5-libs \
libgcc libintl libssl1.1 \
libstdc++ zlib curl git \
# Install dotnet runtime
&& curl -fsL -o ${DOTNET_FILE} https://download.visualstudio.microsoft.com/download/pr/e7f88aae-a10e-4b49-8ccd-31f2df82cf3f/35fb27716a3b7543ef2889f0e4c19949/${DOTNET_FILE} \
&& echo "${DOTNET_FILE_SHA512} ${DOTNET_FILE}" > ${DOTNET_FILE}.sha512 \
&& sha512sum -c ${DOTNET_FILE}.sha512 \
&& mkdir -p ${DOTNET_ROOT} \
&& tar zxf ${DOTNET_FILE} -C ${DOTNET_ROOT} \
# Install scm-backup
&& curl -fsL -o ${SCM_FILE} https://github.com/christianspecht/scm-backup/releases/download/1.5.3/${SCM_FILE} \
&& unzip ${SCM_FILE} \
&& rm -f ${SCM_FILE} ${DOTNET_FILE} ${DOTNET_FILE}.sha512 \
# Install yq (config templating)
&& curl -fSL https://github.com/mikefarah/yq/releases/download/v${YQ_VERSION}/yq_linux_amd64 -o /usr/local/bin/yq \
&& chmod +x /usr/local/bin/yq
COPY entrypoint.sh /usr/local/bin
COPY settings.yml .
RUN chmod +x /usr/local/bin/entrypoint.sh
ENTRYPOINT ["entrypoint.sh"]
Very raw entrypoint.sh
for Dockerfile with variable replacement:
#!/usr/bin/env bash
yq e -i '(.sources[] | select(.title == "bitbucket_backup").name) = env(BITBUCKET_TEAM)' settings.yml
yq e -i '(.sources[] | select(.title == "bitbucket_backup").authName) = env(BITBUCKET_USER)' settings.yml
yq e -i '(.sources[] | select(.title == "bitbucket_backup").password) = env(BITBUCKET_PASS)' settings.yml
exec dotnet ScmBackup.dll
Sample settings.yml
template with bitbucket config:
localFolder: 'backup'
sources:
- title: bitbucket_backup
hoster: bitbucket
type: org
name: team
authName: username
password: app_password
Just create all 3 files in a single directory and run docker build . -t scm-backup
Then you can use a container and mount your target backup repo to /opt/scm-backup/backup
E.g.
docker build . -t scm-backup
docker run -v ${PWD}:/opt/scm-backup/backup -e BITBUCKET_USER=mybbusername -e BITBUCKET_PASS=mybbpassword -e BITBUCKET_TEAM=mybbteam scm-backup
Closing this for now.
I just added a new section to the documentation which lists all Dockerfiles / images that I know of: https://docs.scm-backup.org/en/latest/install/#docker
Hello here
I'm trying to get up and running backups on the linux via docker but stumbled upon this issue:
I don't see what is wrong with permissions because if I do
docker exec
in this container I can easily create file:Also,
git clone
inside container works fine as expectedMy Dockerfile:
Use: