Open 3eka opened 2 years ago
Example:
:; n3dr backup --n3drRepo my-yum --n3drUser predrag_zecevic --n3drPass *** --n3drURL ${REPO1} \
--directory-prefix /backup/tmp/n3dr
...
:; cd /backup/tmp/n3dr
:; n3dr upload --n3drRepo my-yum --n3drUser predrag_zecevic --n3drPass *** --n3drURL ${REPO2}
...
INFO[0000]/pz/n3dr/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'maven2'
FATA[0000]/pz/n3dr/cmd/n3dr/upload.go:23 main.glob..func8() no folders with .pom files detected. Please check whether the 'my-yum' directory contains .pom files
:; :; n3dr upload --n3drRepo my-yum --n3drUser predrag_zecevic --n3drPass *** --n3drURL ${REPO2} --artifactType yum
...
INFO[0000]/pz/n3dr/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'yum'
FATA[0000]/pz/n3dr/cmd/n3dr/upload.go:23 main.glob..func8() Upload of 'yum' is not supported
Regards.
And, just noticed (when backing up all repositories, from Nexus OSS 3.34.1-01):
:; n3dr repositories --backup --n3drURL http://host:port/nexus --https false --n3drUser ... --n3drPass ...
...
Nexus repository: 'composer_external', format: 'composer' download not supported
Nexus repository: 'test-raw', format: 'raw' download not supported
I have used repositories instead of repositoriesV2, because second one fails:
host:port
for --n3drURL, but I have host:port/nexus
path.How to fix that situation (for second Nexus instance it works -- I do not have path after host:port specification)?
Regards.
P.S: It would be nice to have more detailed explanation for each option (--help give just minimum).
Thank you for addressing this. We will have to fix the following:
host:port/nexus
should be allowed as well @3eka Could you try 6.4.3 in conjunction with --basePathPrefix?
Tested. Seems there is other problem:
:; n3dr repositoriesV2 --backup --n3drURL FQDN:8081 --https false --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr --basePathPrefix=nexus
...
INFO[0000]/pz/n3dr/internal/pkg/connection/connection.go:22 github.com/030/n3dr/internal/pkg/connection.(*Nexus3).Client() adding 'nexus' as a prefix to the basePath
FATA[0000]/pz/n3dr/cmd/n3dr/repositoriesV2.go:42 main.glob..func7() cannot get repository names: 'Get "https://FQDN:8081/nexus/service/rest/v1/repositorySettings": http: server gave HTTP response to HTTPS client'
It tries https (although not specified with --https false
), but accepts --basePathPrefix
Regards.
Utility was built from git clone this morning, and returns no version:
:; n3dr --version
n3dr version local
Regards.
The version is only added by the CI. When I download one of the artifacts the the version is 6.4.3:
n3dr --version
n3dr version refs/tags/6.4.3
I have created 2 nexus nodes, both listening on https, 8443 port:
Did backup (to empty directory):
:; n3dr repositoriesV2 --backup --n3drURL OLD:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr
and that resulted in:
:; du -shc /nobackup/tmp/n3dr/my*
24K /nobackup/tmp/n3dr/my-docker
31K /nobackup/tmp/n3dr/my-raw
1,6G /nobackup/tmp/n3dr/my-yum
1,6G total
NOTES*
Upload is still not working (tried mentioned 3 repositories only):
:; n3dr repositoriesV2 --upload --n3drURL NEW:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr
...
INFO[0000]/pz/n3dr/internal/artifactsv2/upload/upload.go:369 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'NEW:8443' repository: 'my-raw'...
INFO[0000]/pz/n3dr/internal/artifactsv2/upload/upload.go:369 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'NEW:8443' repository: 'my-docker'...
INFO[0000]/pz/n3dr/internal/artifactsv2/upload/upload.go:369 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'NEW:8443' repository: 'my-yum'...
...
Only my-raw was uploaded, my-yum and my-docker, were not (repository on NEW node was created exactly as one on NEW node):
:; du -shc /nobackup/tmp/n3dr/my-yum/
1,6G /nobackup/tmp/n3dr/my-yum/
1,6G total
Regards.
The version is only added by the CI. When I download one of the artifacts the the version is 6.4.3:
n3dr --version n3dr version refs/tags/6.4.3
Hmm,
:; aria2c https://github.com/030/n3dr/archive/refs/tags/6.4.3.tar.gz
:; tar xf n3dr-6.4.3.tar.gz
:; cd n3dr-6.4.3
:; bash scripts/build.sh
:; cmd/n3dr/n3dr --version
n3dr version local
What I am doing wrong?
@3eka The binary has already been created by the CI:
curl -L https://github.com/030/n3dr/releases/download/6.4.3/n3dr-ubuntu-20.04 -o n3dr && chmod +x n3dr && ./n3dr --version
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 656 100 656 0 0 8000 0 --:--:-- --:--:-- --:--:-- 8000
100 22.5M 100 22.5M 0 0 22.7M 0 --:--:-- --:--:-- --:--:-- 32.1M
n3dr version refs/tags/6.4.3
In order to assign a version locally:
GITHUB_TAG=some-version ./scripts/build.sh && ./cmd/n3dr/n3dr --version
GITHUB_TAG: 'some-version' N3DR_DELIVERABLE: 'n3dr'
n3dr version some-version
@3eka 6.5.0 supports rpm/yum uploads.
Thanks, will test it today. What about problem described here: https://github.com/030/n3dr/issues/238#issuecomment-1009701830 ? Tried, and got same error as described. Regards.
HI @030, something is still NOT OK.
Case 1) backup of everything from OLD nexus instance misses docker repository backup (same problem as described in https://github.com/030/n3dr/issues/238#issuecomment-1009738435). So, complete backup is missing.
Case 2) backup of docker repository (only) works, but not upload (previously an EMPTY docker repository was created)::
:; n3dr backup --n3drRepo my-docker --n3drURL https://OLDFQDN:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr
...
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/backup.go:34 github.com/030/n3dr/internal/artifacts.TempDownloadDir() Download dir name: '/nobackup/tmp/n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/backup.go:29 main.glob..func1() Processing repository: my-docker
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/backup.go:281 github.com/030/n3dr/internal/artifacts.Nexus3.StoreArtifactsOnDiskChannel() Backing up: 'my-docker'
:; du -shc /nobackup/tmp/n3dr/my-docker
575M /nobackup/tmp/n3dr/my-docker
575M total
:; n3dr repositoriesV2 --upload --n3drURL NEWFQDN:8443 --n3drUser predrag_zecevic --n3drPass ${N3DRPASS} --directory-prefix /nobackup/tmp/n3dr
...
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:71 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:159 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/pz/n3dr-6.5.0/internal/artifactsv2/upload/upload.go:372 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'NEWFQDN:8443' repository: 'my-docker'...
Although it throws no error, nothing was uploaded.
Tested also yum repository backup/upload, also no success (but error is shown):
; n3dr backup --n3drRepo my-yum --n3drURL https://OLDFQDN:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr 1>/dev/null
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:71 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:159 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/backup.go:34 github.com/030/n3dr/internal/artifacts.TempDownloadDir() Download dir name: '/nobackup/tmp/n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/backup.go:29 main.glob..func1() Processing repository: my-yum
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/backup.go:281 github.com/030/n3dr/internal/artifacts.Nexus3.StoreArtifactsOnDiskChannel() Backing up: 'my-yum'
:; du -shc /nobackup/tmp/n3dr/my-yum
1,6G /nobackup/tmp/n3dr/my-yum
1,6G total
:; n3dr repositoriesV2 --upload --n3drURL NEWFQDN:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr 1>/dev/null
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:71 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:159 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/pz/n3dr-6.5.0/internal/artifactsv2/upload/upload.go:372 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'NEWFQDN:8443' repository: 'my-yum'...
FATA[0015]/pz/n3dr-6.5.0/cmd/n3dr/repositoriesV2.go:48 main.glob..func7() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/repodata/dfad0846e198d2b4c08b7462a5657563db3babb3d48345cc3f64ef3299883698-other.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
This upload option is also not working (same error shown when docker repository upload is attempted):
:; n3dr upload --n3drRepo my-yum --n3drURL https://NEWFQDN:8443 --n3drUser ... --n3drPass ... --directory-prefix /nobackup/tmp/n3dr 1>/dev/null
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:71 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:159 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'maven2'
FATA[0000]/pz/n3dr-6.5.0/cmd/n3dr/upload.go:23 main.glob..func9() lstat my-yum: no such file or directory
:; cd /nobackup/tmp/n3dr
:; n3dr upload --n3drRepo my-yum -t=yum --n3drURL https://NEWFQDN:8443 --n3drUser ... --n3drPass ...
...
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:71 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/pz/n3dr-6.5.0/cmd/n3dr/root.go:159 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/pz/n3dr-6.5.0/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'yum'
FATA[0000]/pz/n3dr-6.5.0/cmd/n3dr/upload.go:23 main.glob..func9() Upload of 'yum' is not supported
Am I missing something?
Regards.
Maybe I can ask for --n3drLogo false
switch, to not display logo over whole screen?
Hi @030 ,
how to prevent display of logo?
:; n3dr --help | grep -i logo
--showLogo show N3DR logo or not. Default: true (default true)
Whatever I have tried, it has failed: --showLogo false
also does not work.
BTW: I will try version 6.6.1 tomorrow.
Could you try --showLogo=false
? Example: https://github.com/030/n3dr/blob/main/test/integration-tests.sh#L514
Thanks
First, that works (--showLogo=false
). Does it means that I have to use = between option and its value for everything?
A) backup works:
:; rm -rf /nobackup/tmp/n3dr/*
:; for repo in my-docker my-yum my-raw; do
n3dr backup --showLogo=false --n3drRepo=${repo} --anonymous --n3drURL ${NEXUS_O} --directory-prefix /nobackup/tmp/n3dr
done
:; cd /nobackup/tmp/n3dr
:; du -shc *
3,8G my-docker
110K my-raw
5,8G my-yum
B) load of raw repository work, other not (${NEXUS_N} has configured hosted docker, raw and yum EMPTY repositories. Nexus is available directly, e.g. no /blahblah in its URL):
:; n3dr upload --showLogo=false --artifactType=docker --n3drURL=${NEXUS_N} --n3drUser=predrag_zecevic --n3drPass=*** --n3drRepo my-docker
INFO[0000]/n3dr/cmd/n3dr/root.go:76 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/n3dr/cmd/n3dr/root.go:166 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/n3dr/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'docker'
FATA[0000]/n3dr/cmd/n3dr/upload.go:23 main.glob..func9() Upload of 'docker' is not supported
:; n3dr upload --showLogo=false --artifactType=yum --n3drURL=${NEXUS_N} --n3drUser=predrag_zecevic --n3drPass=*** --n3drRepo my-yum
INFO[0000]/n3dr/cmd/n3dr/root.go:76 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/n3dr/cmd/n3dr/root.go:166 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/n3dr/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'yum'
FATA[0000]/n3dr/cmd/n3dr/upload.go:23 main.glob..func9() Upload of 'yum' is not supported
Regards.
P.S: I used this to compile tool:
N3DR_V=6.6.1
wget --continue https://github.com/030/n3dr/archive/refs/tags/${N3DR_V}.tar.gz
tar xf ${N3DR_V}.tar.gz && cd n3dr-${N3DR_V}
GITHUB_TAG=${N3DR_V} bash scripts/build.sh
Still, it does not show version:
:; go version
go version go1.17.9 illumos/amd64
:; n3dr --version
n3dr version local
HI! Any news on this? I have compiled n3dr few days ago, and I am still unable to upload yum and docker repositories, taken from one nexus instance to this new one. Both instances OLD and NEW are running same Nexus s/w (3.39.0-01) and both have configured repositories. I am able to do backup (example for yum repository -- docker one shows same problem):
:; n3dr backup --n3drRepo=yum --anonymous --n3drURL=https://old:8443 --directory-prefix=/nobackup/tmp/n3dr --showLogo=false
:; cd /nobackup/tmp/n3dr
:; du -sh yum
7,0G yum/
Tying to upload:
:; n3dr upload --showLogo=false --n3drURL=https://new:8443 --n3drUser=... --n3drPass=... --artifactType=yum --n3drRepo=yum --directory-prefix=/nobackup/tmp/n3dr
INFO[0000]/n3dr/cmd/n3dr/root.go:76 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/n3dr/cmd/n3dr/root.go:166 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/n3dr/internal/artifacts/upload.go:364 github.com/030/n3dr/internal/artifacts.Nexus3.Upload() Uploading 'yum'
FATA[0000]/n3dr/cmd/n3dr/upload.go:23 main.glob..func9() Upload of 'yum' is not supported
What is proper syntax to upload? Am I doing something wrong? Regards.
P.S: Docs page does not reflect output from n3dr --help
command.
Did you try repositoriesV2 as well?
Yes, not working either:
/nobackup/tmp/n3dr :; du -sh *
5,7G my-docker
110K my-raw
7,0G my-yum
and this is "import":
:; cd /nobackup/tmp/n3dr
:; n3dr repositoriesV2 --upload --showLogo=false --n3drURL=nexus-node.mydomain.com:8443 --n3drUser=predrag_zecevic --n3drPass=*** --directory-prefix=/nobackup/tmp/n3dr
INFO[0000]/n3dr/cmd/n3dr/root.go:76 main.n3drHiddenHome() n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000]/n3dr/cmd/n3dr/root.go:166 main.parseConfig() Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000]/n3dr/internal/artifactsv2/upload/upload.go:62 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).reposOnDisk() found the following localDiskRepos: '[my-yum my-docker my-raw]'
INFO[0000]/n3dr/internal/artifactsv2/upload/upload.go:320 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'nexus-node.mydomain.com:8443' repository: 'my-raw'...
INFO[0000]/n3dr/internal/artifactsv2/upload/upload.go:320 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'nexus-node.mydomain.com:8443' repository: 'my-yum'...
INFO[0000]/n3dr/internal/artifactsv2/upload/upload.go:320 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() Uploading files to Nexus: 'nexus-node.mydomain.com:8443' repository: 'my-docker'...
INFO[0000]/n3dr/internal/artifactsv2/upload/upload.go:80 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).repoFormatLocalDiskRepo() format of repo: 'my-raw' is: 'raw'
INFO[0001]/n3dr/internal/artifactsv2/upload/upload.go:80 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).repoFormatLocalDiskRepo() format of repo: 'my-docker' is: 'docker'
WARN[0001]/n3dr/internal/artifactsv2/upload/upload.go:326 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).Upload.func1() docker and snapshot repositories skipped
INFO[0001]/n3dr/internal/artifactsv2/upload/upload.go:80 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).repoFormatLocalDiskRepo() format of repo: 'my-yum' is: 'yum'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/SRPMS/repodata/repomd.xml', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/SRPMS/maven-3.8.4-1.src.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/repodata/ff3cdbcf25f89c6c2a526d2a40d62919ff17274f307e744b8124ae61b5fb1824-filelists.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/repodata/f1e0ab9b1546d344db515e18fc56ebe927874cb99f62673bbc91841838b60322-primary.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/8/x86_64/RPMS/sudosh2-1.0.7-1.2e.x86_64.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/8/x86_64/RPMS/cronolog-debuginfo-1.7.2-1.x86_64.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/cronolog-1.7.2-1.x86_64.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/repodata/f511f663d3b08d7bcbc64f37d163d19cea154c574804d8bb0fe8fc04b2556b4d-other.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/cronolog-debuginfo-1.7.2-1.x86_64.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/SRPMS/repodata/d7bb659c1322c3e8fb37d02fb6e984e52d8e27355db6b5c11e4044e220aa8ed9-other.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/SRPMS/repodata/8dc2749e3926c726bc0a8cbdf219e0a96ae927dbf15c90b04ea6fbe15c99d430-primary.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/8/x86_64/RPMS/repodata/ea45fe64ff71036028d9e2a92f5c3d0e4cdcd73768fdfed900fac1e89675253d-primary.xml.gz', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/repodata/repomd.xml', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/7/x86_64/SRPMS/repodata/repomd.xml', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
ERRO[0001]/n3dr/internal/artifactsv2/upload/upload.go:283 github.com/030/n3dr/internal/artifactsv2/upload.(*Nexus3).ReadLocalDirAndUploadArtifacts.func1.1() cannot upload component: '/nobackup/tmp/n3dr/my-yum/centos/6/x86_64/RPMS/php-devel-7.3.24-1.ssl1.1.1h.x86_64.rpm', error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec (status 400): {}'
...
it recognizes type of data, but load fails. When I open https://nexus-node.mydomain.com:8443 I am able to log-in, browse, etc
So, what exactly means error: 'response status code does not match any response statuses defined for this endpoint in the swagger spec error?
Regards
After switching to
:; n3dr --version
n3dr version 6.7.3 # logo file changed
:; go version
go version go1.19.1 illumos/amd64
Backup started to fail (nexus node listens to https port 8443):
:; n3dr repositoriesV2 --backup --directory-prefix /nobackup/tmp/n3dr/nexus --n3drUser=predrag_zecevic --n3drPass=*** --n3drURL=nexus.my-domain.com:8443
INFO[0000] n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000] Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
INFO[0000] false
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'my-yum', 'hosted', yum
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'nuget-group', 'group', nuget
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'maven-snapshots', 'hosted', maven2
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'maven-central', 'proxy', maven2
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'nuget.org-proxy', 'proxy', nuget
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'maven-releases', 'hosted', maven2
INFO[0000]/n3dr/internal/artifactsv2/download.go:164 github.com/030/n3dr/internal/artifactsv2.(*Nexus3).Backup() backing up 'my-docker', 'hosted', docker
FATA[0000]/n3dr/cmd/n3dr/repositoriesV2.go:42 main.glob..func7() parse ":0/v2/_catalog": missing protocol scheme
I guess it is stuck on my-docker
repository (from some reason... was working before -- does go version has something to do with?), because we see under output directory '/nobackup/tmp/n3dr/nexus':
my-yum
maven-central
maven-releases
maven-snapshots
nuget.org-proxy
With --debug switch, it outputs only this line (real password is replaced with ***; I connect with that account w/o problems to web interface):
[0000]/export/home/predrag_zecevic/go/pkg/mod/github.com/030/p2iwd@v0.2.0/internal/pkg/http/http.go:20 github.com/030/p2iwd/internal/pkg/http.(*Auth).RequestAndResponse() Auth: '&{Accept GET *** :0/v2/_catalog predrag_zecevic}'
I really like idea behind this project, but I am kind stuck (wanted to do nexus backup/restore with n3dr).
Best regards.
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
I will add soon new tests result...
Testing (latest?) version shows same symptoms as already described here: https://github.com/030/n3dr/issues/238#issuecomment-1262076140 (download of docker repo fails/terminates program).
:; n3dr --version
n3dr version 7.0.2
@3eka I get the same issue: FATA[0000]/home/runner/work/n3dr/n3dr/cmd/n3dr/repositoriesV2.go:89 main.glob..func5() parse ":0/v2/_catalog": missing protocol scheme
when I have omitted the --dockerPort 9001
and --dockerHost http://localhost
. Could you check whether you have set these parameters? I will update the code to ensure that an error will be returned if the docker host and or port have not been set.
@3eka I get the same issue:
FATA[0000]/home/runner/work/n3dr/n3dr/cmd/n3dr/repositoriesV2.go:89 main.glob..func5() parse ":0/v2/_catalog": missing protocol scheme
when I have omitted the--dockerPort 9001
and--dockerHost http://localhost
. Could you check whether you have set these parameters? I will update the code to ensure that an error will be returned if the docker host and or port have not been set.
Hi @030 ;
Yes, it seems that adding docker options have had a progress here...
Question regarding upload: shall I use also docker switches for upload/restore?
Thanks for fixing this. Regards.
Also, usage of n3dr repositoriesV2 --backup options is inconsistent:
--n3drURL=mynexus.mydomain.com:PORT
(no protocol)
but
--dockerHost=PROTOCOL://mynexus.mydomain.com --dockerPort=PORT
(specifies protocol)
NOTE, there is also --https
switch available
And:
--logLevel string change the log level (default: info, options: trace, debug, info, warn, error or none) (default "info")
--showLogo show N3DR logo or not. Default: true (default true)
Default value shown twice
Then: I have specified:
--logFile=/nobackup/n3dr/n3dr_backup.log
--logLevel=info
But still some output is written to stdout, for example:
INFO[0000] n3drHomeDir: '/export/home/predrag_zecevic/.n3dr'
INFO[0000] Using config file: '/export/home/predrag_zecevic/.n3dr/config.yml'
time="2023-01-04T16:09:50+01:00" level=info msg="backing up 'my-yum', 'hosted', yum" func="github.com/030/n3dr/internal/app/n3dr/artifactsv2.(*Nexus3).Backup" file="/n3dr/internal/app/n3dr/artifactsv2/download.go:233"
...
##%%%%%%%%%%#%############################################################
Which is kind annoying
@3eka I get the same issue:
FATA[0000]/home/runner/work/n3dr/n3dr/cmd/n3dr/repositoriesV2.go:89 main.glob..func5() parse ":0/v2/_catalog": missing protocol scheme
when I have omitted the--dockerPort 9001
and--dockerHost http://localhost
. Could you check whether you have set these parameters? I will update the code to ensure that an error will be returned if the docker host and or port have not been set.
Hi @030 ,
I have added docker options, but still my docker repository is NOT backed-up, e.g. my-docker
directory is not there.
Any other suggestions? Regards.
Nope (if you meant under --directory-prefix
directory?).
Arghh, I just figured out that I need that utility for backing-up docker images too... Right?
Anyway, will download it, build and test... if that is correct thinking
NOTE: added p2iwd to the PATH and issued backup command again. No luck
@3eka no need to add p2iwd to the path. If --directory-prefix
is used then several folders should be created: one for every repository and there should be a p2iwd folder that includes the docker repositories.
I'm having this same issue with docker images not being downloaded. There is no my-docker
or p2iwd
directory when doing a backup of the entire nexus.
When i do a backup of a specific docker repository I get a my-docker
folder with just the manifests, no images.
@kcathey and @3eka Could you try the docker-quickstart and compare it with the parameters you have used?
So I discovered that Nexus doesn't expose the images that are proxied, only those that have be explicitly pushed. I was trying to do an export of my proxied images, to avoid explicitly pushing them. When there are only proxied images in Nexus the /v2/_catalog endpoint returns an empty list, as soon as an image is pushed it returns a list with the pushed image.
So for my part this is no longer a problem and in either case it isn't yours, but seems to be on the Nexus end.
Hi @kcathey,
can you elaborate "as soon as an image is pushed" task? We are uploading docker images to the nexus, after those are built on jenkins. That is not pushing, right?
Thanks for checking this. Regards.
P.S: Seems that we need to change way of uploading images (with pushing)
I do a docker push <nexus tag>
to upload to nexus. As far as I know that's the only way to upload a docker image.
What would you like to be added: So, I can backup my docker hosted repository - that works fine. BUT, it seems it is not possible to upload it back to nexus (check table on start page). Same is true for many other types.
Why is this needed: Well, if I can backup (for example) docker hosted repository, I would also like to be able to upload it somewhere else (another nexus instance)
If I missing it (case exists), please explain. I have read documentation (several times), and could not find options. Thanks