ytti / oxidized

Oxidized is a network device configuration backup tool. It's a RANCID replacement!
Apache License 2.0
2.8k stars 927 forks source link

[githubrepo] - Different URL parsing in latest docker image? #2641

Closed phoenix0984 closed 2 years ago

phoenix0984 commented 2 years ago

Hi all, I recently updated to the latest docker image published 4 days ago and now pushing to our remote gitlab repo fails with:

INFO -- : Configuration updated for ITSS/slbhs-excvc1
INFO -- : GithubRepo: Pushing local repository(/root/.config/oxidized/gitlab/devConfigs.git/)...
INFO -- : GithubRepo: to remote: ssh://git@gitlab.***.com:8022/grp-oxidized/devconfigs.git
ERROR -- : Hook push_to_remote (#<GithubRepo:0x0000558c602675e0>) failed (#<Rugged::NetworkError: unsupported URL protocol>) for event :post_store

It looks like the remote URL can no longer be parsed with the non-default SSH port. The part of my config looks like that:

hooks:
  push_to_remote:
    type: githubrepo
    events: [post_store]
    remote_repo: ssh://git@gitlab.***.com:8022/grp-oxidized/devconfigs.git
    publickey: /root/.config/oxidized/.ssh/gitlab_rsa.pub
    privatekey: /root/.config/oxidized/.ssh/gitlab_rsa

Any help would be highly appreciated 👍

Best regards, Marcel

dpublic commented 2 years ago

I see the same problem but I don't use a port number. Can the Dockerfile be reverted for now and the docker image rebuilt?

phoenix0984 commented 2 years ago

I supsect that the bump to net-ssh 7.0.0.beta1 could be the problem. However I'm failing to run the docker image when reverting the commit and building the image locally:

git checkout -b revert-net-ssh-bump 6f96035142f97a822a146b6d18eb70cd2736756b
Switched to a new branch 'revert-net-ssh-bump'
docker build --no-cache -t gitlab-registry.***.com/org/dockerhub/oxidized:latest .

When redeploying the stack I see the following log entries:

*** Running /etc/my_init.d/00_regen_ssh_host_keys.sh...
*** Running /etc/my_init.d/10_syslog-ng.init...
Oct 21 11:39:19 e6a09a6d10e7 syslog-ng[14]: syslog-ng starting up; version='3.25.1'
*** Booting runit daemon...
*** Runit started as PID 21
runsv auto-reload-config: fatal: unable to start ./run: file does not exist
runsv update-ca-certificates: fatal: unable to start ./run: file does not exist
runsv oxidized: fatal: unable to start ./run: file does not exist
Oct 21 11:39:20 e6a09a6d10e7 cron[30]: (CRON) INFO (pidfile fd = 3)
Oct 21 11:39:20 e6a09a6d10e7 cron[30]: (CRON) INFO (Running @reboot jobs)
runsv auto-reload-config: fatal: unable to start ./run: file does not exist
runsv update-ca-certificates: fatal: unable to start ./run: file does not exist
runsv oxidized: fatal: unable to start ./run: file does not exist
runsv auto-reload-config: fatal: unable to start ./run: file does not exist
runsv update-ca-certificates: fatal: unable to start ./run: file does not exist
runsv oxidized: fatal: unable to start ./run: file does not exist

What am I missing here?

phoenix0984 commented 2 years ago

Forget about my last comment, that was a problem with my build environment on Windows. CRLF line endings... However, the mentioned patch was not the culprit. Switching to https based authentication for now.

If someone can pick this up, I would be thankful though :)

dpublic commented 2 years ago

Since there appears to be no quick fix for this, how about removing the "latest" tag in docker hub, pulling the previous docker image (by it's sha256 hash) from 8 weeks ago, tagging that image to "latest", and pushing that as the latest to get back to the last image that didn't have this problem?

I don't have a docker hub repo but this is what I see can be done after the current "latest" tag is removed:

docker pull oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a

docker tag oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a oxidized/oxidized:latest

docker push oxidized/oxidized:latest

phoenix0984 commented 2 years ago

Since there appears to be no quick fix for this, how about removing the "latest" tag in docker hub, pulling the previous docker image (by it's sha256 hash) from 8 weeks ago, tagging that image to "latest", and pushing that as the latest to get back to the last image that didn't have this problem?

I don't have a docker hub repo but this is what I see can be done after the current "latest" tag is removed:

docker pull oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a

docker tag oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a oxidized/oxidized:latest

docker push oxidized/oxidized:latest

Interesting, where did you get the digest for the previous image from? Why don't you just use that image directly in your config? No need to have the :latest tag... You also mentioned, that you are not using non-default ports, so you could just stick to default syntax, change remote_repo: ssh://git@gitlab.***.com:8022/grp-oxidized/devconfigs.git to remote_repo: git@gitlab.***.com:grp-oxidized/devconfigs.git

There should not be any problem with that, I suppose.

agrevtcev commented 2 years ago

It seems that building container from commit 6f96035142f97a822a146b6d18eb70cd2736756b is a quick fix. At least i could use remote_repo: ssh://git@ip.ip.ip.ip/some-repo/oxidized.git and it worked.

Br, Alex

dpublic commented 2 years ago

I got the digest from the previous working docker image because I was already working with it to set it up. I only noticed that the latest image changed when I brought it up on another system and it couldn't push to remote git repos. Even when you docker pull the latest, the older images are kept locally without tags and they show up on docker images. I just tagged it and pushed it to our internal docker repo to use instead of the public docker image.

My suggestion to retag the public oxidized docker image is so that someone doesn't come along and pulls the latest and it doesn't push to remote git repos and they conclude that oxidized doesn't work well. Oxidized definitely works well for us, but it can make for a good out of the docker box experience if the latest docker image is retagged.

agrevtcev commented 2 years ago

I can't check now - but it seems 9d67c0361bda2476ca117476eba8e97410a36f1f is to blame.

Br, Alex

YellowJacket51 commented 2 years ago

I am having the same issue. Any updates on how to fix this issue?

dpublic commented 2 years ago

Pull the last docker image that worked and tag it with a different name to use if the recent changes are not needed.

docker pull oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a

docker tag oxidized/oxidized@sha256:ba1e9e1fc2d1afac32810d207e01e4d96798697ce63ed447d0f30d43458cc21a oxidized/oxidized:{CHANGE THIS TO ANY TAG NAME YOU WANT}

YellowJacket51 commented 2 years ago

That fixed my issue. @dpublic thanks for your help.

Cheers!