helm install gitea with nodeport can not clone with ssh

[willzhang]

Description

Install gitea with helm use nodeport method, then clone with ssh error with Permission denied

root@ubuntu:~# git clone git@192.168.72.50:30022:gitea_admin/abc.git
Cloning into 'abc'...
git@192.168.72.50's password: 
Permission denied, please try again.
git@192.168.72.50's password: 

and

root@ubuntu:~# ll /root/ |grep .ssh
drwx------  2 root root    80 Nov  5 00:46 .ssh/
root@ubuntu:~# ll /root/.ssh/
total 24
drwx------  2 root root   80 Nov  5 00:46 ./
drwx------ 16 root root 4096 Nov  5 01:14 ../
-rw-r--r--  1 root root  566 Sep 11 13:29 authorized_keys
-rw-------  1 root root 2590 Aug 17 20:50 id_rsa
-rw-r--r--  1 root root  565 Aug 17 20:50 id_rsa.pub
-rw-r--r--  1 root root 3553 Oct 12 19:44 known_hosts

and

root@ubuntu:~# ssh -vT git@192.168.72.50
OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to 192.168.72.50 [192.168.72.50] port 22.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3
debug1: match: OpenSSH_8.9p1 Ubuntu-3 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.72.50:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:ulBoEXxQwsa6rPTSokn47rn5S5H/cfa5Tc4UsqWBch0
debug1: Host '192.168.72.50' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:4
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:s2F4RcfDKQQbDNVofWsH4wF9DQ62q53aGGcdmdWp6a4
debug1: Will attempt key: /root/.ssh/id_dsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /root/.ssh/id_ed25519 
debug1: Will attempt key: /root/.ssh/id_ed25519_sk 
debug1: Will attempt key: /root/.ssh/id_xmss 
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: kex_input_ext_info: publickey-hostbound@openssh.com (unrecognised)
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA SHA256:s2F4RcfDKQQbDNVofWsH4wF9DQ62q53aGGcdmdWp6a4
debug1: Authentications that can continue: publickey,password
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ecdsa_sk
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: Trying private key: /root/.ssh/id_ed25519_sk
debug1: Trying private key: /root/.ssh/id_xmss
debug1: Next authentication method: password
git@192.168.72.50's password: 

Gitea Version

v1.17.3

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

root@ubuntu:~# git version git version 2.25.1

Operating System

ubuntu 20.04

How are you running Gitea?

helm install gitea gitea/gitea -n gitea --create-namespace \
  --set service.http.type=NodePort \
  --set service.ssh.type=NodePort \
  --set gitea.config.server.ROOT_URL=http://192.168.72.50:30033 \
  --set gitea.config.server.SSH_DOMAIN=192.168.72.50:30022 \
  --set service.http.nodePort=30033 \
  --set service.ssh.nodePort=30022

Database

PostgreSQL

[techknowlogick]

when cloning via git on a port that is not 22, git doesn't accept :30022 as the port, it sees it as a path. Could you try something like git clone ssh://192.168.72.50:30022/gitea_admin/abc.git? And also the ssh debugging command was missing the port, you could add -p 3022 to it to connect to the expected port.

Please report back if these help at all, and if they do I can provide some suggestions on how to update the clone URL in the GUI to be able to have the above already set.

[willzhang]

1、i have set --set gitea.admin.password=gitea_admin so i input password gitea_admin, still can not clone with ssh

root@node1-01:/tmp# git clone ssh@192.168.0.22:30022:gitea_admin/test.git
Cloning into 'test'...
SSH warring: Authorized users only. All activity may be monitored and reported 
ssh@192.168.0.22's password: 
Permission denied, please try again.
ssh@192.168.0.22's password: 

2、before run this , i add /root/.ssh/id_rsa.pub to gitea UI

root@node-01:~# ssh -vT git@192.168.0.22 -p 30022
OpenSSH_8.2p1 Ubuntu-4ubuntu0.4, OpenSSL 1.1.1f  31 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to 192.168.0.22 [192.168.0.22] port 30022.
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.0
debug1: match: OpenSSH_9.0 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.0.22:30022 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:8Rws9Opkbr6vW1Gu1QS7/4O+XAuH6BqEkc4iTYBG4Ac
debug1: Host '[192.168.0.22]:30022' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:3
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /root/.ssh/id_rsa RSA SHA256:hfiTtjNYyOdHjXU4/p2g8pWClJN0UKzIwDV5SEAuCm8
debug1: Will attempt key: /root/.ssh/id_dsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa 
debug1: Will attempt key: /root/.ssh/id_ecdsa_sk 
debug1: Will attempt key: /root/.ssh/id_ed25519 
debug1: Will attempt key: /root/.ssh/id_ed25519_sk 
debug1: Will attempt key: /root/.ssh/id_xmss 
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519@openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256@openssh.com,webauthn-sk-ecdsa-sha2-nistp256@openssh.com>
debug1: kex_input_ext_info: publickey-hostbound@openssh.com (unrecognised)
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: /root/.ssh/id_rsa RSA SHA256:hfiTtjNYyOdHjXU4/p2g8pWClJN0UKzIwDV5SEAuCm8
debug1: Server accepts key: /root/.ssh/id_rsa RSA SHA256:hfiTtjNYyOdHjXU4/p2g8pWClJN0UKzIwDV5SEAuCm8
debug1: Authentication succeeded (publickey).
Authenticated to 192.168.0.22 ([192.168.0.22]:30022).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions@openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00@openssh.com want_reply 0
debug1: Remote: /data/git/.ssh/authorized_keys:2: key options: command
debug1: Remote: /data/git/.ssh/authorized_keys:2: key options: command
debug1: Sending environment.
debug1: Sending env LANG = en_US.UTF-8
Hi there, gitea_admin! You've successfully authenticated with the key named test, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow@openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 3232, received 3664 bytes, in 0.2 seconds
Bytes per second: sent 17824.4, received 20206.9
debug1: Exit status 0

[theAkito]

No idea, where you hacked that weird URL together.

This is how an SSH URL looks like with NodePort from Gitea.

ssh://git@192.168.72.50:30022/gitea_admin/abc.git

Is git the real user in your scenario? It is configurable.

Also, make sure you have the correct access rights to the repository.

You need to add an SSH key to the gitea_admin profile on the Gitea Web UI.

You shall not fiddle with some user's .ssh folder on the host.

[lunny]

Is the problem still there? Since the information has been returned, it should be OK.

Hi there, gitea_admin! You've successfully authenticated with the key named test, but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.

[teatreeoilchocolate]

I, too, have a similar issue. ALthough I have not installed it using helm. fetching/pushing/pulling work via https:// but using ssh-keys it fails citing permission-issues.

I have repeatedly toggled keys and verified them to no avail.

GIT_SSH_COMMAND="ssh -v -p 32022" git fetch lists the proper key being used. ssh -vvvT -p 32022 url stops with

debug1: No more authentication methods to try.
git@url: Permission denied (publickey).

I also fiddled around with the SSH ports for listening etc. Same difference.

Here is the setup portion pertaining to the gitea image

apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: giteastateful
  labels:
    run: giteastateful
spec:
  replicas: 1
  serviceName: "gitea-service"
  selector:
    matchLabels:
      run: giteastateful
  template:
    metadata:
      labels:
        run: giteastateful
    spec:
      volumes:
        - name: nfsgiteavol
          persistentVolumeClaim:
            claimName: nfsgiteacl
      imagePullSecrets:
        - name: regcred
      containers:
        - name: gitea
          image: registry.ser.ver/gitea:1.19
          imagePullPolicy: IfNotPresent
          env:
            - name: DB_TYPE
              value: "mysql"
            - name: DB_HOST
              value: "database.url:3306"
            - name: DB_NAME
              value: "gitea"
            - name: DB_USER
              value: "gitea"
            - name: DB_PASSWD
              value: "gitea"
            - name: RUN_MODE
              value: "prod"
            - name: DOMAIN
              value: "ser.ver"
            - name: SSH_DOMAIN
              value: "ser.ver"
            - name: ROOT_URL
              value: "https://ser.ver"
            - name: DISABLE_REGISTRATION
              value: "true"
            - name: INSTALL_LOCK
              value: "true"
            - name: DISABLE_GRAVATAR
              value: "true"
            - name: REQUIRE_SIGNIN_VIEW
              value: "true"
            - name: DEFAULT_THEME
              value: "arc-green"
            - name: SHOW_FOOTER_BRANDING
              value: "false"
            - name: SHOW_FOOTER_VERSION
              value: "false"
            - name: START_SSH_SERVER
              value: "true"
            - name: SHOW_FOOTER_TEMPLATE_LOAD_TIME
              value: "false"
            - name: SSH_PORT
              value: "32022"
            - name: SSH_LISTEN_PORT
              value: "32022"
            - name: ENABLE_USER_HEATMAP
              value: "false"
            - name: ALLOW_ONLY_EXTERNAL_REGISTRATION
              value: "false"
          ports:
            - containerPort: 3000
            - containerPort: 32022
          volumeMounts:
            - mountPath: /data
              name: nfsgiteavol
              subPath: gitea
      dnsConfig:
        searches:
          - default.svc.cluster.local
          - svc.cluster.local
          - cluster.local
---
apiVersion: v1
kind: Service
metadata:
  name: giteastateful-ssh-service
  labels:
    name: giteastateful-ssh-service
spec:
  selector:
    run: giteastateful
  ports:
  - protocol: TCP
    nodePort: 32022
    targetPort: 32022
    port: 32022
    name: ssh
  type: NodePort

[theAkito]

I shall emphasize, that SSH via NodePort on Helm installed Gitea always worked for me, since I started using it a couple of minor versions back. Therefore, it is strongly suggested, that there is some user error going on here.

Here is the setup portion pertaining to the gitea image

@teatreeoilchocolate

Could you please post your complete values.yml after scrambling & obfuscating credentials? Additionally providing the native Gitea configuration file would also be great.

If you would do that, I would compare that to my settings & tell you notable differences. 🙂

ALthough I have not installed it using helm.

Is there a specific, related reason for that?

Is the YAML accumulation you posted all you manually imported into the Cluster?

ssh -vvvT -p 32022 url stops with

Could you please post a larger tail of the log?

Which encryption algorithm do you use for the SSH keys?

If you are using RSA, do you have something like this in your /etc/ssh/sshd_config?

HostKeyAlgorithms +ssh-rsa
PubkeyAcceptedAlgorithms +ssh-rsa

[teatreeoilchocolate]

There is no values.yml. All the values the container uses are listed in the definition above.

The configurations in /etc/ssh/sshd_config do not make sense for this scenario IMHO. and my client's configuration never gave me grief to begin with and I use ssh a lot.

I use the ssh-server provided by the container which is the one for 1.19(I only use a private registry because of the pull limit of docker's).

Please find the complete log for the ssh-command attached.

The reason I am not using HELM is because it is a moving part I do not need. I also find that, with my infrastructure-setup, I do not want to second guess what HELM assumes. I write those files once, when they are in a working order I seldomly touch them other than to provide a new version. kustomize is awesome.

ssh.log

[theAkito]

The reason I am not using HELM is because it is a moving part I do not need.

I'd say, as long as you run into these issues, you should not think, that you don't need it. 😉

Helm is similar to a package manager. You get an all in one thing - you just have to configure it to your needs, as you sometimes need to with specific software.

Skipping that part means you need to do manual work, which has already been accomplished by third parties. Doing this work manually often may lead into issues. Exhibit A - what we are talking about right now. I use Helm and SSH always worked for me.

The configurations in /etc/ssh/sshd_config do not make sense for this scenario IMHO. and my client's configuration never gave me grief to begin with and I use ssh a lot.

Did you actually check what encryption algorithms you use or did you only wing this assumption?

I also find that, with my infrastructure-setup, I do not want to second guess what HELM assumes.

There are no assumptions, except with edge cases, as with YAML parsing failures, like getting a number out of a string or vice versa. But these edge cases are never truly related to Helm itself. It's mostly rooted in problems with YAML & Go Templating, which makes life pretty hard sometimes. So, you can still encounter the exact same edge cases in your manually wung YAMLs.

Additionally, ripping out single definitions & templates from a Helm Chart creates tons more assumptions than just relying on the Helm Chart to work properly. Ripping out one such definition is akin to ripping out a trouser pocket. Yes, you have only the pocket now, which is lightweight compared to the whole leg cover situation, but now you need to stitch all the loose ends manually together & make it work somehow. You are on your own now, as is evident in this issue scenario.

I write those files once, when they are in a working order I seldomly touch them other than to provide a new version. kustomize is awesome.

It doesn't work that way. Right now, I am in the process of migrating a lot of YAML definitions, because of tons of deprecations in Kubernetes definition specifications. With Helm, this is taken care of by the Chart maintainer.

---

I've looked at your SSH log & it means, that either RSA keys are not accepted or some other common SSH issue takes place. For information on what your log may mean, the following resources might prove helpful.

• https://stackoverflow.com/questions/68584977/github-ssh-key-not-accepted-getting-ssh-packet-type-51
• https://superuser.com/questions/1627406/permission-denied-publickey-with-error-message-receive-packet-type-51
• https://superuser.com/questions/1137438/ssh-key-authentication-fails/1145465
• https://unix.stackexchange.com/questions/709054/ssh-publickey-authentication-failure-receive-packet-type-51-sshd-is-not-accep

These are all specifically related to what you experience. The problem is, that a lot of reasons could be the case for the issue.

I would recommend checking out the log of the sshd server you are trying to authenticate with.

Make sure, you put the following into its /etc/ssh/sshd_config beforehand.

LogLevel DEBUG3

[teatreeoilchocolate]

I have since sidestepped the issue by switching to another application and migrating my repositories over. Thank you for your time.

[theAkito]

This issue should now be closed with the conclusion, that one shouldn't fiddle around with complete installations, if one lacks the knowledge or willpower to get through with the consequences of that decision.

Especially, if one is deploying the installation on a Kubernetes cluster, the Helm installation should be used, without dismantling & disabling it first.

Just use Helm on Kubernetes and all is fine.

[teatreeoilchocolate]

This issue should now be closed with the conclusion, that one shouldn't fiddle around with complete installations, if one lacks the knowledge or willpower to get through with the consequences of that decision.

Especially, if one is deploying the installation on a Kubernetes cluster, the Helm installation should be used, without dismantling & disabling it first.

Just use Helm on Kubernetes and all is fine.

I came here for help and to improve a project I found nice, should there be a problem. Especially since the proposed solutions did not work.

I have been running gitea for some time and other applications on my cluster without a problem(without helm) until I updated(other projects do not suffer from that). This thread is not the first dealing in SSH troubles with gitea.

https://discourse.gitea.io/t/solved-using-regular-git-ssh-not-working/286 https://stackoverflow.com/questions/69660928/gitea-ssh-refuses-to-authenticate-for-one-particular-pc-in-the-network https://stackoverflow.com/questions/69688288/gitea-built-in-ssh-server-not-starting-when-sshd-server-running https://devops.stackexchange.com/questions/10772/why-cant-i-connect-to-gitea-using-ssh-via-command-line https://github.com/go-gitea/gitea/issues/5497 (So there is clearly a history of these problems(the last link is 5 years old))

I solely meant to notify you and others reading this of my decision and approach to this problem, lest they wonder how I solved this for myself.

And what do I get? Instead of acknowledgement of my choice and moving on, I get more self-assertion of a person who (A) does not know me (B) does not understand the reasons of why I chose to not follow, what is widely regarded as 'best practice'. Trying to explain the world and helm to me. (C) apparently does not want to (D) thinks it necessary to go AD HOMINI saying as much as me being feeble and lacking in knowledge although this person does not know me.

BUT..... Thank you for that @theAkito . Thank you for showing your true colors and the attitude you reserve for those daring to think different from you and the need for self assertion at the cost of others. I will now be very weary of projects you participated in as I now know what to expect of you and that it might be an ill omen for projects you participated in potentially showing the culture/community they are either willing to grow/accept around their projects.

Good Day to you .

[lunny]

Maybe we should move this to https://gitea.com/gitea/helm-chart/ if it's a helm chart related problem.