Closed cmoulliard closed 11 months ago
Love the idea!
Another huge win we get with this is removing the need to build / bake resources in to a gitserver - if we set up keys or some other type of write access we could directly push the resources to a repo in gitea.
@cmoulliard I am unfamiliar with gitea, can we automate the repo creation and the import of ssh keys? Can we allow for anonymous write operations so we don't need keys/auth?
I am unfamiliar with gitea, can we automate the repo creation
There is an API to create such repository: https://gittea.dev/api/swagger#/repository/createCurrentUserRepo and import ssh key: https://gittea.dev/api/swagger#/admin/adminCreatePublicKey
And several SDK exist like this one for java developers: https://github.com/zeripath/java-gitea-api or go: https://gitea.com/gitea/go-sdk
I am thinking for the case of the idpbuilder, the choice of the git server will also be another hard dependency alongside Argo CD and Argo Workflows that needs to be installed out of bound, right? Also, that this would be only a dependency for the idpbuilder, since in case of another reference implementation, we will use GitHub / GitLab or other repos.
I am thinking for the case of the idpbuilder, the choice of the git server will also be another hard dependency alongside Argo CD and Argo Workflows that needs to be installed out of bound, right?
It depends how we plan to grab the resources in fact.
Example: backstage
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: idpbuilder-local-gitserver-backstage
namespace: argocd
spec:
destination:
namespace: argocd
server: https://kubernetes.default.svc
project: idpbuilder-local-gitserver
source:
path: backstage
repoURL: http://gitserver-embedded.idpbuilder-local.svc/idpbuilder-resources.git
targetRevision: HEAD
syncPolicy:
automated:
selfHeal: true
syncOptions:
- CreateNamespace=true
Instead of doing that, we could also deploy the cnoe backstage (or ingress-nginx - see: #34) as ArgoCD Applications using a Helm chart + values and then it is not needed to use a local git server. This is also what we do to install crossplane as we deploy it using a Helm chart source
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: crossplane-helm
namespace: argocd
spec:
destination:
namespace: crossplane-system
server: "https://kubernetes.default.svc"
source:
chart: crossplane
repoURL: https://charts.crossplane.io/stable
targetRevision: 1.11.1
helm:
releaseName: crossplane
2 approaches currently exists (and perhaps more) to support a more agile platform to deploy the idp core components such as: argocd, argo workflows, vault and cert manager:
@nimakaviani I agree! Theres two ways to think about this gitserver IMO. Right now its simply a way to feed packages in to argo but eventually we will need to consider workflows which include the source control system as part of what comprises a 'developer platform'. e.g., to support automation with argo and PR/Change Requests for promotion and validation workflows.
Use cases aside, I think that there is no reason why not to use gitea if it is better/healthier project than the lib we are currently relying on. If you want to create a PR that just refactors the current functionality on top of gitea @cmoulliard I think that would be a fine place to start. Then we will at least have it locally to work with as a replacement of the existing depedency for idpbuilder and we can decide if we want it to be a dependency for the other use cases as well.
If you want to create a PR that just refactors the current functionality on top of gitea
Such a PR already exists - #39 @jessesanford
Here are the commands to be used post gitea project deployed; a repository, import ssh key and add users
Remark: It is apparently not needed to use a token as Basic auth should be enough. So I replaced -H 'Authorization: token $TOKEN'
with -u "gitea_admin:gitea_admin"
@greghaynes @jessesanford @nabuskey
GITEA_API_URL="https://gitea.localtest.me/api/"
GITEA_USERNAME="gitea_admin"
GITEA_PASSWORD="gitea_admin"
TOKEN=$(curl -s -k -H "Content-Type: application/json" -d '{"name":"init","scopes": ["write:user", "write:admin", "write:repository"]}' -u $GITEA_USERNAME:$GITEA_PASSWORD $GITEA_API_URL/v1/users/gitea_admin/tokens | jq -r .sha1)
echo $TOKEN
## Create a user
curl -k -X 'POST' \
"$GITEA_API_URL/v1/admin/users" \
-H 'accept: application/json' \
-u "gitea_admin:gitea_admin" \
-H 'Content-Type: application/json' \
-d '{
"email": "user1@cnoe.io",
"full_name": "user1",
"login_name": "user1",
"must_change_password": false,
"password": "user11234",
"restricted": false,
"visibility": "public",
"send_notify": false,
"username": "user1"
}'
## Import key
curl -k -X POST\
"$GITEA_API_URL/v1/user/keys" \
-H 'accept: application/json' \
-u "gitea_admin:gitea_admin" \
-H 'Content-Type: application/json' \
-d '{
"id": 1, // This is the ID of the gitea_admin account
"key": "ssh-rsa AAAAB3Nz...KBE1ecj7WSL9",
"read_only": true,
"title": "ch007m@gmail.com"
}'
## Create a repository
curl -v -k -X 'POST' \
"$GITEA_API_URL/v1/user/repos" \
-H 'accept: application/json' \
-u "gitea_admin:gitea_admin" \
-H 'Content-Type: application/json' \
-d '{
"default_branch": "main",
"description": "dummy",
"name": "dummy",
"private": false,
"readme": "dummy"
}'
I cannot ssh to the gitea repo when gitea is deployed on idpbuilder using exactly the same config as mentioned within this ticket and where I also expose on kind the mapping of the port 22:32222
as I got
git remote add origin git@localhost:gitea_admin/backstage-catalog-entities.git
git push -u origin main
kex_exchange_identification: read: Connection reset by peer
Connection reset by ::1 port 22
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
or
ssh -v git@gitea.localtest.me -p 22
OpenSSH_9.0p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/cmoullia/.ssh/config
debug1: Reading configuration data /Users/cmoullia/.orbstack/ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to gitea.localtest.me port 22.
debug1: Connection established.
debug1: identity file /Users/cmoullia/.ssh/id_rsa type 0
debug1: identity file /Users/cmoullia/.ssh/id_rsa-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_ecdsa type -1
debug1: identity file /Users/cmoullia/.ssh/id_ecdsa-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_ecdsa_sk type -1
debug1: identity file /Users/cmoullia/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_ed25519 type -1
debug1: identity file /Users/cmoullia/.ssh/id_ed25519-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_ed25519_sk type -1
debug1: identity file /Users/cmoullia/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_xmss type -1
debug1: identity file /Users/cmoullia/.ssh/id_xmss-cert type -1
debug1: identity file /Users/cmoullia/.ssh/id_dsa type -1
debug1: identity file /Users/cmoullia/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.0
kex_exchange_identification: read: Connection reset by peer
Connection reset by ::1 port 22
I fixed the issue after setting (like for the ingress nginx deployment) the nodeSelector and Tolerations
nodeSelector:
#"kubernetes.io/hostname": "local-control-plane"
ingress-ready: "true"
tolerations:
- effect: NoSchedule
key: node-role.kubernetes.io/master
operator: Equal
- effect: NoSchedule
key: node-role.kubernetes.io/control-plane
operator: Equal
Suggestion
In order to simplify the development, maintenance of idpbuilder, I would like to suggest to use gitea as embedded git server.
Why ?
HowTo
To play locally with gitea:
persistence: enabled: false
gitea: admin: username: "gitea_admin" password: "gitea_admin" email: "gi@tea.com" config: database: DB_TYPE: sqlite3 session: PROVIDER: memory cache: ADAPTER: memory queue: TYPE: level
service: ssh: type: NodePort nodePort: 32222 externalTrafficPolicy: Local
ingress: enabled: true className: nginx hosts:
user/password to be used to logon
gitea_admin/gitea_admin
on http://gitea.localtest.me Repo to be created: "http://gitea.localtest.me/gitea_admin/dummy" Import your public key "http://gitea.localtest.me/user/settings"Create a local git repo + files -> commit
Add the remote git repo
git push -u origin main
:-)
@greghaynes