Helm Charts Added

[blacktig3r]

As per our discussion from PR #348 I have created PR for helm-charts.

@venkatamutyala could you please review and approve it? Thanks, Blacktig3r

[venkatamutyala]

Do we want this chart to be stateless? I don't see a PVC definition. Am i missing something?

[blacktig3r]

@venkatamutyala you are right...There should have been statefulset. I just added in recent commits. Please check.

root@YuvrajP-ub:~# kubectl get statefulset`
kubectNAME                  READY   AGE
onetimesecret-redis   3/3     2m38s

root@YuvrajP-ub:~# kubectl get pvc
NAME                                  STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
redis-storage-onetimesecret-redis-0   Bound    pvc-397bacd1-c524-4024-bfbb-eb41c5478b4f   1Gi        RWO            standard-rwo   2m47s
redis-storage-onetimesecret-redis-1   Bound    pvc-b298b6bd-6a4a-4535-ab95-3d136704ff9e   1Gi        RWO            standard-rwo   2m28s
redis-storage-onetimesecret-redis-2   Bound    pvc-706d5991-1a59-4d16-b0f5-2053c3b2f37d   1Gi        RWO            standard-rwo   2m10s

root@YuvrajP-ub:~# kubectl get pods
NAME                                           READY   STATUS    RESTARTS   AGE
nginx-1-6c5f6b76cf-l95rx                       1/1     Running   0          6d19h
nginx-1-6c5f6b76cf-spzcl                       1/1     Running   0          6d19h
nginx-1-6c5f6b76cf-t4b7m                       1/1     Running   0          6d19h
onetimesecret-onetimesecret-85b8bb77bb-5f8h5   1/1     Running   0          3m11s
onetimesecret-redis-0                          1/1     Running   0          3m11s
onetimesecret-redis-1                          1/1     Running   0          2m52s
onetimesecret-redis-2                          1/1     Running   0          2m33s
onetimesecret-redis-7cb5d478d-9g27r            1/1     Running   0          3m11s

`

[venkatamutyala]

Have you tested this chart on a clean install? I see you have a redis deployment and a statefulset. Shouldn't they both be statefulsets or both be deployments? have you considered using a redis helm chart as a sub chart?

[blacktig3r]

Now I have added redis as a dependency chart and removed unwanted files. Also I have tested chart on clean install. Pods are running fine but onetimesecret's pod restart till redis pod become fully available.

NAME                                           READY   STATUS    RESTARTS      AGE
onetimesecret-onetimesecret-55f79fff5f-v92vr   1/1     Running   3 (42s ago)   67s
onetimesecret-redis-master-0                   1/1     Running   0             66s

[delano]

Great work on this folks. Looks like it's close to being ready.

Once we're happy let's update the readme and we can get it merged in.

[blacktig3r]

sure @delano. @venkatamutyala let me know if anything needs to be updated

[venkatamutyala]

I'll give this a test tomorrow. Some immediate thoughts i had were whether or not we should use a bitnami helm chart or another vendor. I know bitnami is owned by VMWare and that Broadcom recently acquired VMWare. So basically i'm not sure what the future holds for that helm chart.

ArgoCD (another popular FOSS kubernetes project) uses this redis-ha as a sub chart:

https://github.com/argoproj/argo-helm/blob/main/charts/argo-cd/Chart.yaml#L20-L23

There are a few other things that we should probably add like:

• the ability to use an existing ingress controller + annotation support for it.
• pinning to versions/shas for all the apps/images we use.

[blacktig3r]

I have added a ingress configuration and updated the deployment templates to pin specific versions of the images used. and About bitnami redis image, I think Broadcom's acquisition shouldn't affect the ongoing opensource projects. But if you want I can test redis-ha.

[venkatamutyala]

@blacktig3r can you confirm what tag you are using in your test deployment? I'm getting a crashbackoff loop. I found this in my logs:

[Sat Jun  1 21:07:58 UTC 2024] INFO: Running entrypoint.sh...
Fetching gem metadata from https://rubygems.org/.
Resolving dependencies...
Installing bundler-graph 0.2.1
Bundle complete! 24 Gemfile dependencies, 58 gems now installed.
Use `bundle info [gemname]` to see where a bundled gem is installed.
/app/lib/onetime.rb:9: warning: syslog was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add syslog to your Gemfile or gemspec.
/usr/local/bundle/gems/sendgrid-ruby-6.7.0/lib/sendgrid/helpers/eventwebhook/eventwebhook.rb:1: warning: base64 was loaded from the standard library, but will no longer be part of the default gems since Ruby 3.4.0. Add base64 to your Gemfile or gemspec. Also contact author of sendgrid-ruby-6.7.0 to add base64 into its gemspec.

My ruby experience is a bit limited so nothing jumps out at me as a real error. But after those logs it appears to be an exit code 1. I'm wondering if the latest is just not a great tag/image and i should be using something older.

here is my podspec from argocd.

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: '2024-06-01T21:01:50Z'
  generateName: onetimesecret-onetimesecret-d9bfd9c96-
  labels:
    app: onetimesecret
    pod-template-hash: d9bfd9c96
  name: onetimesecret-onetimesecret-d9bfd9c96-sl2g5
  namespace: glueops-core-onetimesecret
  ownerReferences:
    - apiVersion: apps/v1
      blockOwnerDeletion: true
      controller: true
      kind: ReplicaSet
      name: onetimesecret-onetimesecret-d9bfd9c96
      uid: 16875db9-24a8-43ad-a2b7-d333f9455b58
  resourceVersion: '2674812'
  uid: aa396d65-4a74-4a80-8b43-6eaeeba00372
spec:
  containers:
    - env:
        - name: REDIS_URL
          value: 'redis://onetimesecret-redis-master.default.svc.cluster.local:6379/0'
        - name: COLONEL
          value: admin@example.com
        - name: HOST
          value: 'onetimesecret-onetimesecret.default.svc.cluster.local:3000'
        - name: SSL
          value: 'false'
      image: 'ghcr.io/onetimesecret/onetimesecret:latest'
      imagePullPolicy: Always
      name: onetimesecret
      ports:
        - containerPort: 3000
          protocol: TCP
      resources: {}
      terminationMessagePath: /dev/termination-log
      terminationMessagePolicy: File
      volumeMounts:
        - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
          name: kube-api-access-7crmp
          readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  nodeName: k3d-captain-server-0
  preemptionPolicy: PreemptLowerPriority
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext: {}
  serviceAccount: default
  serviceAccountName: default
  terminationGracePeriodSeconds: 30
  tolerations:
    - effect: NoExecute
      key: node.kubernetes.io/not-ready
      operator: Exists
      tolerationSeconds: 300
    - effect: NoExecute
      key: node.kubernetes.io/unreachable
      operator: Exists
      tolerationSeconds: 300
  volumes:
    - name: kube-api-access-7crmp
      projected:
        defaultMode: 420
        sources:
          - serviceAccountToken:
              expirationSeconds: 3607
              path: token
          - configMap:
              items:
                - key: ca.crt
                  path: ca.crt
              name: kube-root-ca.crt
          - downwardAPI:
              items:
                - fieldRef:
                    apiVersion: v1
                    fieldPath: metadata.namespace
                  path: namespace
status:
  conditions:
    - lastProbeTime: null
      lastTransitionTime: '2024-06-01T21:01:50Z'
      status: 'True'
      type: Initialized
    - lastProbeTime: null
      lastTransitionTime: '2024-06-02T02:04:57Z'
      message: 'containers with unready status: [onetimesecret]'
      reason: ContainersNotReady
      status: 'False'
      type: Ready
    - lastProbeTime: null
      lastTransitionTime: '2024-06-02T02:04:57Z'
      message: 'containers with unready status: [onetimesecret]'
      reason: ContainersNotReady
      status: 'False'
      type: ContainersReady
    - lastProbeTime: null
      lastTransitionTime: '2024-06-01T21:01:50Z'
      status: 'True'
      type: PodScheduled
  containerStatuses:
    - containerID: >-
        containerd://994ec6f5faf6730d4832fe064020ed287af99492eb372d8670e6ebe20ea5d0dc
      image: 'docker.io/onetimesecret/onetimesecret:latest'
      imageID: >-
        docker.io/onetimesecret/onetimesecret@sha256:6be8872c82ffb005987742fd6849536e719bae885c4e0ba89475d885938bccb8
      lastState:
        terminated:
          containerID: >-
            containerd://994ec6f5faf6730d4832fe064020ed287af99492eb372d8670e6ebe20ea5d0dc
          exitCode: 1
          finishedAt: '2024-06-02T02:04:56Z'
          reason: Error
          startedAt: '2024-06-02T02:04:50Z'
      name: onetimesecret
      ready: false
      restartCount: 63
      started: false
      state:
        waiting:
          message: >-
            back-off 5m0s restarting failed container=onetimesecret
            pod=onetimesecret-onetimesecret-d9bfd9c96-sl2g5_glueops-core-onetimesecret(aa396d65-4a74-4a80-8b43-6eaeeba00372)
          reason: CrashLoopBackOff
  hostIP: 172.21.0.2
  phase: Running
  podIP: 10.42.6.12
  podIPs:
    - ip: 10.42.6.12
  qosClass: BestEffort
  startTime: '2024-06-01T21:01:50Z'

[blacktig3r]

It restarts until redis become available, once redis pod is up onetimesecret's pod also become stable. For me it took max 3 restarts(max 45 second)to become stable. I was checking if there is option to delay the launch of onetimesecret's pod until redis become available but no success.

[venkatamutyala]

Huh. It's still erroring on my end. Let me take a look at my configs. It's possible my config for redis needs to be updated. I'm currently using the defaults.

[venkatamutyala]

I think i just figured it out. I'll post my solution shortly. Need to step away for the next couple of hours.

[venkatamutyala]

Left a few more comments just now. As we are getting towards the end, do you have a preferred place that we should publish this chart? I think we can do an OCI artifact with github packages/ghcr or we could do github pages too.

[blacktig3r]

Hey @venkatamutyala , I have added changed which you suggested and they are working on my end. I have added SMTP support as an example form so I commented those in files. Can you please take a look once? and About publishing this chart, I have experience with github pages only...didn't worked on oci artifcat.

[venkatamutyala]

Hey @venkatamutyala , I have added changed which you suggested and they are working on my end. I have added SMTP support as an example form so I commented those in files. Can you please take a look once? and About publishing this chart, I have experience with github pages only...didn't worked on oci artifcat.

Just left you a quick comment around this. I'll do my best to reply within 48 hours but give me 72 hours.

Also, I really do appreciate all the work you are doing on this.

[blacktig3r]

Hey @venkatamutyala , I have added changed which you suggested and they are working on my end. I have added SMTP support as an example form so I commented those in files. Can you please take a look once? and About publishing this chart, I have experience with github pages only...didn't worked on oci artifcat.

Just left you a quick comment around this. I'll do my best to reply within 48 hours but give me 72 hours.

Also, I really do appreciate all the work you are doing on this.

Hi , Sorry, because of some emergency family work I was offline for two days. I'll try to take a look today.

[venkatamutyala]

no worries! I'm still researching the publishing side too

[blacktig3r]

Hi @venkatamutyala , I have added conditional approach for container port, REDIS_URL and SMTP_HOST. I'm not sure about the enabling/disabling for SMTP, I think it is not possible. and If needed I think we should ask user inputs for smtp username and password as well and other smtp params we can keep as default. Let me know your thoughts

[venkatamutyala]

What's your availability to pair over a live call and hash through some of this PR together live? Can you suggest a couple times/dates you are available?

[blacktig3r]

we can connect tomorrow(saturday) or Monday @8:30am Indian time zone.

[venkatamutyala]

Yes, let's plan for Monday @8:30AM IST.

Just sent you a google meet invite to your gmail (got it off your linkedin)

[blacktig3r]

@venkatamutyala I am having some trouble if I uses ClusterIP. I'll check in the morning. Also I'll dig more into redis part too. Thanks.

[blacktig3r]

ClusterIP is working now. @venkatamutyala I have updated the Redis's chart version so that we will have the latest image of Redis-7.2.5. Because like we discussed we have to consider the compatibility of Valkey. Let me know if I have missed any change and then we can move forward to publish it

[blacktig3r]

ClusterIP is working now. @venkatamutyala I have updated the Redis's chart version so that we will have the latest image of Redis-7.2.5. Because like we discussed we have to consider the compatibility of Valkey. Let me know if I have missed any change and then we can move forward to publish it

Also can we resolve our older "change" conversations If I have resolved those.

[venkatamutyala]

ClusterIP is working now. @venkatamutyala I have updated the Redis's chart version so that we will have the latest image of Redis-7.2.5. Because like we discussed we have to consider the compatibility of Valkey. Let me know if I have missed any change and then we can move forward to publish it

I think it's actually 7.2.4:

https://github.com/valkey-io/valkey/issues/43

7.2.5 from valkey appears to be their own flavor. Let me know if you found something that says otherwise.

[blacktig3r]

ClusterIP is working now. @venkatamutyala I have updated the Redis's chart version so that we will have the latest image of Redis-7.2.5. Because like we discussed we have to consider the compatibility of Valkey. Let me know if I have missed any change and then we can move forward to publish it

I think it's actually 7.2.4:

valkey-io/valkey#43

7.2.5 from valkey appears to be their own flavor. Let me know if you found something that says otherwise.

My bad..On valkey's official repo I saw the recent release was 7.2.5 that's why I was considering the same version for redis too. Corrected in latest commit

[blacktig3r]

Can you squash all your commits?

Done. I have squashed all of my commits

[venkatamutyala]

@blacktig3r are you able to merge on your own? If not, i can merge it in.

[blacktig3r]

I'm not able to merge. Getting "Only those with write access to this repository can merge pull requests."