khuedoan / homelab

Fully automated homelab from empty disk to running services with a single command.
https://homelab.khuedoan.com
GNU General Public License v3.0
8.1k stars 742 forks source link

Recreate "Platform" apps #125

Closed sabhishek closed 10 months ago

sabhishek commented 10 months ago

Is there a way to recreate the apps under "Platform"? There is no specific Make target for it and the install script doesn't seem to be idempotent either. I deleted gitea NS and was trying to restore it before attempting a cluster rebuild.

Thank you

khuedoan commented 10 months ago

Which command did you used to delete Gitea? Here's the bootstrap process:

Even if you deleted the gitea Application, the ApplicationSet will recreate it automatically.

sabhishek commented 10 months ago

gitea was intermittently degraded on my setup, I ended up kubectl delete namespace gitea , thinking exactly what you mentioned. Make bootstrap however, doesn't seem to recreate gitea NS.

homelab/homelab]# make bootstrap
make -C bootstrap
make[1]: Entering directory '/home/homelab/homelab/bootstrap'
kubectl create namespace argocd --dry-run=client --output=yaml \
    | kubectl apply -f -
namespace/argocd configured
cd argocd && ./apply.sh
NAME            CLASS   HOSTS              ADDRESS         PORTS     AGE
argocd-server   nginx   argocd.REDACTED.com   192.168.1.224   80, 443   53d
--
--
customresourcedefinition.apiextensions.k8s.io/applicationsets.argoproj.io condition met
cd root && ./apply.sh
Error from server (NotFound): namespaces "gitea" not found
applicationset.argoproj.io/bootstrap unchanged
applicationset.argoproj.io/system unchanged
applicationset.argoproj.io/platform unchanged
applicationset.argoproj.io/apps unchanged
make[1]: Leaving directory '/home/homelab/homelab/bootstrap'
[nix-shell:/homelab/homelab]#
khuedoan commented 10 months ago

Yeah I just included make bootstrap to explain the bootstrap process, but you don't need to run that again. Here's a few places to look:

I would recommend deleting the Application instead of deleting the namespace though, ArgoCD may try to reconcile the state while the namespace is being deleted.

sabhishek commented 10 months ago

Thanks for your comment.

  1. Namespace gitea has been deleted and I see no dangling resources
  2. [nix-shell:/home/homelab/homelab]# kubectl describe applicationset platform -n argocd
    
    Name:         platform
    Namespace:    argocd
    Labels:       argocd.argoproj.io/instance=root
    Annotations:  <none>
    --
    --
    Status:
    Conditions:
    Last Transition Time:  2023-11-22T23:14:26Z
    Message:               error while checking out repo: Error during fetching repo: `git fetch origin master --tags --force --prune` failed exit status 128: fatal: unable to access 'https://github.com/sabhishek/homelab/': Could not resolve host: github.com
    Reason:                ApplicationGenerationFromParamsError
    Status:                True
    Type:                  ErrorOccurred
There was a DNS issue in my homelab which I have resolved now. 

3.  [nix-shell:/home/homelab/homelab]# kubectl describe apps gitea -n argocd

Name: gitea Namespace: argocd Labels: Annotations: API Version: argoproj.io/v1alpha1 Kind: Application

-- Status: Conditions: Last Transition Time: 2023-11-27T00:35:14Z Message: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup argocd-repo-server on 10.43.0.10:53: read udp 10.42.0.116:43886->10.43.0.10:53: read: connection refused" Type: ComparisonError Last Transition Time: 2023-11-22T23:46:08Z Message: dial tcp: lookup argocd-redis: i/o timeout Type: DeletionError

-- Operation State: Finished At: 2023-11-04T01:09:23Z Message: ComparisonError: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing dial tcp: lookup argocd-repo-server on 10.43.0.10:53: read udp 10.42.0.116:34434->10.43.0.10:53: i/o timeout" (retried 10 times). Operation: --

Warning StatusRefreshed 2m14s argocd-application-controller Unable to delete application resources: dial tcp: lookup argocd-redis: i/o timeout Warning StatusRefreshed 2m13s argocd-application-controller Unable to delete application resources: dial tcp: lookup argocd-redis: i/o timeout Warning StatusRefreshed 2m3s argocd-application-controller Unable to delete application resources: dial tcp: lookup argocd-redis: i/o timeout Warning StatusRefreshed 2m3s argocd-application-controller Unable to delete application resources: dial tcp: lookup argocd-redis: i/o timeout



I have attempted to delete gitea app from the ArgoCD UI, it seems to be stuck :/
khuedoan commented 10 months ago

lookup argocd-repo-server on 10.43.0.10:53: read udp 10.42.0.116:43886->10.43.0.10:53: read: connection refused lookup argocd-redis: i/o timeout

Are your argocd-repo-server, argocd-redis and coredns pods still healthy?

sabhishek commented 10 months ago

Sorry for the late comment . Coredns wasn't healthy, I fixed the host dns issue and it was fixed as well. A minor problem, The applications got synced from GitHub and all my homelab apps are *.khuedon.com . I should be able to fix it. Thanks for your help. Let me know if I could buy you a coffee .

khuedoan commented 10 months ago

Thank you for the offer, a list of sponsoring options can be found in the project sidebar under the "Sponsor this Project" section 😄