Closed tmd313 closed 2 years ago
Hello!
Do you try to access Operator or ArangoDB endpoint? From API calls I see you to to reach ArangoDB, but endpoint and JWT token is related to Operator itself.
Once you install chart you get only Operator, after that you need to define ArangoDeployment: https://github.com/arangodb/kube-arangodb/blob/master/examples/production-cluster.yaml
It will create for you service production-cluster-ea
with LoadBalancer type and production-cluster-jwt
with JWT secret.
Best Regards, Adam.
Thank you sir. I will do so, I wasn’t sure about that and what I had so far. I’ll do that and inspect it. I’m assuming then I will get root user with empty password at that time, but more likely at either 8529 or 8530, from examples I’ve seen so far. And will that make a service for coordinators, or is that implicit now. Sorry, I’m asking more questions. I’ll do what you advise and take it from there.
Thanks so much,
Tim
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Adam Janikowski @.> Sent: Wednesday, April 21, 2021 5:23 PM To: arangodb/kube-arangodb @.> Cc: Dillon, Tim @.>; Author @.> Subject: Re: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
Hello! Do you try to access Operator or ArangoDB endpoint? From API calls I see you to to reach ArangoDB, but endpoint and JWT token is related to Operator itself. Once you install chart you get only External @.**@.>) Report This Emailhttps://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL3RpbS5kaWxsb25AY29tbXNjb3BlLmNvbS9lYjY1NWVjNzk4MzFmYTljYTQwMWRiMTJiMThhNjYyOC8xNjE5MDQwMTg1Ljg1#key=d33e94acc88a588be295dff2623d3f77 FAQhttps://www.inky.com/banner-faq/ Protection by INKYhttps://www.inky.com
Hello!
Do you try to access Operator or ArangoDB endpoint? From API calls I see you to to reach ArangoDB, but endpoint and JWT token is related to Operator itself.
Once you install chart you get only Operator, after that you need to define ArangoDeployment: https://github.com/arangodb/kube-arangodb/blob/master/examples/production-cluster.yamlhttps://secure-web.cisco.com/1ygwDpvm846vdYldhImSp91Tn1OKrWliFrFTPL6qh2s2h1E5xSoAKoLB3ZBY1PJzNkTMgAwNY5_Cy0uTSrh1KUywhyXPw70tlA8HG8XzW5SDampgvEvFf4gKKC5QheSrMaWqz1MfkAEQ7ZyH1_l-mJBh-gCz_fVqRu-qQ-Vy5vQ15UfJETS-VThqheUdmmmsb5FTLwMcTpW-xAYuAABk2jcI-ZtYuYv3LlRGgIS7LZ7a-CyXtYwa3-YuMaKmlCsKS8EcV1f9wwPmovOpZ-Oe3MVAlewpEQWfoFMOLfYlv2nRmon_UTIKjwkTv9TL2GiHJ/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fblob%2Fmaster%2Fexamples%2Fproduction-cluster.yaml
It will create for you service production-cluster-ea with LoadBalancer type and production-cluster-jwt with JWT secret.
Best Regards, Adam.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1xtiS8h1_YMVAoMleg63seRd8V9KcJDdB4oBjC44UEDoZ34a3LRaLBhLmcdtMKn6MwOehXXMV07wcnx19hvJu7g3FXvHv5L1gyXefy269CGjxqdKE-FzCY_P-qLPRkgy8RLFh9xtMnQWOb38XcLuSEjk749HHgSrFPB8GF35lv6cgQmNz6VI1niYf0pzpDcxmD463BGnHiilgbELFR0T9nSznvzmwXhO_Md8iUTQFKIbdrtHWU-2amysePi3gCPgJFI96hX76A4eE-DCa_8JQRx24leIobDtkLF8PTT61mKvo4ESqD7vJDwCPJRQ56vGT/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fissues%2F716%23issuecomment-824366433, or unsubscribehttps://secure-web.cisco.com/1XrxUWRLRbQZXBSdXMiKijOX7OKqUzLO60CKsLEYoi6v-RBKrb2MCdtnAB3WjEMzPSNGjk8iu9s-Mzcjbco5cGvnWcCfsT0p-uSfSo68s5GWrm7Udg-2S2R-aZJ-xd7zdNowJrN-rFHwH5fTluxMX3HEttJFPnPmqzIAvlZICf63-L_kFZSACK34RrIAaz9HNT2JDsaVDHQxymgHRVApodenr557wyIEo5O5rRywRz5xIvZZarOHKidIWEOK7IzxlqzmHdb9rC-YOWpFHfjsYj9X8pSClk1-tHfBOmdyj6_MwetBCSvmknY8QtAivKCKU/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHZ2IP6VNDIIUC24CP2UGV3TJ463NANCNFSM43LBOSDQ.
BTW, I was using 1.1.5 and came across a comment on the 1.1.5.1 that that was a bit more aligned with kubernetes version 1.19, we're using 1.19.7 currently. Should I move to the 1.1.5.1? Thanks, Tim
Hello!
Not needed, 1.17 is fully compatible with 1.19. We changed it only due to fact that one of our API users required this.
Best, Adam.
I ran this, using the following, the other day I downloaded the latest of this image from arangodb/arangodb:latest and moved it into our repo since my local site has a problem with docker hub such that we cannot get images from it.
cat production-cluster.yaml
apiVersion: "database.arangodb.com/v1"
kind: "ArangoDeployment"
metadata:
name: "production-cluster"
spec:
mode: Cluster
image: '
I get these artifacts in bold from kubectl apply -f production-cluster.yaml
All of the others were already there. I added the service production-cluster-ea2 in attempt to get a LoadBalancer type of service with an external-IP as our kubernetes cluster does not give them out automatically:
kubemaster1 arangodbCharts ]$ kubectl get pods,svc,secrets,deployments --all-namespaces -o wide|egrep arango
default pod/arango-kube-arangodb-1618862786-operator-6fd78fb5ff-97c8w 1/1 Running 4 6d20h 172.16.224.3 p12-tdillon-compute1.broadbus.com
BUT, I don’t see any new pods? Should there be some?
I still cannot connect to arangodb at :8529 port on any of the nodePorts or LB at external-IP, :8529. I do “see” the nodePort via “lsof -i :30630 on our compute node(s).
Is there a way to pass in an LB External-IP to the cluster?
Thanks
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Adam Janikowski @.> Sent: Wednesday, April 21, 2021 5:23 PM To: arangodb/kube-arangodb @.> Cc: Dillon, Tim @.>; Author @.> Subject: Re: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
Hello! Do you try to access Operator or ArangoDB endpoint? From API calls I see you to to reach ArangoDB, but endpoint and JWT token is related to Operator itself. Once you install chart you get only External @.**@.>) Report This Emailhttps://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL3RpbS5kaWxsb25AY29tbXNjb3BlLmNvbS9lYjY1NWVjNzk4MzFmYTljYTQwMWRiMTJiMThhNjYyOC8xNjE5MDQwMTg1Ljg1#key=d33e94acc88a588be295dff2623d3f77 FAQhttps://www.inky.com/banner-faq/ Protection by INKYhttps://www.inky.com
Hello!
Do you try to access Operator or ArangoDB endpoint? From API calls I see you to to reach ArangoDB, but endpoint and JWT token is related to Operator itself.
Once you install chart you get only Operator, after that you need to define ArangoDeployment: https://github.com/arangodb/kube-arangodb/blob/master/examples/production-cluster.yamlhttps://secure-web.cisco.com/1ygwDpvm846vdYldhImSp91Tn1OKrWliFrFTPL6qh2s2h1E5xSoAKoLB3ZBY1PJzNkTMgAwNY5_Cy0uTSrh1KUywhyXPw70tlA8HG8XzW5SDampgvEvFf4gKKC5QheSrMaWqz1MfkAEQ7ZyH1_l-mJBh-gCz_fVqRu-qQ-Vy5vQ15UfJETS-VThqheUdmmmsb5FTLwMcTpW-xAYuAABk2jcI-ZtYuYv3LlRGgIS7LZ7a-CyXtYwa3-YuMaKmlCsKS8EcV1f9wwPmovOpZ-Oe3MVAlewpEQWfoFMOLfYlv2nRmon_UTIKjwkTv9TL2GiHJ/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fblob%2Fmaster%2Fexamples%2Fproduction-cluster.yaml
It will create for you service production-cluster-ea with LoadBalancer type and production-cluster-jwt with JWT secret.
Best Regards, Adam.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1xtiS8h1_YMVAoMleg63seRd8V9KcJDdB4oBjC44UEDoZ34a3LRaLBhLmcdtMKn6MwOehXXMV07wcnx19hvJu7g3FXvHv5L1gyXefy269CGjxqdKE-FzCY_P-qLPRkgy8RLFh9xtMnQWOb38XcLuSEjk749HHgSrFPB8GF35lv6cgQmNz6VI1niYf0pzpDcxmD463BGnHiilgbELFR0T9nSznvzmwXhO_Md8iUTQFKIbdrtHWU-2amysePi3gCPgJFI96hX76A4eE-DCa_8JQRx24leIobDtkLF8PTT61mKvo4ESqD7vJDwCPJRQ56vGT/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fissues%2F716%23issuecomment-824366433, or unsubscribehttps://secure-web.cisco.com/1XrxUWRLRbQZXBSdXMiKijOX7OKqUzLO60CKsLEYoi6v-RBKrb2MCdtnAB3WjEMzPSNGjk8iu9s-Mzcjbco5cGvnWcCfsT0p-uSfSo68s5GWrm7Udg-2S2R-aZJ-xd7zdNowJrN-rFHwH5fTluxMX3HEttJFPnPmqzIAvlZICf63-L_kFZSACK34RrIAaz9HNT2JDsaVDHQxymgHRVApodenr557wyIEo5O5rRywRz5xIvZZarOHKidIWEOK7IzxlqzmHdb9rC-YOWpFHfjsYj9X8pSClk1-tHfBOmdyj6_MwetBCSvmknY8QtAivKCKU/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHZ2IP6VNDIIUC24CP2UGV3TJ463NANCNFSM43LBOSDQ.
Hello!
Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled.
About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment-resource.html#specexternalaccessloadbalancerip-string
Best Regards, Adam.
The attached log snippet from the leading operator shows the issues. The first one that cropped up was:
2021-04-26T16:13:36Z DBG Failed to fetch deployment shard sync state error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w 2021-04-26T16:13:36Z DBG Failed to fetch deployment health error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w
It then shows what appears to be attempting to bring up AGNTs (3), PRMRs (3) and CRDNs (3) but no live cluster.
Let me know if there’s anything else I can collect to help with the diagnosis.
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Adam Janikowski @.> Sent: Tuesday, April 27, 2021 4:33 AM To: arangodb/kube-arangodb @.> Cc: Dillon, Tim @.>; Author @.> Subject: Re: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
Hello! Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled. About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment External @.**@.>) Report This Emailhttps://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL3RpbS5kaWxsb25AY29tbXNjb3BlLmNvbS9kYTQ0NGM3MzkzZjA4NDNlMDA5MTBlYjZlNjMyZDRjMy8xNjE5NTEyMzkzLjA5#key=4ce51a4f8c0f1e1bc6d11f93317198e8 FAQhttps://www.inky.com/banner-faq/ Protection by INKYhttps://www.inky.com
Hello!
Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled.
About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment-resource.html#specexternalaccessloadbalancerip-stringhttps://secure-web.cisco.com/1QyzGAbDFQIocZIYPG_SeOK_C2gUrQV9RQC3QOKs9CervwFqBsOjjxawhoeDayNQSq75ClJF9NaB4BKjP9NKELzbHzRFk9xUjIhMfWGvbeVCYr1egQIXIeguj_RpjIvYUreXd8IgswpdnsBtHdonz1B7hmV3l3b89NRhwa277fhn-OCxcf1Wqe3NDs_y3j0OitPm6095h-eYGa6gpuzU_Z5EEKyzHufPR2cBqdr9OHzJmCfoCTKrKDw0FmmdTLDbEXUZPynbmFTrnO_S-ymqQhyLCCMO0BRViKcNUJ4R8e6JhUcUumj5wsjukb2rP43Za/https%3A%2F%2Fwww.arangodb.com%2Fdocs%2Fstable%2Fdeployment-kubernetes-deployment-resource.html%23specexternalaccessloadbalancerip-string
Best Regards, Adam.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1Guvybh1KSYYoxCb6bG_ldBNlh1g-e00M4bZdVkHSm5UzTTxGYoUBTHC7WrILNT_0VRipaP0tnPmQm3_b_JiI9th2M1UvEK5yRMVhYSbjBWIzeRfWwrUlXTYYP7IKzx42UTsd4tcZDN1MA64YH61QGwJ85OB46LPLn-lxXof7x57PKiSiwAEas1yPfztxieQEVXFosP0NOlYXWsR0Qv65SLeg3ZUQoe02NZgUNBlK7N6ltdFT7mzUx_dsRoP53LuNqh_Jwf_do4DirVU7T9w-DpwcDVa-TpURdLfX99NmRfIU5rzXwMmgByW_nwzRSiOA/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fissues%2F716%23issuecomment-827424199, or unsubscribehttps://secure-web.cisco.com/1fTdmEsMWItUbcgFIw92LiaAZSDwqsqG6uwvwfFl7Xo3_f_kyhdNKsdBp4tukf9ABSTzTx7tRYKgquijmUZ31QnaDQEEEVSY8FZNeZNidoqt56g7AKb6p5bJz5lEsFHepZt2NIgvVd5N8vWedtKr13VDtXofyz_Mpzf8YaqStTs4g7vD80YAOzf_hWRHF2VxtrRV_fNP68_vBB5wYTzVaCise_ZqyA-RN-n_ov2ePhTkFffnSKoR8bnXoBM1Wz8Z1_qfb2mNjmZUMgDZ25GkYEYhedsbU0W5DsVQy1VB0AoEkHuHi8jxdf2EaC00YqsMg/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHZ2IP7V5NIZLG36N6EN2DDTKZZEJANCNFSM43LBOSDQ.
So Adam, looks like I need to create a jwt secret (auth.jwtSecretName) and a cacert secret (tls.caSecretName) for the deployment. I could use some direction on what exactly to put in those. I can, of course, create the secrets but not sure on generating the content for them.
As for this:
spec.bootstrap.passwordSecretNames.root: string This setting specifies a secret name for the credentials of the root user. When a deployment is created the operator will setup the root user account according to the credentials given by the secret. If the secret doesn’t exist the operator creates a secret with a random password. There are two magic values for the secret name:
What are the credential field keys it will be looking for? Perhaps “username” and “password”? or maybe something less obvious, or is it implied that it is the root user and I only need a proper tag, aka key, for the base64 encrypted password?
Can you point me to any examples in the documentation, please?
I’m getting there now. The link you gave me yesterday has more info in it than the pages similar to it that I have come across. The expanded "example-arangodb-cluster" Yaml/”ArangoDeployment” spec is helpful.
Thanks, Tim
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Dillon, Tim Sent: Tuesday, April 27, 2021 9:47 AM To: arangodb/kube-arangodb @.>; arangodb/kube-arangodb @.> Cc: Author @.***> Subject: RE: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
The attached log snippet from the leading operator shows the issues. The first one that cropped up was:
2021-04-26T16:13:36Z DBG Failed to fetch deployment shard sync state error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w 2021-04-26T16:13:36Z DBG Failed to fetch deployment health error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w
It then shows what appears to be attempting to bring up AGNTs (3), PRMRs (3) and CRDNs (3) but no live cluster.
Let me know if there’s anything else I can collect to help with the diagnosis.
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Adam Janikowski @.**@.>> Sent: Tuesday, April 27, 2021 4:33 AM To: arangodb/kube-arangodb @.**@.>> Cc: Dillon, Tim @.**@.>>; Author @.**@.>> Subject: Re: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
Hello! Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled. About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment External @.**@.>) Report This Emailhttps://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL3RpbS5kaWxsb25AY29tbXNjb3BlLmNvbS9kYTQ0NGM3MzkzZjA4NDNlMDA5MTBlYjZlNjMyZDRjMy8xNjE5NTEyMzkzLjA5#key=4ce51a4f8c0f1e1bc6d11f93317198e8 FAQhttps://www.inky.com/banner-faq/ Protection by INKYhttps://www.inky.com
Hello!
Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled.
About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment-resource.html#specexternalaccessloadbalancerip-stringhttps://secure-web.cisco.com/1QyzGAbDFQIocZIYPG_SeOK_C2gUrQV9RQC3QOKs9CervwFqBsOjjxawhoeDayNQSq75ClJF9NaB4BKjP9NKELzbHzRFk9xUjIhMfWGvbeVCYr1egQIXIeguj_RpjIvYUreXd8IgswpdnsBtHdonz1B7hmV3l3b89NRhwa277fhn-OCxcf1Wqe3NDs_y3j0OitPm6095h-eYGa6gpuzU_Z5EEKyzHufPR2cBqdr9OHzJmCfoCTKrKDw0FmmdTLDbEXUZPynbmFTrnO_S-ymqQhyLCCMO0BRViKcNUJ4R8e6JhUcUumj5wsjukb2rP43Za/https%3A%2F%2Fwww.arangodb.com%2Fdocs%2Fstable%2Fdeployment-kubernetes-deployment-resource.html%23specexternalaccessloadbalancerip-string
Best Regards, Adam.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1Guvybh1KSYYoxCb6bG_ldBNlh1g-e00M4bZdVkHSm5UzTTxGYoUBTHC7WrILNT_0VRipaP0tnPmQm3_b_JiI9th2M1UvEK5yRMVhYSbjBWIzeRfWwrUlXTYYP7IKzx42UTsd4tcZDN1MA64YH61QGwJ85OB46LPLn-lxXof7x57PKiSiwAEas1yPfztxieQEVXFosP0NOlYXWsR0Qv65SLeg3ZUQoe02NZgUNBlK7N6ltdFT7mzUx_dsRoP53LuNqh_Jwf_do4DirVU7T9w-DpwcDVa-TpURdLfX99NmRfIU5rzXwMmgByW_nwzRSiOA/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fissues%2F716%23issuecomment-827424199, or unsubscribehttps://secure-web.cisco.com/1fTdmEsMWItUbcgFIw92LiaAZSDwqsqG6uwvwfFl7Xo3_f_kyhdNKsdBp4tukf9ABSTzTx7tRYKgquijmUZ31QnaDQEEEVSY8FZNeZNidoqt56g7AKb6p5bJz5lEsFHepZt2NIgvVd5N8vWedtKr13VDtXofyz_Mpzf8YaqStTs4g7vD80YAOzf_hWRHF2VxtrRV_fNP68_vBB5wYTzVaCise_ZqyA-RN-n_ov2ePhTkFffnSKoR8bnXoBM1Wz8Z1_qfb2mNjmZUMgDZ25GkYEYhedsbU0W5DsVQy1VB0AoEkHuHi8jxdf2EaC00YqsMg/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHZ2IP7V5NIZLG36N6EN2DDTKZZEJANCNFSM43LBOSDQ.
I went further and defined this yaml to launch the production-cluster, but it has issues setting up the ExternalIP address for the cluster. This is the yaml I launch with:
apiVersion: "database.arangodb.com/v1" kind: "ArangoDeployment" metadata: name: "production-cluster" spec: mode: Cluster environment: Production image: "10.184.128.10:5000/arangodb/arangodb:lastest" externalAccess: type: LoadBalancer loadBalancerIP: “10.184.138.169” auth: jwtSecretName: None <-- until I can figure out the requirements for the content of the secret of this name agents: count: 3 args:
Here’s what we “see” via kubectl:
arangodbCharts ]$ kubectl get pods,svc,secrets,deployments --all-namespaces -o wide|egrep arango
default pod/arango-kube-arangodb-1618862786-operator-6fd78fb5ff-97c8w 1/1 Running 4 8d 172.16.224.3 p12-tdillon-compute1.broadbus.com
And attached is the log of this latest attempt. It seems it has issues w/DNS (as seen by the :53 address that it fails on) and name resolution. The name that it complains about, does resolve. This is the error:
2021-04-27T20:17:20Z DBG Failed to fetch deployment health error="Get \"https://production-cluster.default.svc:8529/_admin/server/role\": dial tcp: lookup production-cluster.default.svc on 10.96.0.10:53: no such host" component=deployment deployment=production-cluster operator-id=97c8w
Note the resolution, from another pod in the k8s cluster:
@.***:/# ping production-cluster.default.svc PING production-cluster.default.svc.cluster.local (10.97.55.184) 56(84) bytes of data.
Also note, the presence of that IP in the services shown above by the “kubectl get” command:
-kubemaster1 arangodbCharts ]$ kubectl get pods,svc,secrets,deployments --all-namespaces -o wide|grep 10.97.55.184
default service/production-cluster ClusterIP 10.97.55.184
One particular question also on the “Pending” shown above for the externalIP I defined in the yaml being: 10.184.138.169 That is pingable and active in the local environment in k8s:
@.*** arangodbCharts ]$ ping 10.184.138.169 PING 10.184.138.169 (10.184.138.169) 56(84) bytes of data. 64 bytes from 10.184.138.169: icmp_seq=1 ttl=64 time=0.288 ms ^C --- 10.184.138.169 ping statistics --- 1 packets transmitted, 1 received, 0% packet loss, time 0ms
Any pointers you can supply are greatly appreciated.
Thanks, Tim
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Dillon, Tim Sent: Tuesday, April 27, 2021 3:36 PM To: arangodb/kube-arangodb @.>; arangodb/kube-arangodb @.> Cc: Author @.***> Subject: RE: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
So Adam, looks like I need to create a jwt secret (auth.jwtSecretName) and a cacert secret (tls.caSecretName) for the deployment. I could use some direction on what exactly to put in those. I can, of course, create the secrets but not sure on generating the content for them.
As for this:
spec.bootstrap.passwordSecretNames.root: string This setting specifies a secret name for the credentials of the root user. When a deployment is created the operator will setup the root user account according to the credentials given by the secret. If the secret doesn’t exist the operator creates a secret with a random password. There are two magic values for the secret name:
What are the credential field keys it will be looking for? Perhaps “username” and “password”? or maybe something less obvious, or is it implied that it is the root user and I only need a proper tag, aka key, for the base64 encrypted password?
Can you point me to any examples in the documentation, please?
I’m getting there now. The link you gave me yesterday has more info in it than the pages similar to it that I have come across. The expanded "example-arangodb-cluster" Yaml/”ArangoDeployment” spec is helpful.
Thanks, Tim
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Dillon, Tim Sent: Tuesday, April 27, 2021 9:47 AM To: arangodb/kube-arangodb @.**@.>>; arangodb/kube-arangodb @.**@.>> Cc: Author @.**@.>> Subject: RE: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
The attached log snippet from the leading operator shows the issues. The first one that cropped up was:
2021-04-26T16:13:36Z DBG Failed to fetch deployment shard sync state error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w 2021-04-26T16:13:36Z DBG Failed to fetch deployment health error="JWT Secret is missing" component=deployment deployment=production-cluster operator-id=97c8w
It then shows what appears to be attempting to bring up AGNTs (3), PRMRs (3) and CRDNs (3) but no live cluster.
Let me know if there’s anything else I can collect to help with the diagnosis.
Tim Dillon Distinguished Software Engineer
ARRIS AND RUCKUS HAVE JOINED COMMSCOPE 900 Chelmsford St, Lowell, MA 01851 USA Office: +01-978-614-3145
From: Adam Janikowski @.**@.>> Sent: Tuesday, April 27, 2021 4:33 AM To: arangodb/kube-arangodb @.**@.>> Cc: Dillon, Tim @.**@.>>; Author @.**@.>> Subject: Re: [arangodb/kube-arangodb] kube-arangodb v. 1.1.5 does not create root user with empty password and fails using jwt access (#716)
Hello! Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled. About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment External @.**@.>) Report This Emailhttps://shared.outlook.inky.com/report?id=Y29tbXNjb3BlL3RpbS5kaWxsb25AY29tbXNjb3BlLmNvbS9kYTQ0NGM3MzkzZjA4NDNlMDA5MTBlYjZlNjMyZDRjMy8xNjE5NTEyMzkzLjA5#key=4ce51a4f8c0f1e1bc6d11f93317198e8 FAQhttps://www.inky.com/banner-faq/ Protection by INKYhttps://www.inky.com
Hello!
Can you check logs of Operator? It should say why pods cant be created. For example image cannot be pulled.
About question: https://www.arangodb.com/docs/stable/deployment-kubernetes-deployment-resource.html#specexternalaccessloadbalancerip-stringhttps://secure-web.cisco.com/1QyzGAbDFQIocZIYPG_SeOK_C2gUrQV9RQC3QOKs9CervwFqBsOjjxawhoeDayNQSq75ClJF9NaB4BKjP9NKELzbHzRFk9xUjIhMfWGvbeVCYr1egQIXIeguj_RpjIvYUreXd8IgswpdnsBtHdonz1B7hmV3l3b89NRhwa277fhn-OCxcf1Wqe3NDs_y3j0OitPm6095h-eYGa6gpuzU_Z5EEKyzHufPR2cBqdr9OHzJmCfoCTKrKDw0FmmdTLDbEXUZPynbmFTrnO_S-ymqQhyLCCMO0BRViKcNUJ4R8e6JhUcUumj5wsjukb2rP43Za/https%3A%2F%2Fwww.arangodb.com%2Fdocs%2Fstable%2Fdeployment-kubernetes-deployment-resource.html%23specexternalaccessloadbalancerip-string
Best Regards, Adam.
— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHubhttps://secure-web.cisco.com/1Guvybh1KSYYoxCb6bG_ldBNlh1g-e00M4bZdVkHSm5UzTTxGYoUBTHC7WrILNT_0VRipaP0tnPmQm3_b_JiI9th2M1UvEK5yRMVhYSbjBWIzeRfWwrUlXTYYP7IKzx42UTsd4tcZDN1MA64YH61QGwJ85OB46LPLn-lxXof7x57PKiSiwAEas1yPfztxieQEVXFosP0NOlYXWsR0Qv65SLeg3ZUQoe02NZgUNBlK7N6ltdFT7mzUx_dsRoP53LuNqh_Jwf_do4DirVU7T9w-DpwcDVa-TpURdLfX99NmRfIU5rzXwMmgByW_nwzRSiOA/https%3A%2F%2Fgithub.com%2Farangodb%2Fkube-arangodb%2Fissues%2F716%23issuecomment-827424199, or unsubscribehttps://secure-web.cisco.com/1fTdmEsMWItUbcgFIw92LiaAZSDwqsqG6uwvwfFl7Xo3_f_kyhdNKsdBp4tukf9ABSTzTx7tRYKgquijmUZ31QnaDQEEEVSY8FZNeZNidoqt56g7AKb6p5bJz5lEsFHepZt2NIgvVd5N8vWedtKr13VDtXofyz_Mpzf8YaqStTs4g7vD80YAOzf_hWRHF2VxtrRV_fNP68_vBB5wYTzVaCise_ZqyA-RN-n_ov2ePhTkFffnSKoR8bnXoBM1Wz8Z1_qfb2mNjmZUMgDZ25GkYEYhedsbU0W5DsVQy1VB0AoEkHuHi8jxdf2EaC00YqsMg/https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAHZ2IP7V5NIZLG36N6EN2DDTKZZEJANCNFSM43LBOSDQ.
Hello!
To get content of file you can try to omit this field - just comment auth part and apply deployment. Operator will create all required secrets for you. In principal jwt secret is just secret with 'token' in data. Token can be any random string with size up to 32.
Best Regards, Adam
I was trying to use the API to get in via JWT. The token created in the secret: arango-kube-arangodb-1618862786-operator-token-bwdvh I took out, converted it with | base64 -d and then used that via jwtgen using the recipe described in the docs to create a superuser jwt/token. I also created the cacert by taking the ca.crt from the secret, decoding it w/base64 -d, and putting it in a cacert file. I attempted connecting via kubernetes internal network by going into another pod (I can't get any kind of shell in the current/slim arangodb pod (likely the alpine:311 is so lean) and using curl. These are what I was using in an attempt to get to the server using the new jwt I generated using the decoded token in the secret.
Or if I can find out a way I'm supposed to generate the root user password that could help too. (BTW, I don't need to nor want to obfuscate the token below, just so that you can see what it looks like.)
the myjson has only this in it (again root user password is supposed to be empty):
{ "username": "root", "password": "" }
The script did this. I get a 400/bad request on the first one and a 404 on each subsequent API call. Seems like it's not really a 404 issue though as these are basic, are they not? Script:
TOKEN="eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpYXQiOjE2MTg5NDczODQsImV4cCI6MTYxODk1MDk4NSwiaXNzIjoiYXJhbmdvZGIiLCJzZXJ2ZXJfaWQiOiJteWNsaWVudCJ9.J1chHuzfMAkT6_xQrG_BJB9IHsCv7vbnjqLxnKVw5b4"
CACERT="/tmp/cacert"
curl -i -X POST --cacert ${CACERT} -k -H "Content-Type:application/json; charset=utf-8; Accept:application/json" -H "Authorization: Bearer ${TOKEN}" -H "Content-Length: 43" "https://arango-kube-arangodb-1618862786-operator.default.svc:8528/_open/auth -d@/tmp/myjson" curl -i -X GET --cacert ${CACERT} -k -H "Content-Type:application/json, Accept:application/json" -H "Authorization: Bearer ${TOKEN}" "https://arango-kube-arangodb-1618862786-operator.default.svc:8528/_admin/server/jwt"
curl -i -X GET --cacert ${CACERT} -k -H "Content-Type:application/json, Accept:application/json" -H "Authorization: Bearer ${TOKEN}" "https://arango-kube-arangodb-1618862786-operator.default.svc:8528/api/version"
curl -i -X GET --cacert ${CACERT} -k -H "Content-Type:application/json, Accept:application/json" -H "Authorization: Bearer ${TOKEN}" "https://arango-kube-arangodb-1618862786-operator.default.svc:8528/_api/version"
curl -i -X GET --cacert ${CACERT} -k -H "Content-Type:application/json, Accept:application/json" -H "Authorization: bearer ${TOKEN}" "https://arango-kube-arangodb-1618862786-operator.default.svc:8528/_api/query/current"
When I go to the https external endpoint:8528 (I changed it to a LoadBalancer (from the ClusterIP it originally was) and gave it an externalIP address) and all I get is a login screen with login: and password: fields but the green box on the lower right stays "grayed out" until I enter something and neither '' nor "" work as the "empty" password.
Here's the output from a run of the script above, which also contains an /api/version url w/out the _ underscore, just to try that too.
this is from within another pod in the weave network:
./queryRootUser.sh
HTTP/1.1 400 Bad Request Content-Type: text/plain; charset=utf-8 Connection: close
400 Bad RequestHTTP/1.1 404 Not Found Content-Type: text/plain Date: Wed, 21 Apr 2021 20:03:34 GMT Content-Length: 18
404 page not foundHTTP/1.1 404 Not Found Content-Type: text/plain Date: Wed, 21 Apr 2021 20:03:34 GMT Content-Length: 18
404 page not foundHTTP/1.1 404 Not Found Content-Type: text/plain Date: Wed, 21 Apr 2021 20:03:34 GMT Content-Length: 18
404 page not foundHTTP/1.1 404 Not Found Content-Type: text/plain Date: Wed, 21 Apr 2021 20:03:34 GMT Content-Length: 18
404 page not found
Please help and/or point me to reference docs that I may have missed, I've been scouring around for days on this and read the documentation over and over again.
Thanks, Tim