search

jfrog / charts

JFrog official Helm Charts
https://jfrog.com/integration/helm-repository/
Apache License 2.0
256 stars 445 forks source link

artifactory-oss failing with "Master key is missing" #725

Closed govindkailas closed 4 years ago

govindkailas commented 4 years ago

BUG REPORT Artifactory-OSS failing with Master key missing error.

Version of Helm and Kubernetes: Helm: v3.1.2 and K8s: v1.16.4

Which chart: 2.1.2

What happened: I did a helm install with the below command, I just wanted a simple artifactory instance without postgress and ingress settings.

helm install artifactory-oss --set artifactory.nginx.enabled=false --set artifactory.postgresql.enabled=false --set postgresql.enabled=false --set artifactory.artifactory.service.type=NodePort --set artifactory.resources.requests.cpu="500m" --set artifactory.resources.limits.cpu="2" --set artifactory.resources.requests.memory="1Gi" --set artifactory.resources.limits.memory="4Gi" --set artifactory.javaOpts.xms="1g" --set artifactory.javaOpts.xmx="3g" jfrog/artifactory-oss -n tkgdev-artifactory-dev

What you expected to happen: I should see artifactory pod in running state and it should be accessible via nodeport.

How to reproduce it (as minimally and precisely as possible): Run the above-mentioned helm install command

Anything else we need to know: Logs from the pod

kubectl logs -f pod/artifactory-oss-artifactory-0 -n tkgdev-artifactory-dev
Preparing to run Artifactory in Docker
Running as uid=1030(artifactory) gid=1030(artifactory) groups=1030(artifactory)
Dockerfile for this image can found inside the container.
To view the Dockerfile: 'cat /docker/artifactory-oss/Dockerfile.artifactory'.
2020-03-27T13:11:58.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.extraJavaOpts (-Dartifactory.async.corePoolSize=8) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:11:59.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.extraJavaOpts (-Dartifactory.async.corePoolSize=8) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:03.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved artifactory.database.maxOpenConnections (80) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:04.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved access.database.maxOpenConnections (80) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:06.000Z [shell] [INFO ] [] [installerCommon.sh:1140       ] [main] - Checking open files and processes limits
2020-03-27T13:12:07.000Z [shell] [INFO ] [] [installerCommon.sh:1143       ] [main] - Current max open files is 1048576
2020-03-27T13:12:07.000Z [shell] [INFO ] [] [installerCommon.sh:1154       ] [main] - Current max open processes is unlimited
2020-03-27T13:12:07.000Z [shell] [INFO ] [] [installerCommon.sh:1221       ] [main] - Testing directory /opt/jfrog/artifactory/var has read/write permissions for user id 1030
2020-03-27T13:12:07.000Z [shell] [INFO ] [] [installerCommon.sh:1236       ] [main] - Permissions for /opt/jfrog/artifactory/var are good
2020-03-27T13:12:09.000Z [shell] [INFO ] [] [installerCommon.sh:2529       ] [main] - Setting JF_SHARED_NODE_ID to artifactory-oss-artifactory-0
2020-03-27T13:12:09.000Z [shell] [INFO ] [] [installerCommon.sh:2529       ] [main] - Setting JF_SHARED_NODE_IP to 172.50.44.2
2020-03-27T13:12:10.000Z [shell] [INFO ] [] [installerCommon.sh:2529       ] [main] - Setting JF_SHARED_NODE_NAME to artifactory-oss-artifactory-0
2020-03-27T13:12:12.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.extraJavaOpts (-Dartifactory.async.corePoolSize=8) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:12.000Z [shell] [INFO ] [] [artifactoryCommon.sh:104      ] [main] - Final command: -server -Xms512m -Xmx2g -Xss256k -XX:+UseG1GC -XX:OnOutOfMemoryError="kill -9 %p" --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang.reflect=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.base/java.text=ALL-UNNAMED --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.desktop/java.awt.font=ALL-UNNAMED -Dfile.encoding=UTF8 -Djruby.compile.invokedynamic=false -Djruby.bytecode.version=1.8 -Dorg.apache.tomcat.util.buf.UDecoder.ALLOW_ENCODED_SLASH=true -Djava.security.egd=file:/dev/./urandom -Dartdist=zip -Djf.product.home=/opt/jfrog/artifactory -Dartifactory.async.corePoolSize=8
2020-03-27T13:12:14.000Z [shell] [INFO ] [] [artifactoryCommon.sh:121      ] [main] - Saving /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml as /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml.orig
2020-03-27T13:12:15.000Z [shell] [INFO ] [] [artifactoryCommon.sh:129      ] [main] - Using Tomcat template to generate : /opt/jfrog/artifactory/app/artifactory/tomcat/conf/server.xml
2020-03-27T13:12:16.000Z [shell] [INFO ] [] [artifactoryCommon.sh:631      ] [main] - Resolved ${artifactory.port||8081} to default value : 8081
2020-03-27T13:12:16.000Z [shell] [INFO ] [] [artifactoryCommon.sh:631      ] [main] - Resolved ${artifactory.tomcat.connector.maxThreads||200} to default value : 200
2020-03-27T13:12:17.000Z [shell] [INFO ] [] [artifactoryCommon.sh:631      ] [main] - Resolved ${access.http.port||8040} to default value : 8040
2020-03-27T13:12:18.000Z [shell] [INFO ] [] [artifactoryCommon.sh:631      ] [main] - Resolved ${access.tomcat.connector.maxThreads||50} to default value : 50
2020-03-27T13:12:20.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved JF_PRODUCT_HOME (/opt/jfrog/artifactory) from environment variable
2020-03-27T13:12:21.000Z [shell] [INFO ] [] [artifactoryCommon.sh:631      ] [main] - Resolved ${shared.tomcat.workDir||/opt/jfrog/artifactory/var/work/artifactory/tomcat} to default value : /opt/jfrog/artifactory/var/work/artifactory/tomcat

========================
JF Environment variables
========================

JF_SHARED_NODE_NAME                 : artifactory-oss-artifactory-0
JF_SYSTEM_YAML                      : /opt/jfrog/artifactory/var/etc/system.yaml
JF_ARTIFACTORY_PID                  : /opt/jfrog/artifactory/app/run/artifactory.pid
JF_SHARED_EXTRAJAVAOPTS             : -Dartifactory.async.corePoolSize
JF_PRODUCT_DATA_INTERNAL            : /var/opt/jfrog/artifactory
JF_PRODUCT_HOME                     : /opt/jfrog/artifactory
JF_ROUTER_TOPOLOGY_LOCAL_REQUIREDSERVICETYPES : jfrt,jfac,jfmd,jffe
JF_SHARED_NODE_IP                   : 172.50.44.2
JF_SHARED_NODE_ID                   : artifactory-oss-artifactory-0
JF_ARTIFACTORY_USER                 : artifactory
Starting router...
router not running. Proceed to start it up.
router started. PID: 2850
Starting frontend...
frontend not running. Proceed to start it up.
2020-03-27T13:12:27.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.logging.consoleLog.enabled (false) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:28.839Z [jfrou] [INFO ] [13be4d1abb5ab136] [bootstrap.go:70               ] [main                ] - Router (jfrou) service initialization started. Version: 1.2.1 Revision: fe2dd4ed3212d39bdd665c92e17293e8071eb99d PID: 2850 Home: /opt/jfrog/artifactory
2020-03-27T13:12:28.840Z [jfrou] [INFO ] [13be4d1abb5ab136] [bootstrap.go:73               ] [main                ] - JFrog Router IP: 172.50.44.2
2020-03-27T13:12:28.000Z [shell] [INFO ] [] [installerCommon.sh:635        ] [main] - Redirection is set to false. Skipping log redirection
frontend started. PID: 3022
2020-03-27T13:12:31.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.logging.consoleLog.enabled (false) from /opt/jfrog/artifactory/var/etc/system.yaml
Starting metadata...
2020-03-27T13:12:32.000Z [shell] [INFO ] [] [installerCommon.sh:635        ] [main] - Redirection is set to false. Skipping log redirection
2020-03-27T13:12:33.939Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 5 seconds with 2m0s timeout
JF_METADATA_ACCESSCLIENT_URL: http://localhost:8081/access
metadata started. PID: 3257
[DEBUG] Resolved system configuration file path: /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:37.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.logging.consoleLog.enabled (false) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:38.000Z [shell] [INFO ] [] [installerCommon.sh:635        ] [main] - Redirection is set to false. Skipping log redirection
2020-03-27T13:12:38.000Z [shell] [INFO ] [] [installerCommon.sh:903        ] [main] - Redirection is set to false. Skipping catalina log redirection
2020-03-27T13:12:39.038Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 10 seconds with 2m0s timeout
NOTE: Picked up JDK_JAVA_OPTIONS:  --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
2020-03-27T13:12:40.000Z [shell] [INFO ] [] [systemYamlHelper.sh:270       ] [main] - Resolved shared.logging.consoleLog.enabled (false) from /opt/jfrog/artifactory/var/etc/system.yaml
2020-03-27T13:12:41.939Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 5 seconds with 2m0s timeout [init]
2020-03-27T13:12:42.000Z [shell] [INFO ] [] [installerCommon.sh:635        ] [main] - Redirection is set to false. Skipping log redirection
2020-03-27T13:12:44.040Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 15 seconds with 2m0s timeout
2020-03-27T13:12:46.942Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 10 seconds with 2m0s timeout [init]
2020-03-27T13:12:49.042Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 20 seconds with 2m0s timeout
2020-03-27T13:12:49.140Z [jffe ] [INFO ] [                ] [                              ] [main                ] - frontend (jffe) service initialization started. Version: 1.3.1 Revision: 0.0.0 PID: 3022 Home: /opt/jfrog/artifactory
2020-03-27T13:12:51.944Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 15 seconds with 2m0s timeout [init]
2020-03-27T13:12:54.137Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 25 seconds with 2m0s timeout
2020-03-27T13:12:57.038Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 20 seconds with 2m0s timeout [init]
2020-03-27T13:12:59.237Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 30 seconds with 2m0s timeout
2020-03-27T13:13:02.041Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 25 seconds with 2m0s timeout [init]
2020-03-27T13:13:02.546L [tomct] [INFO ] [                ] [org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandler ["http-nio-8081"]
2020-03-27T13:13:03.439L [tomct] [INFO ] [                ] [org.apache.tomcat.util.net.NioSelectorPool] [org.apache.tomcat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read
2020-03-27T13:13:03.938L [tomct] [INFO ] [                ] [org.apache.coyote.http11.Http11NioProtocol] [org.apache.coyote.AbstractProtocol init] - Initializing ProtocolHandler ["http-nio-8040"]
2020-03-27T13:13:03.940L [tomct] [INFO ] [                ] [org.apache.tomcat.util.net.NioSelectorPool] [org.apache.tomcat.util.net.NioSelectorPool getSharedSelector] - Using a shared selector for servlet write/read
2020-03-27T13:13:04.238L [tomct] [INFO ] [                ] [org.apache.catalina.core.StandardService] [org.apache.catalina.core.StandardService startInternal] - Starting service [Catalina]
2020-03-27T13:13:04.240Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 35 seconds with 2m0s timeout
2020-03-27T13:13:04.240L [tomct] [INFO ] [                ] [org.apache.catalina.core.StandardEngine] [org.apache.catalina.core.StandardEngine startInternal] - Starting Servlet Engine: Apache Tomcat/8.5.51
2020-03-27T13:13:05.039L [tomct] [INFO ] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml]
2020-03-27T13:13:05.136L [tomct] [INFO ] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deploying deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/access.xml]
2020-03-27T13:13:06.241L [tomct] [WARNING] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access.war] inside the host appBase has been specified, and will be ignored
2020-03-27T13:13:06.336L [tomct] [WARNING] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - A docBase [/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/artifactory.war] inside the host appBase has been specified, and will be ignored
2020-03-27T13:13:07.043Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 30 seconds with 2m0s timeout [init]
2020-03-27T13:13:09.242Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 40 seconds with 2m0s timeout
2020-03-27T13:13:12.136Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 35 seconds with 2m0s timeout [init]
2020-03-27T13:13:14.337Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 45 seconds with 2m0s timeout
2020-03-27T13:13:17.139Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 40 seconds with 2m0s timeout [init]
2020-03-27T13:13:19.340Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 50 seconds with 2m0s timeout
2020-03-27T13:13:22.141Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 45 seconds with 2m0s timeout [init]
2020-03-27T13:13:24.736Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 55 seconds with 2m0s timeout
2020-03-27T13:13:27.236Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 50 seconds with 2m0s timeout [init]
2020-03-27T13:13:29.837Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 60 seconds with 2m0s timeout
2020-03-27T13:13:32.336Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 55 seconds with 2m0s timeout [init]
2020-03-27T13:13:33.939Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 65 seconds with 2m0s timeout
2020-03-27T13:13:37.339Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 60 seconds with 2m0s timeout [init]
2020-03-27T13:13:38.941Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 70 seconds with 2m0s timeout
2020-03-27T13:13:42.341Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 65 seconds with 2m0s timeout [init]
2020-03-27T13:13:44.237Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 75 seconds with 2m0s timeout
2020-03-27T13:13:47.536Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 70 seconds with 2m0s timeout [init]
2020-03-27T13:13:49.436Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 80 seconds with 2m0s timeout
2020-03-27T13:13:52.540Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 75 seconds with 2m0s timeout [init]
2020-03-27T13:13:54.438Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 85 seconds with 2m0s timeout
2020-03-27T13:13:57.542Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 80 seconds with 2m0s timeout [init]
2020-03-27T13:13:59.537Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 90 seconds with 2m0s timeout
2020-03-27T13:14:02.544Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 85 seconds with 2m0s timeout [init]
2020-03-27T13:14:07.638Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 90 seconds with 2m0s timeout [init]
2020-03-27T13:14:08.938Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 100 seconds with 2m0s timeout
2020-03-27T13:14:12.640Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 95 seconds with 2m0s timeout [init]
2020-03-27T13:14:14.038Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 105 seconds with 2m0s timeout
2020-03-27T13:14:15.636Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.a.c.h.HaNodeProperties:67   ] [ocalhost-startStop-2] - Artifactory is running in non-clustered mode.
2020-03-27T13:14:16.140Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [tifactoryHomeConfigListener:70] [ocalhost-startStop-2] - Resolved Home: '/opt/jfrog/artifactory
2020-03-27T13:14:17.437Z [jfrt ] [INFO ] [                ] [o.j.c.w.FileWatchingManager:75] [Thread-4            ] - Starting watch of folder configurations
2020-03-27T13:14:17.737Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 100 seconds with 2m0s timeout [init]
2020-03-27T13:14:19.137Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 110 seconds with 2m0s timeout
2020-03-27T13:14:21.441Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [.a.w.s.BasicConfigManagers:140] [ocalhost-startStop-2] - Artifactory (jfrt) service initialization started. Version: 7.3.2 Revision: 70302900 PID: 3411 Home: /opt/jfrog/artifactory
2020-03-27T13:14:21.941Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [.a.c.ConvertersManagerImpl:142] [ocalhost-startStop-2] - Triggering PRE_INIT conversion, from 7.3.2 to org.artifactory.version.ArtifactoryVersionImpl@64ab6575
2020-03-27T13:14:22.038Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [.a.c.ConvertersManagerImpl:144] [ocalhost-startStop-2] - Finished PRE_INIT conversion, current version is: org.artifactory.version.ArtifactoryVersionImpl@64ab6575
2020-03-27T13:14:22.838Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 105 seconds with 2m0s timeout [init]
2020-03-27T13:14:24.237Z [jfrou] [INFO ] [13be4d1abb5ab136] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 115 seconds with 2m0s timeout
2020-03-27T13:14:27.840Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 110 seconds with 2m0s timeout [init]
2020-03-27T13:14:28.338Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:28.936Z [jfrou] [FATAL] [13be4d1abb5ab136] [bootstrap.go:92               ] [main                ] - Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key
2020-03-27T13:14:31.936Z [jfmd ] [INFO ] [6e3bc05ad06938c5] [security_keys.go:114          ] [main                ] - Master key is missing. Pending for 115 seconds with 2m0s timeout [init]
2020-03-27T13:14:33.743Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:36.842Z [jfmd ] [ERROR] [6e3bc05ad06938c5] [keys.go:22                    ] [main                ] - Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key
goroutine 1 [running]:
runtime/debug.Stack(0x96, 0x14ab560, 0xc0003523c0)
    /src/runtime/debug/stack.go:24 +0x9d
jfrog.com/metadata/common/log.(*standardLogger).Panicfc(0xc0004d87b8, 0x14cd340, 0xc000399710, 0xc0003523c0, 0x96, 0x0, 0x0, 0x1)
    /src/jfrog.com/metadata/common/log/standard_logger.go:44 +0x6a
jfrog.com/metadata/services/common.MustResolveSecurityKeys(0x14cd340, 0xc000399710, 0x14cdec0, 0xc0004bd630, 0x14dd540, 0xc0004d87b8)
    /src/jfrog.com/metadata/services/common/keys.go:22 +0x1c0
main.main()
    /src/jfrog.com/metadata/metadata.go:28 +0x3dc
 [init]
panic: Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key
goroutine 1 [running]:
runtime/debug.Stack(0x96, 0x14ab560, 0xc0003523c0)
    /src/runtime/debug/stack.go:24 +0x9d
jfrog.com/metadata/common/log.(*standardLogger).Panicfc(0xc0004d87b8, 0x14cd340, 0xc000399710, 0xc0003523c0, 0x96, 0x0, 0x0, 0x1)
    /src/jfrog.com/metadata/common/log/standard_logger.go:44 +0x6a
jfrog.com/metadata/services/common.MustResolveSecurityKeys(0x14cd340, 0xc000399710, 0x14cdec0, 0xc0004bd630, 0x14dd540, 0xc0004d87b8)
    /src/jfrog.com/metadata/services/common/keys.go:22 +0x1c0
main.main()
    /src/jfrog.com/metadata/metadata.go:28 +0x3dc

goroutine 1 [running]:
github.com/rs/zerolog.(*Logger).Panic.func1(0xc000422300, 0x2c5)
    /pkg/mod/github.com/rs/zerolog@v1.17.2/log.go:338 +0x4f
github.com/rs/zerolog.(*Event).msg(0xc0000e63c0, 0xc000422300, 0x2c5)
    /pkg/mod/github.com/rs/zerolog@v1.17.2/event.go:146 +0x186
github.com/rs/zerolog.(*Event).Msgf(0xc0000e63c0, 0xc000352460, 0x99, 0xc00038fd80, 0x1, 0x1)
    /pkg/mod/github.com/rs/zerolog@v1.17.2/event.go:126 +0x83
jfrog.com/metadata/common/log.(*standardLogger).logMessage(0xc0004d87b8, 0x14cd340, 0xc000399710, 0xc0000e63c0, 0xc000352460, 0x99, 0xc00038fd80, 0x1, 0x1)
    /src/jfrog.com/metadata/common/log/standard_logger.go:88 +0xbc
jfrog.com/metadata/common/log.(*standardLogger).Panicfc(0xc0004d87b8, 0x14cd340, 0xc000399710, 0xc0003523c0, 0x96, 0xc00038fd80, 0x1, 0x1)
    /src/jfrog.com/metadata/common/log/standard_logger.go:45 +0x1df
jfrog.com/metadata/services/common.MustResolveSecurityKeys(0x14cd340, 0xc000399710, 0x14cdec0, 0xc0004bd630, 0x14dd540, 0xc0004d87b8)
    /src/jfrog.com/metadata/services/common/keys.go:22 +0x1c0
main.main()
    /src/jfrog.com/metadata/metadata.go:28 +0x3dc
2020-03-27T13:14:38.941Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:44.337Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:49.537Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:49.637Z [jfac ] [INFO ] [e7dc0737db197c5f] [licationContextInitializer:162] [ocalhost-startStop-1] - Access (jfac) service initialization started. Version: 5.3.9 Revision: 50309900 PID: 3411 Home: /opt/jfrog/artifactory
2020-03-27T13:14:51.839Z [jfac ] [INFO ] [e7dc0737db197c5f] [o.j.a.AccessApplication:50    ] [ocalhost-startStop-1] - Starting AccessApplication v5.3.9 on artifactory-oss-artifactory-0 with PID 3411 (/opt/jfrog/artifactory/app/artifactory/tomcat/webapps/access/WEB-INF/lib/access-application-5.3.9.jar started by artifactory in /var/opt/jfrog/artifactory)
2020-03-27T13:14:51.840Z [jfac ] [INFO ] [e7dc0737db197c5f] [o.j.a.AccessApplication:651   ] [ocalhost-startStop-1] - The following profiles are active: production
2020-03-27T13:14:54.738Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:14:59.839Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:15:05.037Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:15:10.536Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:15:15.743Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:15:20.840Z [jfrt ] [INFO ] [8af00c91cc94b4a3] [o.j.s.c.KeyUtils:108          ] [ocalhost-startStop-2] - Master key is missing. Pending for 5 seconds with 60 seconds timeout
2020-03-27T13:15:24.336Z [jfrt ] [ERROR] [8af00c91cc94b4a3] [tifactoryHomeConfigListener:59] [ocalhost-startStop-2] - Failed initializing Home. Failed resolving master key; Missing master key
2020-03-27T13:15:24.341Z [jfrt ] [ERROR] [8af00c91cc94b4a3] [actoryContextConfigListener:82] [ocalhost-startStop-2] - Failed initializing Artifactory context: Artifactory home not initialized.
2020-03-27T13:15:24.436L [tomct] [SEVERE] [                ] [org.apache.catalina.core.StandardContext] [org.apache.catalina.core.StandardContext startInternal] - One or more listeners failed to start. Full details will be found in the appropriate container log file
2020-03-27T13:15:24.640L [tomct] [WARNING] [                ] [org.apache.catalina.util.SessionIdGeneratorBase] [org.apache.catalina.util.SessionIdGeneratorBase createSecureRandom] - Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [199] milliseconds.
2020-03-27T13:15:24.736L [tomct] [SEVERE] [                ] [org.apache.catalina.core.StandardContext] [org.apache.catalina.core.StandardContext startInternal] - Context [/artifactory] startup failed due to previous errors
2020-03-27T13:15:28.036L [tomct] [INFO ] [                ] [org.apache.catalina.startup.HostConfig] [org.apache.catalina.startup.HostConfig deployDescriptor] - Deployment of deployment descriptor [/opt/jfrog/artifactory/app/artifactory/tomcat/conf/Catalina/localhost/artifactory.xml] has finished in [142,903] ms
2020-03-27T13:16:08.445Z [jffe ] [ERROR] [                ] [                              ] [main                ] - Error: connect ECONNREFUSED 127.0.0.1:8046
    at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)
2020-03-27T13:16:08.537Z [jffe ] [INFO ] [                ] [                              ] [main                ] - exit detected
2020-03-27T13:16:08.538Z [jffe ] [INFO ] [                ] [                              ] [main                ] - cleanup options : {"cleanup":true,"exit":true}
2020-03-27T13:16:08.538Z [jffe ] [INFO ] [                ] [                              ] [main                ] - doing clean
2020-03-27T13:16:08.538Z [jffe ] [INFO ] [                ] [                              ] [main                ] - unregistering from router
2020-03-27T13:16:08.539Z [jffe ] [INFO ] [                ] [                              ] [main                ] - client is already unregistered from router
2020-03-27T13:16:08.539Z [jffe ] [INFO ] [                ] [                              ] [main                ] - exiting app

Output of kubectl get all

kubectl get all -n tkgdev-artifactory-dev
NAME                                READY   STATUS    RESTARTS   AGE
pod/artifactory-oss-artifactory-0   0/1     Running   3          19m

NAME                                  TYPE       CLUSTER-IP     EXTERNAL-IP   PORT(S)                         AGE
service/artifactory-oss-artifactory   NodePort   192.30.66.10   <none>        8082:31552/TCP,8081:31831/TCP   19m

NAME                                           READY   AGE
statefulset.apps/artifactory-oss-artifactory   0/1     19m

Output of describe pod

kubectl describe  pod/artifactory-oss-artifactory-0 -n tkgdev-artifactory-dev
Name:           artifactory-oss-artifactory-0
Namespace:      tkgdev-artifactory-dev
Priority:       0
Node:           wdc-tkg-d1-w06.vmware.com/10.166.112.229
Start Time:     Fri, 27 Mar 2020 18:25:37 +0530
Labels:         app=artifactory
                chart=artifactory-9.1.5
                component=artifactory
                controller-revision-hash=artifactory-oss-artifactory-67f4f9668
                heritage=Helm
                release=artifactory-oss
                role=artifactory
                statefulset.kubernetes.io/pod-name=artifactory-oss-artifactory-0
Annotations:    checksum/access-creds: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
                checksum/binarystore: 91c9ad6acb619c8e8f63aa0581eeb9a2a17f16c66560f07339f2c4dcc4c17796
                checksum/systemyaml: 861622abf33f684de5fb0539a116b941d97cf915906c1ca9b981b8a81a2f9a21
                kubernetes.io/limit-ranger:
                  LimitRanger plugin set: memory request for container artifactory; memory limit for container artifactory; memory request for init containe...
Status:         Running
IP:             172.50.44.2
Controlled By:  StatefulSet/artifactory-oss-artifactory
Init Containers:
  delete-db-properties:
    Container ID:  docker://ddf74474eedc07123e7748296e4c571e4e4668a7dbab49a299f88d5d7cfd354a
    Image:         alpine:3.10
    Image ID:      docker-pullable://alpine@sha256:7c3773f7bcc969f03f8f653910001d99a9d324b4b9caa008846ad2c3089f5a5f
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      rm -fv /var/opt/jfrog/artifactory/etc/db.properties
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Mar 2020 18:25:44 +0530
      Finished:     Fri, 27 Mar 2020 18:25:44 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /var/opt/jfrog/artifactory from artifactory-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from artifactory-oss-artifactory-token-96jgc (ro)
  remove-lost-found:
    Container ID:  docker://178c86f1466e1d61673fe86c1d69a440ed93259b0d6ad96e1257a2837518ac27
    Image:         alpine:3.10
    Image ID:      docker-pullable://alpine@sha256:7c3773f7bcc969f03f8f653910001d99a9d324b4b9caa008846ad2c3089f5a5f
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      rm -rfv /var/opt/jfrog/artifactory/lost+found /var/opt/jfrog/artifactory/data/.lock
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Mar 2020 18:25:45 +0530
      Finished:     Fri, 27 Mar 2020 18:25:45 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /var/opt/jfrog/artifactory from artifactory-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from artifactory-oss-artifactory-token-96jgc (ro)
  copy-system-yaml:
    Container ID:  docker://beee052efbcb3f2d0817b9a808cc76419cff34636f1fe01174dbbc158c006e40
    Image:         alpine:3.10
    Image ID:      docker-pullable://alpine@sha256:7c3773f7bcc969f03f8f653910001d99a9d324b4b9caa008846ad2c3089f5a5f
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/sh
      -c
      sleep 30; echo "Copy system.yaml to /var/opt/jfrog/artifactory/etc"; mkdir -p /var/opt/jfrog/artifactory/etc; mkdir -p /var/opt/jfrog/artifactory/etc/access/keys/trusted; cp -fv /tmp/etc/system.yaml /var/opt/jfrog/artifactory/etc/system.yaml; echo "Remove /var/opt/jfrog/artifactory/lost+found folder if exists"; rm -rfv /var/opt/jfrog/artifactory/lost+found;

    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Mar 2020 18:25:46 +0530
      Finished:     Fri, 27 Mar 2020 18:26:16 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /tmp/etc/system.yaml from systemyaml (rw,path="system.yaml")
      /var/opt/jfrog/artifactory from artifactory-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from artifactory-oss-artifactory-token-96jgc (ro)
  migration-artifactory:
    Container ID:  docker://e04d4fd713bfea7d8a1ab333c3cc5464496b7463d6c0740c5e2fa6df915ab57b
    Image:         docker.bintray.io/jfrog/artifactory-oss:7.3.2
    Image ID:      docker-pullable://docker.bintray.io/jfrog/artifactory-oss@sha256:4527852ed4c806c4f88229c06c7af8e0b4e632f07522fdd6952ac39160aaedcf
    Port:          <none>
    Host Port:     <none>
    Command:
      /bin/bash
      -c
      scriptsPath="/opt/jfrog/artifactory/app/bin"; mkdir -p $scriptsPath; echo "Copy migration scripts and Run migration"; cp -fv /tmp/migrate.sh $scriptsPath/migrate.sh; cp -fv /tmp/migrationHelmInfo.yaml $scriptsPath/migrationHelmInfo.yaml; cp -fv /tmp/migrationStatus.sh $scriptsPath/migrationStatus.sh; mkdir -p /var/opt/jfrog/artifactory/log; bash $scriptsPath/migrationStatus.sh 7.3.2 > >(tee /var/opt/jfrog/artifactory/log/helm-migration.log) 2>&1;

    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 27 Mar 2020 18:26:17 +0530
      Finished:     Fri, 27 Mar 2020 18:26:20 +0530
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /artifactory_bootstrap/binarystore.xml from binarystore-xml (rw,path="binarystore.xml")
      /tmp/migrate.sh from migration-scripts (rw,path="migrate.sh")
      /tmp/migrationHelmInfo.yaml from migration-scripts (rw,path="migrationHelmInfo.yaml")
      /tmp/migrationStatus.sh from migration-scripts (rw,path="migrationStatus.sh")
      /var/opt/jfrog/artifactory from artifactory-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from artifactory-oss-artifactory-token-96jgc (ro)
Containers:
  artifactory:
    Container ID:  docker://eda3fedd67ded28a46b70d1e8ace4249810342ded670d18e911810cee6e33ed5
    Image:         docker.bintray.io/jfrog/artifactory-oss:7.3.2
    Image ID:      docker-pullable://docker.bintray.io/jfrog/artifactory-oss@sha256:4527852ed4c806c4f88229c06c7af8e0b4e632f07522fdd6952ac39160aaedcf
    Ports:         8082/TCP, 8081/TCP
    Host Ports:    0/TCP, 0/TCP
    Command:
      /bin/sh
      -c
      set -e; if [ -d /artifactory_extra_conf ] && [ -d /artifactory_bootstrap ]; then
        echo "Copying bootstrap config from /artifactory_extra_conf to /artifactory_bootstrap";
        cp -Lrfv /artifactory_extra_conf/ /artifactory_bootstrap/;
      fi; /entrypoint-artifactory.sh

    State:          Running
      Started:      Fri, 27 Mar 2020 18:41:50 +0530
    Last State:     Terminated
      Reason:       Error
      Exit Code:    137
      Started:      Fri, 27 Mar 2020 18:36:40 +0530
      Finished:     Fri, 27 Mar 2020 18:41:49 +0530
    Ready:          False
    Restart Count:  3
    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi
    Liveness:     http-get http://:8082/router/api/v1/system/health delay=180s timeout=10s period=10s #success=1 #failure=10
    Readiness:    http-get http://:8082/router/api/v1/system/health delay=60s timeout=10s period=10s #success=1 #failure=10
    Environment:  <none>
    Mounts:
      /artifactory_bootstrap/binarystore.xml from binarystore-xml (rw,path="binarystore.xml")
      /artifactory_bootstrap/info/installer-info.json from installer-info (rw,path="installer-info.json")
      /var/opt/jfrog/artifactory from artifactory-volume (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from artifactory-oss-artifactory-token-96jgc (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  artifactory-volume:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  artifactory-volume-artifactory-oss-artifactory-0
    ReadOnly:   false
  binarystore-xml:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  artifactory-oss-artifactory-binarystore
    Optional:    false
  migration-scripts:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      artifactory-oss-artifactory-migration-scripts
    Optional:  false
  installer-info:
    Type:      ConfigMap (a volume populated by a ConfigMap)
    Name:      artifactory-oss-artifactory-installer-info
    Optional:  false
  systemyaml:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  artifactory-oss-artifactory-systemyaml
    Optional:    false
  artifactory-oss-artifactory-token-96jgc:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  artifactory-oss-artifactory-token-96jgc
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason                  Age                  From                                Message
  ----     ------                  ----                 ----                                -------
  Warning  FailedScheduling        <unknown>            default-scheduler                   pod has unbound immediate PersistentVolumeClaims (repeated 6 times)
  Warning  FailedScheduling        <unknown>            default-scheduler                   pod has unbound immediate PersistentVolumeClaims (repeated 6 times)
  Normal   Scheduled               <unknown>            default-scheduler                   Successfully assigned tkgdev-artifactory-dev/artifactory-oss-artifactory-0 to wdc-tkg-d1-w06.vmware.com
  Normal   SuccessfulAttachVolume  16m                  attachdetach-controller             AttachVolume.Attach succeeded for volume "pvc-d75e6829-76cf-4af4-baa3-43d3e447ea76"
  Normal   Pulled                  16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Container image "alpine:3.10" already present on machine
  Normal   Created                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Created container delete-db-properties
  Normal   Started                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Started container delete-db-properties
  Normal   Pulled                  16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Container image "alpine:3.10" already present on machine
  Normal   Created                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Created container remove-lost-found
  Normal   Started                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Started container remove-lost-found
  Normal   Pulled                  16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Container image "alpine:3.10" already present on machine
  Normal   Created                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Created container copy-system-yaml
  Normal   Started                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Started container copy-system-yaml
  Normal   Pulled                  16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Container image "docker.bintray.io/jfrog/artifactory-oss:7.3.2" already present on machine
  Normal   Created                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Created container migration-artifactory
  Normal   Started                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Started container migration-artifactory
  Normal   Pulled                  16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Container image "docker.bintray.io/jfrog/artifactory-oss:7.3.2" already present on machine
  Normal   Created                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Created container artifactory
  Normal   Started                 16m                  kubelet, wdc-tkg-d1-w06.vmware.com  Started container artifactory
  Warning  Unhealthy               13m (x10 over 15m)   kubelet, wdc-tkg-d1-w06.vmware.com  Readiness probe failed: Get http://172.50.44.2:8082/router/api/v1/system/health: dial tcp 172.50.44.2:8082: connect: connection refused
  Warning  Unhealthy               107s (x27 over 13m)  kubelet, wdc-tkg-d1-w06.vmware.com  Liveness probe failed: Get http://172.50.44.2:8082/router/api/v1/system/health: dial tcp 172.50.44.2:8082: connect: connection refused

Output of pvc I see that pvc got created and its attached to the pod as well,

kubectl get pvc -n tkgdev-artifactory-dev
NAME                                               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
artifactory-volume-artifactory-oss-artifactory-0   Bound    pvc-d75e6829-76cf-4af4-baa3-43d3e447ea76   20Gi       RWO            vsan-bloc-sc   50m

Out of curiosity went inside the pod and inspected the folder, there is no key. Why would OSS version need a key? 

kubectl exec -it pod/artifactory-oss-artifactory-0 bash -n tkgdev-artifactory-dev
artifactory@artifactory-oss-artifactory-0:/opt/jfrog/artifactory$ cd  /opt/jfrog/artifactory
artifactory@artifactory-oss-artifactory-0:/opt/jfrog/artifactory$ cd var/etc/security/
artifactory@artifactory-oss-artifactory-0:/opt/jfrog/artifactory/var/etc/security$ ls -tlr
total 4
drwxr-sr-x    3 artifact artifact      4096 Mar 27 12:57 keys
artifactory@artifactory-oss-artifactory-0:/opt/jfrog/artifactory/var/etc/security$ ls -ltr keys/
total 4
drwxr-sr-x    2 artifact artifact      4096 Mar 27 12:57 trusted
artifactory@artifactory-oss-artifactory-0:/opt/jfrog/artifactory/var/etc/security$ ls -ltr keys/trusted/
total 0
eldada commented 4 years ago

Hi @govindkailas . I tried your exact setup and it worked as expected although parameters are not passed in properly as you have another level of artifactory (since the OSS chart uses the main artifactory chart as a dependency). The issue in your case is that you need to pass all the parameters with another preceding artifactory.. For example:

helm install artifactory-oss \
  --set artifactory.nginx.enabled=false \
  --set artifactory.postgresql.enabled=false \
  --set postgresql.enabled=false \
  --set artifactory.artifactory.service.type=NodePort \
  --set artifactory.artifactory.resources.requests.cpu="500m" \
  --set artifactory.artifactory.resources.limits.cpu="2" \
  --set artifactory.artifactory.resources.requests.memory="1Gi" \
  --set artifactory.artifactory.resources.limits.memory="4Gi" \
  --set artifactory.artifactory.javaOpts.xms="1g" \
  --set artifactory.artifactory.javaOpts.xmx="3g" \
  jfrog/artifactory-oss -n tkgdev-artifactory-dev

(I see you did it right for some of the parameters.

Looking at the output of your kubectl describe, I see Artifactory does not get the assigned resources you sent in your command line, but much lower, which are probably defaults set in your cluster or namespace...

    Limits:
      cpu:     200m
      memory:  512Mi
    Requests:
      cpu:        25m
      memory:     256Mi

The limits here are far too low for Artifactory to run properly.

Fix the parameters and test again. Let us know the results.

govindkailas commented 4 years ago

Yes, that's true. There is a default resource limit set on namespace. Any idea why is the passed resource request is not hournered? Also what's the master key missing error which is coming in the log? Nevertheless I will remove the resource limits on namespace and try again.

govindkailas commented 4 years ago

I have removed the default resource limits set on the namespace, now I see that artifcatory-oss is not picking up the limits we pass from the helm command line. I used the same helm install command that you provided.

3m9s Warning FailedCreate statefulset/artifactory-oss-artifactory create Pod artifactory-oss-artifactory-0 in StatefulSet artifactory-oss-artifactory failed error: pods "artifactory-oss-artifactory-0" is forbidden: failed quota: tkgdev-artifactory-dev-quota: must specify limits.cpu,limits.memory,requests.cpu,requests.memory

eldada commented 4 years ago

Yes, that's true. There is a default resource limit set on namespace. Any idea why is the passed resource request is not hournered? Also what's the master key missing error which is coming in the log? Nevertheless I will remove the resource limits on namespace and try again.

This was not honoured because of the missing extra artifactory. prefix.

As for

failed error: pods "artifactory-oss-artifactory-0" is forbidden: failed quota: tkgdev-artifactory-dev-quota: must specify limits.cpu,limits.memory,requests.cpu,requests.memory

I assume there's some policy on your cluster. I tested again with my command I provided and it started up properly with the correct resources set.

govindkailas commented 4 years ago

Thanks @eldada, I had to adjust the resource quota to get this working. Now I notice that there are two ports open, Is there a way to keep it as one single port(say 8081)? 

kubectl get all -n tkgdev-artifactory-dev
NAME                                READY   STATUS    RESTARTS   AGE
pod/artifactory-oss-artifactory-0   1/1     Running   0          4m21s

NAME                                  TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
service/artifactory-oss-artifactory   LoadBalancer   192.25.94.244   <pending>     8082:31767/TCP,8081:31276/TCP   4m23s

NAME                                           READY   AGE
statefulset.apps/artifactory-oss-artifactory   1/1     4m24s
JfrogPrasanna commented 4 years ago

@govindkailas The two ports are indeed needed, one for ui and as an entry for other products via a new service called router, and the other for artifactory APIs.

eldada commented 4 years ago

Important to note that you don't have to use both ports. Port 8082 can route to Artifactory too, but we keep the port 8081 for backward compatibility and for better performance (as you skip the router and go directly to Artifactory). I'm closing this issue. If needed, you can reopen to continue discussion.

ningjunwei2 commented 1 year ago

@eldada I meet this problem。my helm is V3 ,artifactory-oss is 7.46.11 . i use ceph bucket.there is my s3 config,and artifactory statefulsets.apps config

persistence: mountPath: "/var/opt/jfrog/artifactory"

enabled: true

## A manually managed Persistent Volume and Claim
## Requires persistence.enabled: true
## If defined, PVC must be created manually before volume will be bound
# existingClaim:

      #accessMode: ReadWriteOnce
## Storage default size. Should be increased for production deployments.
#size: 800Gi

## Use a custom Secret to be mounted as your binarystore.xml
## NOTE: This will ignore all settings below that make up binarystore.xml
#customBinarystoreXmlSecret:
## Redundancy required For HA deployments, with "cluster" persistence storage type
#redundancy: 3
#lenientLimit: 1
## Cache default size. Should be increased for production deployments.
#maxCacheSize: 5000000000
#cacheProviderDir: cache

## Set the persistence storage type. This will apply the matching binarystore.xml to Artifactory config
## Supported types are:
## file-system (default)
## cluster-file-system
## nfs
## google-storage
## google-storage-v2
## cluster-google-storage-v2
## aws-s3-v3
## s3-storage-v3-direct
## cluster-s3-storage-v3
## azure-blob
## azure-blob-storage-direct
## cluster-azure-blob-storage
type: aws-s3-v3

## Use binarystoreXml to provide a custom binarystore.xml
## This is intentionally commented and below previous content of binarystoreXml is moved under files/binarystore.xml
## binarystoreXml:

## For artifactory.persistence.type nfs
## If using NFS as the shared storage, you must have a running NFS server that is accessible by your Kubernetes
## cluster nodes.
## Need to have the following set
nfs:
  # Must pass actual IP of NFS server with '--set For artifactory.persistence.nfs.ip=${NFS_IP}'
  ip:
  haDataMount: "/data"
  haBackupMount: "/backup"
  dataDir: "/var/opt/jfrog/artifactory"
  backupDir: "/var/opt/jfrog/artifactory-backup"
  capacity: 500Gi
awsS3V3:
  testConnection: true
  identity: 2O0FS4D3XX2V3XIULxxx
  credential: rvLgggDf9QmUeORCBwnnDJ9gt3HjD9Mrxxxxx
  region: us-east-1
  bucketName: artifactory
  path: artifactory/filestore
  endpoint: rook-ceph-rgw-my-store-obj.rook-ceph
  maxConnections: 50
  kmsServerSideEncryptionKeyId:
  kmsKeyRegion:
  kmsCryptoMode:
  useInstanceCredentials: false
  usePresigning: false
  signatureExpirySeconds: 300
  signedUrlExpirySeconds: 30
  cloudFrontDomainName:
  cloudFrontKeyPairId:
  cloudFrontPrivateKey:
  enableSignedUrlRedirect: false
  enablePathStyleAccess: false

narwal@rd-k8s-ceph-test-master-04:~/jfrog/artifactory-oss$ kubectl get statefulsets.apps -n artifactory-oss artifactory-oss-1667962421 -o yaml apiVersion: apps/v1 kind: StatefulSet metadata: annotations: meta.helm.sh/release-name: artifactory-oss-1667962421 meta.helm.sh/release-namespace: artifactory-oss creationTimestamp: "2022-11-09T02:53:43Z" generation: 2 labels: app: artifactory app.kubernetes.io/managed-by: Helm chart: artifactory-107.46.11 component: artifactory databaseUpgradeReady: "yes" heritage: Helm release: artifactory-oss-1667962421 name: artifactory-oss-1667962421 namespace: artifactory-oss resourceVersion: "108890810" uid: 695fb0dc-eaa9-42dc-baa4-5a0e36cd19e0 spec: podManagementPolicy: OrderedReady replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: app: artifactory release: artifactory-oss-1667962421 role: artifactory serviceName: artifactory template: metadata: annotations: checksum/access-config: 4238695fead7796f691b71efaca2e260e8e92f1fd7b5e4755f8cb0bab57f2227 checksum/admin-creds: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b checksum/binarystore: b9fdbd65000d46d48fb264981adec8c1c5636288fa8b47ecbca5b19b78e8704c checksum/database-secrets: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b checksum/systemyaml: 274487d37e4cad59c753db30277d5c54c16bcd0f092ce1aef59f6efeb96999dc creationTimestamp: null labels: app: artifactory chart: artifactory-107.46.11 component: artifactory heritage: Helm release: artifactory-oss-1667962421 role: artifactory spec: affinity: podAntiAffinity: preferredDuringSchedulingIgnoredDuringExecution:

artifactory log: 2022-11-09T03:50:41.339Z [jfrou] [FATAL] [2d0e6acda77764c6] [bootstrap.go:99 ] [main ] [] - Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key 2022-11-09T03:50:41.756Z [jfmd ] [ERROR] [ ] [keys.go:23 ] [main ] [] - Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key goroutine 1 [running]: runtime/debug.Stack() /src/runtime/debug/stack.go:24 +0x65 jfrog.com/jfrog-go-commons/v7/pkg/log.(standardLogger).Panicfc(0xc0003440c0, {0x1a78140, 0xc000597b30}, {0xc00001e0a0, 0x96}, {0x0, 0x0, 0x0}) goroot/pkg/mod/jfrog.com/jfrog-go-commons/v7@v7.58.0/pkg/log/standard_logger.go:95 +0xd2 jfrog.com/metadata/v7/services/common.MustResolveSecurityKeys({0x1a78140, 0xc000597b30}, {0x1a7c0f0, 0xc0000c9f40}, {0x1a801a8, 0xc0003440c0?}) jfrog.com/metadata/v7@v7.48.2/services/common/keys.go:23 +0x1ca main.main() jfrog.com/metadata/v7@v7.48.2/metadata.go:31 +0x345 [init] panic: Failed resolving master key: failed resolving 'shared.security.masterKey' key; file does not exist: /opt/jfrog/artifactory/var/etc/security/master.key goroutine 1 [running]: runtime/debug.Stack() /src/runtime/debug/stack.go:24 +0x65 jfrog.com/jfrog-go-commons/v7/pkg/log.(standardLogger).Panicfc(0xc0003440c0, {0x1a78140, 0xc000597b30}, {0xc00001e0a0, 0x96}, {0x0, 0x0, 0x0}) goroot/pkg/mod/jfrog.com/jfrog-go-commons/v7@v7.58.0/pkg/log/standard_logger.go:95 +0xd2 jfrog.com/metadata/v7/services/common.MustResolveSecurityKeys({0x1a78140, 0xc000597b30}, {0x1a7c0f0, 0xc0000c9f40}, {0x1a801a8, 0xc0003440c0?}) jfrog.com/metadata/v7@v7.48.2/services/common/keys.go:23 +0x1ca main.main() jfrog.com/metadata/v7@v7.48.2/metadata.go:31 +0x345

goroutine 1 [running]: github.com/rs/zerolog.(Logger).Panic.func1({0xc000050900?, 0x0?}) goroot/pkg/mod/github.com/rs/zerolog@v1.27.0/log.go:359 +0x2d github.com/rs/zerolog.(Event).msg(0xc0006281e0, {0xc000050900, 0x2e6}) goroot/pkg/mod/github.com/rs/zerolog@v1.27.0/event.go:156 +0x2a5 github.com/rs/zerolog.(Event).Msgf(0xc0006281e0, {0xc00001e140?, 0x2754cb8?}, {0xc000475d38?, 0x2756ec0?, 0xc00001e140?}) goroot/pkg/mod/github.com/rs/zerolog@v1.27.0/event.go:129 +0x4e jfrog.com/jfrog-go-commons/v7/pkg/log.(standardLogger).logMessage(0xc0003440c0, {0x1a78140, 0xc000597b30}, 0x8?, {0xc00001e140, 0x99}, {0xc000475d38, 0x1, 0x1}) goroot/pkg/mod/jfrog.com/jfrog-go-commons/v7@v7.58.0/pkg/log/standard_logger.go:118 +0x245 jfrog.com/jfrog-go-commons/v7/pkg/log.(*standardLogger).Panicfc(0xc0003440c0, {0x1a78140, 0xc000597b30}, {0xc00001e0a0, 0x96}, {0x0, 0x0, 0x0}) goroot/pkg/mod/jfrog.com/jfrog-go-commons/v7@v7.58.0/pkg/log/standard_logger.go:96 +0x1d0 jfrog.com/metadata/v7/services/common.MustResolveSecurityKeys({0x1a78140, 0xc000597b30}, {0x1a7c0f0, 0xc0000c9f40}, {0x1a801a8, 0xc0003440c0?}) jfrog.com/metadata/v7@v7.48.2/services/common/keys.go:23 +0x1ca main.main() jfrog.com/metadata/v7@v7.48.2/metadata.go:31 +0x345