search

jfrog / charts

JFrog official Helm Charts
https://jfrog.com/integration/helm-repository/
Apache License 2.0
259 stars 448 forks source link

Cannot write to /opt/jfrog/router/app/router/router.pid: Permission denied #1857

Closed oniabifo closed 3 months ago

oniabifo commented 9 months ago

Artifactory installed from official helm chart is not running due to a permission denied issue Screenshot 2024-01-30 at 1 55 02 PM

Is this a request for help?: Yes

Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT

Version of Helm and Kubernetes: 107.77.3

Which chart: artifactory

Which product license (Enterprise/Pro/oss):

JFrog support reference (if already raised with support team):

What happened: Artifactory is not running on kubernetes

What you expected to happen: Artifactory UP and running.

How to reproduce it (as minimally and precisely as possible): install helm chart with mentioned version and default values.

Anything else we need to know: Our kubernetes clusters prevent apps from writing on filesystem

chukka commented 9 months ago

@oniabifo Thanks for reaching out ! we need more details to reproduce . Can you share below information ?

  1. kubernestes flavour and version ( is it openshift ,if yes , what is the version used)
  2. helm command used
  3. complete console logs
oniabifo commented 9 months ago

Hello @chukka Thanks for reaching out

kubernestes flavour and version v1.21.5 and it is on AWS

helm command used This is the helm command used

helm upgrade --install artifactory jfrog/artifactory \
      --set nginx.service.ssloffload=$SSL_OFFLOAD \
      --set nginx.tlsSecretName=tls-ingress \
      --set artifactory.masterKey=$MASTER_KEY \
      --set artifactory.joinKey=$JOIN_KEY \
      --set artifactory.license.secret=artifactory-license \
      --set artifactory.persistence.enabled=false \
      --set artifactory.nginx.enabled=false \
      --set artifactory.uid=1000 \
      --set artifactory.gid=1000 \
      --set artifactory.license.dataKey=$ARTIFACTORY_DATA_KEY $NAMESPACE_COMMAND

complete console logs

This is the output of kubectl logs podname Testing directory /var/opt/jfrog/router has read/write permissions for user id 1000 Permissions for /var/opt/jfrog/router are good Setting JF_SHARED_NODE_ID to artifactory-0 Setting JF_SHARED_NODE_IP to 10.98.136.72 Setting JF_SHARED_NODE_NAME to artifactory-0 Using default router's certificate and private key ======= Router PID 265 /opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied runtime/cgo: pthread_create failed: Operation not permitted SIGABRT: abort PC=0x7f00665a054c m=0 sigcode=18446744073709551610

goroutine 0 [idle]: runtime: g 0: unknown pc 0x7f00665a054c stack: frame={sp:0x7ffccb238fa0, fp:0x0} stack=[0x7ffccaa3a520,0x7ffccb239530) 0x00007ffccb238ea0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238eb0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ec0: 0x6e696c2d72657475 0x3436646d612d7875 0x00007ffccb238ed0: 0x454e524542554b00 0x565245535f534554 0x00007ffccb238ee0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ef0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f00: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f10: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f20: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f30: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f40: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f50: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f60: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f70: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f80: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f90: 0x0000000000000000 0x00007f00665a053e 0x00007ffccb238fa0: <0x0000000000000000 0x0000000000000000 0x00007ffccb238fb0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fc0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fd0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fe0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ff0: 0x0000000000000000 0x0000000000000000 0x00007ffccb239000: 0x0000000000000000 0x0000000000000000 0x00007ffccb239010: 0x0000000000000000 0x0000000000000000 0x00007ffccb239020: 0x0000000000000000 0xf8b108eb38f38600 0x00007ffccb239030: 0x00007f00664fc740 0x0000000000000006 0x00007ffccb239040: 0x000000000856b2e0 0x0000000000000000 0x00007ffccb239050: 0x000000000729da00 0x00007f0066553d06 0x00007ffccb239060: 0x00007f00666fae90 0x00007f00665277f3 0x00007ffccb239070: 0x0000000000000020 0x0000000000000000 0x00007ffccb239080: 0x0000000000000000 0x0000000000000000 0x00007ffccb239090: 0x0000000000000000 0x0000000000000000 runtime: g 0: unknown pc 0x7f00665a054c stack: frame={sp:0x7ffccb238fa0, fp:0x0} stack=[0x7ffccaa3a520,0x7ffccb239530) 0x00007ffccb238ea0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238eb0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ec0: 0x6e696c2d72657475 0x3436646d612d7875 0x00007ffccb238ed0: 0x454e524542554b00 0x565245535f534554 0x00007ffccb238ee0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ef0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f00: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f10: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f20: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f30: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f40: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f50: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f60: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f70: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f80: 0x0000000000000000 0x0000000000000000 0x00007ffccb238f90: 0x0000000000000000 0x00007f00665a053e 0x00007ffccb238fa0: <0x0000000000000000 0x0000000000000000 0x00007ffccb238fb0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fc0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fd0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238fe0: 0x0000000000000000 0x0000000000000000 0x00007ffccb238ff0: 0x0000000000000000 0x0000000000000000 0x00007ffccb239000: 0x0000000000000000 0x0000000000000000 0x00007ffccb239010: 0x0000000000000000 0x0000000000000000 0x00007ffccb239020: 0x0000000000000000 0xf8b108eb38f38600 0x00007ffccb239030: 0x00007f00664fc740 0x0000000000000006 0x00007ffccb239040: 0x000000000856b2e0 0x0000000000000000 0x00007ffccb239050: 0x000000000729da00 0x00007f0066553d06 0x00007ffccb239060: 0x00007f00666fae90 0x00007f00665277f3 0x00007ffccb239070: 0x0000000000000020 0x0000000000000000 0x00007ffccb239080: 0x0000000000000000 0x0000000000000000 0x00007ffccb239090: 0x0000000000000000 0x0000000000000000

goroutine 1 [running]: runtime.systemstack_switch() runtime/asm_amd64.s:474 +0x8 fp=0xc0000a6740 sp=0xc0000a6730 pc=0x5f0448 runtime.main() runtime/proc.go:169 +0x6d fp=0xc0000a67e0 sp=0xc0000a6740 pc=0x5bd4cd runtime.goexit() runtime/asm_amd64.s:1650 +0x1 fp=0xc0000a67e8 sp=0xc0000a67e0 pc=0x5f2421

rax 0x0 rbx 0x7f00664fc740 rcx 0x7f00665a054c rdx 0x6 rdi 0x109 rsi 0x109 rbp 0x109 rsp 0x7ffccb238fa0 r8 0x7ffccb239070 r9 0x7f00666b04e0 r10 0x8 r11 0x246 r12 0x6 r13 0x0 r14 0x729da00 r15 0x6 rip 0x7f00665a054c rflags 0x246 cs 0x33 fs 0x0 gs 0x0

chukka commented 9 months ago

@oniabifo can you let us know the license that you are using Enterprise/Pro/oss ?

oniabifo commented 9 months ago

@chukka I am using the 30 days free trial from the jfrog website for this

chukka commented 9 months ago

@oniabifo I tried with below commands , it works with out any issues , can you try again with latest version 107.77.3

helm repo update jfrog
helm upgrade --install artifactory jfrog/artifactory --set artifactory.uid=1000,artifactory.gid=1000 --namespace test --create-namespace --version 107.77.3
oniabifo commented 9 months ago

@chukka still failing. Also noticed nginx isn't starting up

Using deprecated password for user _internal.
2024-02-01 14:58:17  [169 entrypoint-nginx.sh] Preparing to run Nginx in Docker
2024-02-01 14:58:17   [14 entrypoint-nginx.sh] Dockerfile for this image can found inside the container.
2024-02-01 14:58:17   [15 entrypoint-nginx.sh] To view the Dockerfile: 'cat /docker/nginx-artifactory-pro/Dockerfile.nginx'.
2024-02-01 14:58:17   [19 entrypoint-nginx.sh] Setting up directories if missing
2024-02-01 14:58:17  [160 entrypoint-nginx.sh] Generating ssl example.key and example.crt
2024-02-01 14:58:17   [83 entrypoint-nginx.sh] Artifactory configuration already in /var/opt/jfrog/nginx/conf.d/artifactory.conf
2024-02-01 14:58:17  [177 entrypoint-nginx.sh] Adding logrotate configuration
2024-02-01 14:58:17   [92 entrypoint-nginx.sh] Adding logrotate to crontab
2024-02-01 14:58:17  [187 entrypoint-nginx.sh] Starting updateConf.sh in the background
Using deprecated password for user _internal.
2024-02-01 14:58:17  [192 entrypoint-nginx.sh] Starting nginx daemon...
chukka commented 9 months ago

@oniabifo can you confirm if you are using latest version of chart 107.77.3 ?

Can you provide me output of this command helm list -n $NAMESPACE_COMMAND

oniabifo commented 9 months ago

@chukka it returned nothing.

Screenshot 2024-02-06 at 2 43 40 PM

I also have another question, i pulled this image releases-docker.jfrog.io/jfrog/nginx-artifactory-pro:7.77.3 and i tried running it locally on docker and i noticed it never got past this stage

Using deprecated password for user _internal.
2024-02-06 22:39:52  [169 entrypoint-nginx.sh] Preparing to run Nginx in Docker
2024-02-06 22:39:52   [14 entrypoint-nginx.sh] Dockerfile for this image can found inside the container.
2024-02-06 22:39:52   [15 entrypoint-nginx.sh] To view the Dockerfile: 'cat /docker/nginx-artifactory-pro/Dockerfile.nginx'.
2024-02-06 22:39:52   [19 entrypoint-nginx.sh] Setting up directories if missing
2024-02-06 22:39:52  [160 entrypoint-nginx.sh] Generating ssl example.key and example.crt
2024-02-06 22:39:52   [77 entrypoint-nginx.sh] Unable to find Artifactory configuration (/var/opt/jfrog/nginx/conf.d/artifactory.conf). Copying a default one
2024-02-06 22:39:52  [177 entrypoint-nginx.sh] Adding logrotate configuration
2024-02-06 22:39:52   [92 entrypoint-nginx.sh] Adding logrotate to crontab
2024-02-06 22:39:52  [187 entrypoint-nginx.sh] Starting updateConf.sh in the background
Using deprecated password for user _internal.
2024-02-06 22:39:52  [192 entrypoint-nginx.sh] Starting nginx daemon...

Could there be a reason why? it seems it is running updateConf.sh in the background and there might be something going on in that file

gitta-jfrog commented 8 months ago

Closing, If additional support is required please let us know.

nola-it commented 7 months ago

Has this been closed as a resolution has been provided? Could you clarify as I am getting a similar error while trying to install xRay?

gitta-jfrog commented 7 months ago

The issue closed without resolution as our team wasn't able to reproduce the issue with the information provided. Please open a new ticket with all the details for additional investigation. In addition, We are recommending to reach out JFrog Support team if possible. Thanks!

awolde commented 7 months ago

Hitting this error as well, I have a cluster with kubeadm, k8s version v1.29.3. Closing it without a resolution doesn't make sense @gitta-jfrog . This is the error I'm getting from the router container:

Testing directory /var/opt/jfrog/router has read/write permissions for user id 1000
Permissions for /var/opt/jfrog/router are good
Setting JF_SHARED_NODE_ID to artifactory-0
Setting JF_SHARED_NODE_IP to 192.168.145.218
Setting JF_SHARED_NODE_NAME to artifactory-0
Using default router's certificate and private key
======= Router PID 256
/opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied
2024-04-11T14:20:58.192Z [jfrou] [FATAL] [64938e467fb39198] [home.go:56                    ] [main                ] [] - Could not create service home: stat /opt/jfrog/router/var/data/router: permission denied

Im getting similar issues with the other containers 'permission denied'. I think there is some setup we need to do for the PV. This helm chart expects a pv to exists before it can claim it with pvc. My pv setup is with nfs. Here is my setup:

awolde@k8s-lab-01:~$ k -n jfrog get pvc
NAME                               STATUS   VOLUME         CAPACITY   ACCESS MODES   STORAGECLASS   VOLUMEATTRIBUTESCLASS   AGE
artifactory-volume-artifactory-0   Bound    db-pv-volume   300Gi      RWO,RWX        standard       <unset>                 9h
data-artifactory-postgresql-0      Bound    cc-pv-volume   300Gi      RWO,RWX        standard       <unset>                 9h
awolde@k8s-lab-01:~$ k -n jfrog get pv 
NAME           CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                    STORAGECLASS   VOLUMEATTRIBUTESCLASS   REASON   AGE
cc-pv-volume   300Gi      RWO,RWX        Delete           Bound    jfrog/data-artifactory-postgresql-0      standard       <unset>                          9h
db-pv-volume   300Gi      RWO,RWX        Delete           Bound    jfrog/artifactory-volume-artifactory-0   standard       <unset>                          9h
oumkale commented 7 months ago

Hi @awolde and @oniabifo,

Thank you for raising these concerns. However, we were unable to reproduce them, and it seems the issue stems from Docker on the host. Could you please take a look into below ref and let us know if you need any help?

Reference issue: https://github.com/docker-library/golang/issues/467

Momotoculteur commented 3 months ago

Hello, Same problem here, running on HA version chart, on a EKS (BottlerocketOS)

I was on 107.59.X, chart based on 107.63.X is broken by a malformated yaml chart, so i'm trying to update to 107.68.X

I already had that error on router component :

inconnu

But that was jut a warning, and now it don't want to boot up

┌──────────────────────────────────────────────────────────────────── Logs(jfrog/artifactory-ha-107-77-14-artifactory-ha-primary-0:router)[tail] ────────────────────────────────────────────────────────────────────┐
│                                                                         Autoscroll:On      FullScreen:Off     Timestamps:Off     Wrap:On                                                                           │
│ Testing directory /var/opt/jfrog/router has read/write permissions for user id 1030                                                                                                                                │
│ Permissions for /var/opt/jfrog/router are good                                                                                                                                                                     │
│ Setting JF_SHARED_NODE_ID to artifactory-ha-107-77-14-artifactory-ha-primary-0                                                                                                                                     │
│ Setting JF_SHARED_NODE_IP to 10.207.57.179                                                                                                                                                                         │
│ Setting JF_SHARED_NODE_NAME to artifactory-ha-107-77-14-artifactory-ha-primary-0                                                                                                                                   │
│ Using default router's certificate and private key                                                                                                                                                                 │
│ ======= Router PID 267                                                                                                                                                                                             │
│ /opt/jfrog/router/app/bin/entrypoint-router.sh: line 112: /opt/jfrog/router/app/router/router.pid: Permission denied                                                                                               │
│ 2024-08-07T17:31:26.235Z [jfrou] [WARN ] [                ] [config_holder.go:325          ] [main                ] [] - Could not resolve config value for key 'router.serviceRegistry.accessUrl', although it wa │
│ s just returned as part of all keys... Ignoring.                                                                                                                                                                   │
│ 2024-08-07T17:31:26.235Z [jfrou] [WARN ] [                ] [config_holder.go:325          ] [main                ] [] - Could not resolve config value for key 'router.serviceRegistry.topologyUrl', although it  │
│ was just returned as part of all keys... Ignoring.                                                                                                                                                                 │
│ 2024-08-07T17:31:26.236Z [jfrou] [FATAL] [648ea037a14a2f64] [config_holder.go:50           ] [main                ] [] - Could not initialize configuration: unable to resolve service registry URL (please config │
│ ure the JFrog Platform Deployment URL explicitly, e.g. using the environment variable: JF_SHARED_JFROGURL)                                                                                                         │
│ Stream closed EOF for jfrog/artifactory-ha-107-77-14-artifactory-ha-primary-0 (router)
gitta-jfrog commented 3 months ago

Hi @Momotoculteur

If you are having support as part of your subscription - please open support ticket and our support engineers will work with you up to full resolution.

if you are not having support - I'll ask you to open a new Github issue with all the details (version before, version after upgrade, database type and values yaml) and I'll try to assist as much as I can. Upgrade to newer version (Artifactory 7.68 is already released one year ago is highly recommended, and will make easier for us to provide support)

Thanks Yarden