run as non root

[hishamanver]

as per title

we have issues running this container on openshift due to security constraints (running on openshift - https://cloud.redhat.com/blog/a-guide-to-openshift-and-uids)

converting this to use non root user without needing to pin a uid

struggled to build the image as is so tested with the following

Test Dockerfile

FROM harbor.tools.telstra.com/public-cache/natsio/nats-box:0.14.1

RUN mkdir -p /opt && \
    chmod g+rw /opt

ENV NKEYS_PATH /opt/nsc/nkeys
ENV XDG_DATA_HOME /opt/nsc
ENV XDG_CONFIG_HOME /opt/nsc/.config

WORKDIR /opt
USER 1001

Results (on docker):

docker run -it --rm  harbor.tools.telstra.com/automation-tools/nats-box:test1   
             _             _               
 _ __   __ _| |_ ___      | |__   _____  __
| '_ \ / _` | __/ __|_____| '_ \ / _ \ \/ /
| | | | (_| | |_\__ \_____| |_) | (_) >  < 
|_| |_|\__,_|\__|___/     |_.__/ \___/_/\_\

nats-box v0.14.1
3d3628d24152:/opt$ nsc
nsc creates NATS operators, accounts, users, and manage their permissions.

Usage:
  nsc [flags]
  nsc [command]

Available Commands:
  add         Add assets such as accounts, imports, users
  completion  Generate the autocompletion script for the specified shell
  delete      Delete imports and exports
  describe    Describe assets such as operators, accounts, users, and jwt files
  edit        Edit assets such as accounts, imports, and users
  env         Prints and manage the nsc environment
  export      Export assets such as nkeys
  generate    Generate activations, creds, configs or nkeys
  help        Help about any command
  import      Import assets such as nkeys
  init        Initialize an environment by creating an operator, account and user
  keys        Manage keys for operators, accounts, and users
  list        List assets such as accounts, imports, users
  load        install entities for an operator, account and key
  pull        Pull an operator or account jwt replacing the local jwt with the server's version
  push        Push an account jwt to an Account JWT Server
  reissue     Re-issue objects with a new identity key
  revocations Manage revocation for users and activations from an account
  select      Set the current operator or account
  tool        NATS tools: pub, sub, req, rep, rtt
  update      Update this tool to latest version
  validate    Validate an operator, account(s), and users

Flags:
  -H, --all-dirs string       sets --config-dir, --data-dir, and --keystore-dir to the same value
      --config-dir string     nsc config directory
      --data-dir string       nsc data store directory
  -h, --help                  help for nsc
  -i, --interactive           ask questions for various settings
      --keystore-dir string   nsc keystore directory
  -K, --private-key string    Key used to sign. Can be specified as role (where applicable),
                              public key (private portion is retrieved)
                              or file path to a private key or private key 
  -v, --version               version for nsc

Use "nsc [command] --help" for more information about a command.
3d3628d24152:/opt$ find .
.
./nsc
./nsc/.config
./nsc/.config/nats
./nsc/.config/nats/nsc
./nsc/.config/nats/nsc/nsc.json
./nsc/nats
./nsc/nats/nsc
./nsc/nats/nsc/stores
3d3628d24152:/opt$ 

Results (on openshift)

kubectl exec -it pov-system-nats-box-5d4c8c98bd-mslqb sh
kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
/opt $ nsc
nsc creates NATS operators, accounts, users, and manage their permissions.

Usage:
  nsc [flags]
  nsc [command]

Available Commands:
  add         Add assets such as accounts, imports, users
  completion  Generate the autocompletion script for the specified shell
  delete      Delete imports and exports
  describe    Describe assets such as operators, accounts, users, and jwt files
  edit        Edit assets such as accounts, imports, and users
  env         Prints and manage the nsc environment
  export      Export assets such as nkeys
  generate    Generate activations, creds, configs or nkeys
  help        Help about any command
  import      Import assets such as nkeys
  init        Initialize an environment by creating an operator, account and user
  keys        Manage keys for operators, accounts, and users
  list        List assets such as accounts, imports, users
  load        install entities for an operator, account and key
  pull        Pull an operator or account jwt replacing the local jwt with the server's version
  push        Push an account jwt to an Account JWT Server
  reissue     Re-issue objects with a new identity key
  revocations Manage revocation for users and activations from an account
  select      Set the current operator or account
  tool        NATS tools: pub, sub, req, rep, rtt
  update      Update this tool to latest version
  validate    Validate an operator, account(s), and users

Flags:
  -H, --all-dirs string       sets --config-dir, --data-dir, and --keystore-dir to the same value
      --config-dir string     nsc config directory
      --data-dir string       nsc data store directory
  -h, --help                  help for nsc
  -i, --interactive           ask questions for various settings
      --keystore-dir string   nsc keystore directory
  -K, --private-key string    Key used to sign. Can be specified as role (where applicable),
                              public key (private portion is retrieved)
                              or file path to a private key or private key 
  -v, --version               version for nsc

Use "nsc [command] --help" for more information about a command.
/opt $ ls -al
total 0
drwxrwxr-x    1 root     root            17 Oct 26 07:35 .
dr-xr-xr-x    1 root     root            55 Oct 26 07:35 ..
drwxr-xr-x    4 10012600 root            33 Oct 26 07:35 nsc
/opt $ find .
.
./nsc
./nsc/.config
./nsc/.config/nats
./nsc/.config/nats/context
./nsc/.config/nats/context.txt
./nsc/.config/nats/nsc
./nsc/.config/nats/nsc/nsc.json
./nsc/nats
./nsc/nats/nsc
./nsc/nats/nsc/stores

[sboulkour]

Same issue for me. With current image, I get the following error (only log from nats-box container I get) : mkdir: can't create directory '/nsc/': Permission denied

[wallyqs]

would using runAsUser be an alternative here?

spec:
  securityContext:
    runAsUser: 1000
    runAsGroup: 1000

[hishamanver]

Unfortunately not, as openshift assigns a random uid from a pool allocated by cluster administrators that we have no control over.

[wallyqs]

@hishamanver interesting, thanks for clarifying. Then I guess there would be a similar issue happen when running the nats-server official image and other components...

[hishamanver]

Oddly not, it just works 🤔

[wallyqs]

is that with the nats:2.10.7-alpine or using the scratch nats:2.10.7 image?

[hishamanver]

nats:2.10.3-alpine which is used by the helm chart here https://github.com/nats-io/k8s/blob/nats-1.1.2/helm/charts/nats/values.yaml#L310-L311

[sboulkour]

@wallyqs my error while running nats-box is due to FS access rights, see my earlier comment. I too use OpenShift as K8s distribution and its hardenned environnement.

Main nats image may not have this problem as its not trying to write in / directly, and uses a PVC for JetStream datas. Im new to NATS so please correct if Im wrong here. I can confirm nats-2.10-alpine just works for me too.

[sboulkour]

@hishamanver See PR https://github.com/nats-io/nats-box/pull/65 couldnt test yet, but this shoul fix our Openshift specific issue.

[hishamanver]

@hishamanver See PR #65 couldnt test yet, but this shoul fix our Openshift specific issue.

@sboulkour Just tested on openshift and this does not work, I get the following log from the failed container in openshift:

trap true INT TERM; sleep infinity & wait: cd: line 9: can't cd to /root: Permission denied

when running the container locally using docker and emulating a random user (which is what openshift does) I can understand why its failing:

# normal non-root execution as default nats user
$docker run --rm -it  harbor.tools.telstra.com/public-cache/natsio/nats-box:0.14.2-nonroot 
             _             _               
 _ __   __ _| |_ ___      | |__   _____  __
| '_ \ / _` | __/ __|_____| '_ \ / _ \ \/ /
| | | | (_| | |_\__ \_____| |_) | (_) >  < 
|_| |_|\__,_|\__|___/     |_.__/ \___/_/\_\

nats-box v0.14.2
bb267464d043:~$ id
uid=1000(nats) gid=1000(nats) groups=1000(nats)
bb267464d043:~$ echo $HOME
/home/nats
bb267464d043:~$ 

# custom non-root execution as arbitrary user
$docker run --rm -it -u 100000001:100000001 harbor.tools.telstra.com/public-cache/natsio/nats-box:0.14.2-nonroot
             _             _               
 _ __   __ _| |_ ___      | |__   _____  __
| '_ \ / _` | __/ __|_____| '_ \ / _ \ \/ /
| | | | (_| | |_\__ \_____| |_) | (_) >  < 
|_| |_|\__,_|\__|___/     |_.__/ \___/_/\_\

nats-box v0.14.2
37a7a7106231:~$ pwd
/
37a7a7106231:~$ echo $HOME
/
37a7a7106231:~$ 

It is due to the fact that 0.14.2 is not up to date with main https://github.com/nats-io/nats-box/compare/v0.14.2...main. We need the WORKDIR step in order to get this to not take effect https://github.com/nats-io/nats-box/blob/main/entrypoint.sh#L3-L5

Once the image is rebuilt from main things should work, however for a bit of housekeeping I have updated the MR to clean up the redundant step in the entrypoint file, feel free to close this if not required

@wallyqs any chance we can get an updated release with the updates from main?

[hishamanver]

fixed in https://github.com/nats-io/nats-box/releases/tag/v0.14.3