search

oracle / docker-images

Official source of container configurations, images, and examples for Oracle products and projects
https://developer.oracle.com/use-cases/#containers
Universal Permissive License v1.0
6.54k stars 5.42k forks source link

Error deploying OracleRealApplicationClusters #2826

Open pushpi55 opened 3 months ago

pushpi55 commented 3 months ago

I am following the steps mentioned in the github page to create racnode https://github.com/oracle/docker-images/tree/main/OracleDatabase/RAC/OracleRealApplicationClusters I installed all the prerequisite conatiners, i.e DNS and Storage containers for NFS volumes. docker start racnoded1 is giving the following error

07-03-2024 07:55:07 UTC : : Generating DB Responsefile Running DB creation 07-03-2024 07:55:07 UTC : : Running DB creation 07-03-2024 07:55:07 UTC : : Workaround for Bug 32449232 : Removing /u01/app/grid/kfod 07-03-2024 07:55:13 UTC : : Checking DB status 07-03-2024 07:55:13 UTC : : ORCLCDB is not up and running on racnoded1 07-03-2024 07:55:13 UTC : : Error has occurred in Grid Setup, Please verify

Upon delving more in the installation logs, found that the img files generated by Storage container, does not sufficient permissions and they are not part of proper group and user.

Access Control List check ...FAILED PRVF-9991 : Owner of device "/oradata/asm_disk01.img" did not match the expected owner. [Expected = "grid"; Found = "oracle"] on nodes: [racnoded1] PRVF-9991 : Owner of device "/oradata/asm_disk02.img" did not match the expected owner. [Expected = "grid"; Found = "oracle"] on nodes: [racnoded1] PRVF-9991 : Owner of device "/oradata/asm_disk03.img" did not match the expected owner. [Expected = "grid"; Found = "oracle"] on nodes: [racnoded1] PRVF-9991 : Owner of device "/oradata/asm_disk05.img" did not match the expected owner. [Expected = "grid"; Found = "oracle"] on nodes: [racnoded1] PRVF-9991 : Owner of device "/oradata/asm_disk04.img" did not match the expected owner. [Expected = "grid"; Found = "oracle"] on nodes: [racnoded1] PRVF-9992 : Group of device "/oradata/asm_disk01.img" did not match the expected group. [Expected = "asmadmin"; Found = "oinstall"] on nodes: [racnoded1] PRVF-9992 : Group of device "/oradata/asm_disk02.img" did not match the expected group. [Expected = "asmadmin"; Found = "oinstall"] on nodes: [racnoded1] PRVF-9992 : Group of device "/oradata/asm_disk03.img" did not match the expected group. [Expected = "asmadmin"; Found = "oinstall"] on nodes: [racnoded1] PRVF-9992 : Group of device "/oradata/asm_disk05.img" did not match the expected group. [Expected = "asmadmin"; Found = "oinstall"] on nodes: [racnoded1] PRVF-9992 : Group of device "/oradata/asm_disk04.img" did not match the expected group. [Expected = "asmadmin"; Found = "oinstall"] on nodes: [racnoded1]

There are other failures as well. Pasting it here

Pre-check for cluster services setup was unsuccessful on all the nodes.

Failures were encountered during execution of CVU verification request "stage -pre crsinst".

Swap Size ...FAILED racnoded1: PRVF-7573 : Sufficient swap size is not available on node "racnoded1" [Required = 16GB (1.6777216E7KB) ; Found = 0.0 bytes]

Device Checks for ASM ...FAILED ASM device sharedness check ...FAILED Shared Storage Accessibility:/oradata/asm_disk01.img,/oradata/asm_disk02.img,/oradata/asm_d isk03.img,/oradata/asm_disk05.img,/oradata/asm_disk04.img ...FAILED racnoded1: PRVG-0809 : Mount options for file system "dbdevfssmnt-prod01vcn.com:/hom es/prod/test/docker_volumes/asm_vol/ORCLCDB" mounted on path "/oradata/asm_disk01.img" at node "racnoded1" did not meet the requirements for this platform [Expected = "rw|!ro,hard|!soft,rsize>=32768,wsize>=32768,tcp|proto=tcp,v3|nfs v3|vers=3|nfsvers=3|v4|nfsv4|vers=4|nfsvers=4|vers=4.0|vers=4.1|n fsvers=4.1,timeo>=600,acregmin=0&acregmax=0&acdirmin=0&acdirmax=0 |actimeo=0" ; Found = "rw,nosuid,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255 ,hard,noacl,proto=tcp,timeo=600,retrans=3,sec=sys,mountaddr=100.7 0.122.116,mountvers=3,mountport=2048,mountproto=tcp,local_lock=no ne,addr=x.y.z.a"]

created the NFS storage using the container, command used. 

          docker run -d -t --hostname racnode-storage --dns-search=example.com  --cap-add SYS_ADMIN --cap-add AUDIT_WRITE --volume /home/test/docker_volumes/asm_vol/$ORACLE_DBNAME:/oradata --init --network=rac_priv1_nw --ip=192.168.17.80 --tmpfs=/run  --volume /sys/fs/cgroup:/sys/fs/cgroup:ro --name racnode-storage oracle/rac-storage-server:19.3.0

Used the following command to create RACnode

docker create -t -i --hostname racnoded1 --volume /boot:/boot:ro --volume /dev/shm --tmpfs /dev/shm:rw,exec,size=4G --volume /opt/containers/rac_host_file:/etc/hosts --volume /opt/.secrets:/run/secrets:ro --dns=172.16.1.25 --dns-search=example.com --privileged=false --volume /home/test/docker_volumes/asm_vol/ORCLCDB:/oradata --cap-add=SYS_NICE --cap-add=SYS_RESOURCE --cap-add=NET_ADMIN -e DNS_SERVERS="172.16.1.25" -e NODE_VIP=172.16.1.130 -e VIP_HOSTNAME=racnoded1-vip -e PRIV_IP=192.168.17.100 -e PRIV_HOSTNAME=racnoded1-priv -e PUBLIC_IP=172.16.1.100 -e PUBLIC_HOSTNAME=racnoded1 -e SCAN_NAME=racnodedc1-scan -e OP_TYPE=INSTALL -e DOMAIN=example.com -e ASM_DISCOVERY_DIR=/oradata -e ASM_DEVICE_LIST=/oradata/asm_disk01.img,/oradata/asm_disk02.img,/oradata/asm_disk03.img,/oradata/asm_disk04.img,/oradata/asm_disk05.img -e CMAN_HOSTNAME=racnodedc1-cman -e CMAN_IP=172.16.1.164 -e COMMON_OS_PWD_FILE=common_os_pwdfile.enc -e PWD_KEY=pwd.key -e RESET_FAILED_SYSTEMD="true" --restart=always --tmpfs=/run -v /sys/fs/cgroup:/sys/fs/cgroup:ro --cpu-rt-runtime=95000 --ulimit rtprio=99 --name racnoded1 oracle/database-rac:21.3.0

Let me know if you need any other info Docker info

docker info Client: Debug Mode: false Server: Containers: 3 Running: 2 Paused: 0 Stopped: 1 Images: 142 Server Version: 19.03.11-ol Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: false Logging Driver: json-file Cgroup Driver: cgroupfs Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: runc Default Runtime: runc Init Binary: docker-init containerd version: 7eba5930496d9bbe375fdf71603e610ad737d2b2 runc version: 51d5e94 init version: fec3683 Security Options: seccomp Profile: default Kernel Version: 5.4.17-2136.330.7.1.el7uek.x86_64 Operating System: Oracle Linux Server 7.9 OSType: linux Architecture: x86_64 CPUs: 4 Total Memory: 31.06GiB Name: phoenix526501 ID: JHPS:YBTM:3KDE:TXA5:JBGL:WYYL:FJVT:5EPW:LGEH:DG2Z:7MME:XCSN Docker Root Dir: /scratch/docker Debug Mode: false HTTP Proxy: http://internalproxy HTTPS Proxy: http://internalproxy No Proxy: localhost,127.0.0.1, Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false Registries:

saurabhuja commented 3 months ago

Please give the output of below-

  1. docker logs -f rac-dnsserver
  2. docker logs racnode-storage
  3. docker ps -a
  4. docker network ls
  5. docker volume ls
  6. docker volume inspect racstorage
  7. ls -lart /docker_volumes/asm_vol/$ORACLE_DBNAME/
saurabhuja commented 3 months ago

Looks like u r not following document , why do u have -volume /home/test/docker_volumes/asm_vol/ORCLCDB:/oradata , it has to come from docker volume racstorage. Can you try using it?

saurabhuja commented 3 months ago

I followed below commands and it went fine in OL7.9

docker network create --driver=bridge --subnet=172.16.1.0/24 rac_pub1_nw docker network create --driver=bridge --subnet=192.168.17.0/24 rac_pzriv1_nw docker create --hostname racdns --dns-search=example.com --cap-add=AUDIT_WRITE -e DOMAIN_NAME="example.com" -e WEBMIN_ENABLED=false -e RAC_NODE_NAME_PREFIXD="racnoded" -e SETUP_DNS_CONFIG_FILES="setup_true" --privileged=false --name rac-dnsserver oracle/rac-dnsserver:latest docker network disconnect bridge rac-dnsserver docker network connect rac_pub1_nw --ip 172.16.1.25 rac-dnsserver docker network connect rac_pzriv1_nw --ip 192.168.17.25 rac-dnsserver docker start rac-dnsserver docker logs rac-dnsserver export ORACLE_DBNAME=ORCLCDB mkdir -p /docker_volumes/asm_vol/$ORACLE_DBNAME rm -rf /docker_volumes/asm_vol/$ORACLE_DBNAME/asm_disk0* docker run -d -t --hostname racnode-storage --dns-search=example.com --cap-add SYS_ADMIN --cap-add AUDIT_WRITE --volume /docker_volumes/asm_vol/$ORACLE_DBNAME:/oradata --init --network=rac_pzriv1_nw --ip=192.168.17.80 --tmpfs=/run --volume /sys/fs/cgroup:/sys/fs/cgroup:ro --name racnode-storage oracle/rac-storage-server:19.3.0 docker logs -f racnode-storage docker volume ls docker volume rm racstorage docker volume create --driver local --opt type=nfs --opt o=addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0 --opt device=192.168.17.80:/oradata racstorage docker create -t -i --hostname racnoded1 --volume /boot:/boot:ro --volume /dev/shm --tmpfs /dev/shm:rw,exec,size=4G --volume /opt/containers/rac_host_file:/etc/hosts --volume /opt/.secrets:/run/secrets:ro --dns=172.16.1.25 --dns-search=example.com --privileged=false --volume racstorage:/oradata --cap-add=SYS_NICE --cap-add=SYS_RESOURCE --cap-add=NET_ADMIN -e DNS_SERVERS="172.16.1.25" -e NODE_VIP=172.16.1.130 -e VIP_HOSTNAME=racnoded1-vip -e PRIV_IP=192.168.17.100 -e PRIV_HOSTNAME=racnoded1-priv -e PUBLIC_IP=172.16.1.100 -e PUBLIC_HOSTNAME=racnoded1 -e SCAN_NAME=racnodedc1-scan -e OP_TYPE=INSTALL -e DOMAIN=example.com -e ASM_DISCOVERY_DIR=/oradata -e ASM_DEVICE_LIST=/oradata/asm_disk01.img,/oradata/asm_disk02.img,/oradata/asm_disk03.img,/oradata/asm_disk04.img,/oradata/asm_disk05.img -e CMAN_HOSTNAME=racnodedc1-cman -e CMAN_IP=172.16.1.164 -e COMMON_OS_PWD_FILE=common_os_pwdfile.enc -e PWD_KEY=pwd.key -e RESET_FAILED_SYSTEMD="true" --restart=always --tmpfs=/run -v /sys/fs/cgroup:/sys/fs/cgroup:ro --cpu-rt-runtime=95000 --ulimit rtprio=99 --memory 16G --memory-swap 32G --name racnoded1 oracle/database-rac:21.3.0 docker network disconnect bridge racnoded1 docker network connect rac_pub1_nw --ip 172.16.1.100 racnoded1 docker network connect rac_pzriv1_nw --ip 192.168.17.100 racnoded1 docker start racnoded1 docker logs -f racnoded1 07-04-2024 08:00:13 UTC : : #################################### 07-04-2024 08:00:13 UTC : : ORACLE RAC DATABASE IS READY TO USE! 07-04-2024 08:00:13 UTC : : ####################################

saurabhuja commented 3 months ago

your docker volume inspect racstorage shall come like below-

docker volume inspect racstorage [ { "CreatedAt": "2024-07-04T07:31:22Z", "Driver": "local", "Labels": {}, "Mountpoint": "/var/lib/docker/volumes/racstorage/_data", "Name": "racstorage", "Options": { "device": "192.168.17.80:/oradata", "o": "addr=192.168.17.80,rw,bg,hard,tcp,vers=3,timeo=600,rsize=32768,wsize=32768,actimeo=0", "type": "nfs" }, "Scope": "local" } ]