search

oracle / docker-images

Official source of container configurations, images, and examples for Oracle products and projects
https://developer.oracle.com/use-cases/#containers
Universal Permissive License v1.0
6.58k stars 5.44k forks source link

ORACLE RAC /etc/sysconfig/docker is not picked up by docker daemon #838

Closed largefp2018 closed 6 years ago

largefp2018 commented 6 years ago

vi /etc/sysconfig/docker OPTIONS='--selinux-enabled --cpu-rt-runtime=950000 --cpu-rt-period=1000000 --exec-opt native.cgroupdriver=systemd'

systemctl daemon-reload
systemctl stop docker systemctl start docker systemctl status docker

Main PID: 1911 (dockerd) Memory: 84.4M CGroup: /system.slice/docker.service ├─ 1911 /usr/bin/dockerd

psaini79 commented 6 years ago

Hi,

It seems dockerd daemon is not picking up enteries from /etc/sysconfig/docker. Please try following and it should work:

/usr/lib/systemd/system/docker.service

ExecStart=/usr/bin/dockerd

add command argument: ExecStart=/usr/bin/dockerd --cpu-rt-runtime=950000 --cpu-rt-period=1000000 --exec-opt=native.cgroupdriver=systemd

save the file. systemctl daemon-reload systemctl restart docker

systemctl status docker

You should see dockerd daemon is running with following options: /usr/bin/dockerd --cpu-rt-runtime=950000 --cpu-rt-period=1000000 --exec-opt native.cgroupdriver=systemd

Let me know if you face any issue,

psaini79 commented 6 years ago

Also, if you still see any issue, please do following changes:

echo 950000 > /sys/fs/cgroup/cpu/cpu.rt_runtime_us echo 1000000> /sys/fs/cgroup/cpu/cpu.rt_period_us echo 95000 > /sys/fs/cgroup/cpu,cpuacct/system.slice/cpu.rt_runtime_us

Retstart container and let me know if you face any issue.

Djelibeybi commented 6 years ago

You might want to check with the vendor that provides your Docker support on how to set custom Docker settings.

largefp2018 commented 6 years ago

Hi Djelibeybi,

According to your step, This issue was solved.

Thank you very much.

donjoxie commented 6 years ago

HI Djelibeybi

I have 2 questions: Is this solution a final solution or a workaround? What are the effects of these parameters? I found that after setting these parameters, the cpu usage rate becomes higher in my test env.

/usr/lib/systemd/system/docker.service #ExecStart=/usr/bin/dockerd add command argument: ExecStart=/usr/bin/dockerd --cpu-rt-runtime=950000 --cpu-rt-period=1000000 --exec-opt=native.cgroupdriver=systemd save the file. systemctl daemon-reload systemctl restart docker systemctl status docker You should see dockerd daemon is running with following options: /usr/bin/dockerd --cpu-rt-runtime=950000 --cpu-rt-period=1000000 --exec-opt native.cgroupdriver=systemd Thanks so much!
psaini79 commented 6 years ago

Please provide difference in CPU usage details in both the cases. also, what else are you running on docker host? Did you see the CPU difference after installing RAC on DOCKER or before installation?

psaini79 commented 6 years ago

This workaround is useful to run RT processes inside the container.

donjoxie commented 6 years ago

Hi

Thanks for your replay and sorry for delay.I found CPU% more then 100%,this is normal or not?

[root@docker-host ~]# docker stats CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 120.27% 2.343GiB / 7.795GiB 30.05% 137kB / 186kB 27.7GB / 66.2MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 153.92% 2.336GiB / 7.795GiB 29.97% 137kB / 186kB 27.7GB / 66.3MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 153.92% 2.336GiB / 7.795GiB 29.97% 137kB / 186kB 27.7GB / 66.3MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 168.82% 2.345GiB / 7.795GiB 30.08% 137kB / 186kB 27.7GB / 66.4MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 168.82% 2.345GiB / 7.795GiB 30.08% 137kB / 186kB 27.7GB / 66.4MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 133.56% 2.339GiB / 7.795GiB 30.01% 137kB / 186kB 27.7GB / 66.4MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 133.56% 2.339GiB / 7.795GiB 30.01% 137kB / 186kB 27.7GB / 66.4MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 178.43% 2.339GiB / 7.795GiB 30.01% 137kB / 186kB 27.7GB / 66.8MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 178.43% 2.339GiB / 7.795GiB 30.01% 137kB / 186kB 27.7GB / 66.8MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 181.29% 2.342GiB / 7.795GiB 30.05% 137kB / 191kB 27.7GB / 67MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 181.29% 2.342GiB / 7.795GiB 30.05% 137kB / 191kB 27.7GB / 67MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 120.89% 2.315GiB / 7.795GiB 29.70% 137kB / 204kB 27.8GB / 67MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 120.89% 2.315GiB / 7.795GiB 29.70% 137kB / 204kB 27.8GB / 67MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 189.59% 2.319GiB / 7.795GiB 29.75% 137kB / 204kB 27.8GB / 67.8MB 0 CONTAINER CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS 1b11b69235ed 189.59% 2.319GiB / 7.795GiB 29.75% 137kB / 204kB 27.8GB / 67.8MB 0

Thanks so much! Regards Donjo.Xie

psaini79 commented 6 years ago

Hi, Please paste following output:

docker inspect racnode cat /proc/cpuinfo --> from the host free -g --> from the host

donjoxie commented 6 years ago

[root@docker-host ~]# docker inspect racnode1 [ { "Id": "1b11b69235ed7a401d5859d0b81d661bb9f768617dbb5441db4d9c0e3a03176e", "Created": "2018-04-11T11:19:02.290604613Z", "Path": "/usr/sbin/oracleinit", "Args": [], "State": { "Status": "running", "Running": true, "Paused": false, "Restarting": false, "OOMKilled": false, "Dead": false, "Pid": 1990, "ExitCode": 0, "Error": "", "StartedAt": "2018-05-04T02:55:00.265769705Z", "FinishedAt": "2018-05-03T13:19:44.93820045Z" }, "Image": "sha256:f69b49d67b70ce2a3100ebd106e01ad7aee61ff381102c82fe76265990090944", "ResolvConfPath": "/var/lib/docker/containers/1b11b69235ed7a401d5859d0b81d661bb9f768617dbb5441db4d9c0e3a03176e/resolv.conf", "HostnamePath": "/var/lib/docker/containers/1b11b69235ed7a401d5859d0b81d661bb9f768617dbb5441db4d9c0e3a03176e/hostname", "HostsPath": "/opt/containers/rac_host_file", "LogPath": "/var/lib/docker/containers/1b11b69235ed7a401d5859d0b81d661bb9f768617dbb5441db4d9c0e3a03176e/1b11b69235ed7a401d5859d0b81d661bb9f768617dbb5441db4d9c0e3a03176e-json.log", "Name": "/racnode1", "RestartCount": 0, "Driver": "overlay2", "MountLabel": "", "ProcessLabel": "", "AppArmorProfile": "", "ExecIDs": null, "HostConfig": { "Binds": [ "/boot:/boot:ro", "/opt/containers/rac_host_file:/etc/hosts", "/sys/fs/cgroup:/sys/fs/cgroup:ro" ], "ContainerIDFile": "", "LogConfig": { "Type": "json-file", "Config": {} }, "NetworkMode": "default", "PortBindings": {}, "RestartPolicy": { "Name": "always", "MaximumRetryCount": 0 }, "AutoRemove": false, "VolumeDriver": "", "VolumesFrom": null, "CapAdd": [ "SYS_ADMIN", "SYS_NICE", "SYS_RESOURCE", "NET_ADMIN" ], "CapDrop": null, "Dns": [], "DnsOptions": [], "DnsSearch": [ "example.com" ], "ExtraHosts": null, "GroupAdd": null, "IpcMode": "", "Cgroup": "", "Links": null, "OomScoreAdj": 0, "PidMode": "", "Privileged": false, "PublishAllPorts": false, "ReadonlyRootfs": false, "SecurityOpt": null, "Tmpfs": { "/dev/shm": "rw,exec,size=4G", "/run": "" }, "UTSMode": "", "UsernsMode": "", "ShmSize": 67108864, "Runtime": "runc", "ConsoleSize": [ 0, 0 ], "Isolation": "", "CpuShares": 0, "Memory": 0, "NanoCpus": 0, "CgroupParent": "", "BlkioWeight": 0, "BlkioWeightDevice": null, "BlkioDeviceReadBps": null, "BlkioDeviceWriteBps": null, "BlkioDeviceReadIOps": null, "BlkioDeviceWriteIOps": null, "CpuPeriod": 0, "CpuQuota": 0, "CpuRealtimePeriod": 0, "CpuRealtimeRuntime": 95000, "CpusetCpus": "", "CpusetMems": "", "Devices": [ { "PathOnHost": "/dev/sdb1", "PathInContainer": "/dev/asm_disk1", "CgroupPermissions": "rwm" }, { "PathOnHost": "/dev/sdc1", "PathInContainer": "/dev/asm_disk2", "CgroupPermissions": "rwm" }, { "PathOnHost": "/dev/sdd1", "PathInContainer": "/dev/asm_disk3", "CgroupPermissions": "rwm" }, { "PathOnHost": "/dev/sde1", "PathInContainer": "/dev/asm_disk4", "CgroupPermissions": "rwm" }, { "PathOnHost": "/dev/sdf1", "PathInContainer": "/dev/asm_disk5", "CgroupPermissions": "rwm" } ], "DeviceCgroupRules": null, "DiskQuota": 0, "KernelMemory": 0, "MemoryReservation": 0, "MemorySwap": 0, "MemorySwappiness": -1, "OomKillDisable": false, "PidsLimit": 0, "Ulimits": [ { "Name": "rtprio", "Hard": 99, "Soft": 99 } ], "CpuCount": 0, "CpuPercent": 0, "IOMaximumIOps": 0, "IOMaximumBandwidth": 0 }, "GraphDriver": { "Data": { "LowerDir": "/var/lib/docker/overlay2/33b74fdc902abf82e1860183d95dde6b09bcf570e281deb5241ba9f7b7dc669f-init/diff:/var/lib/docker/overlay2/726d54c3642ac801f8d09de187740335c3e027abd19b4a2f15d62363a1a86cb1/diff:/var/lib/docker/overlay2/e41a85b6509b2a9e32381c1801b39d5bec3c8e7c95c8f1e86ad5c62e58869ab7/diff:/var/lib/docker/overlay2/1332eadb9c31705c73d238282672d6c5ba9d3cbec7b557efe9b234c820c83f21/diff:/var/lib/docker/overlay2/5b4a42913c5c253fe4b1a326e45f58b370fa50ced9b832f012c7817fc123c0fc/diff:/var/lib/docker/overlay2/4b61e6061cd5d01b8b9583d35512fc71f45784046b9d753eb7763376d0a32e08/diff:/var/lib/docker/overlay2/66ff51251f79ba7dadcdb999201808eea4ab6e6c0d12abbe6d225fcc62bf37df/diff:/var/lib/docker/overlay2/487a73b8f200d283ba3a7c1cc86f429a5e248893f8cc3df1ed4dd1ff2d0a4c43/diff:/var/lib/docker/overlay2/6693aab7ef81479eb906947dcf78f7f1258b55a0da6f575f189fef45c8083330/diff:/var/lib/docker/overlay2/c0fc92ed22d47aa1b2a1654c4622974f8a123b950623d5f4b606782f4875a94f/diff", "MergedDir": "/var/lib/docker/overlay2/33b74fdc902abf82e1860183d95dde6b09bcf570e281deb5241ba9f7b7dc669f/merged", "UpperDir": "/var/lib/docker/overlay2/33b74fdc902abf82e1860183d95dde6b09bcf570e281deb5241ba9f7b7dc669f/diff", "WorkDir": "/var/lib/docker/overlay2/33b74fdc902abf82e1860183d95dde6b09bcf570e281deb5241ba9f7b7dc669f/work" }, "Name": "overlay2" }, "Mounts": [ { "Type": "volume", "Name": "be685befb7067054691667a3877ac10fde14e8423bcbc028209ea811c862981f", "Source": "/var/lib/docker/volumes/be685befb7067054691667a3877ac10fde14e8423bcbc028209ea811c862981f/_data", "Destination": "/common_scripts", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "volume", "Name": "30ccdc32c3ebd14f2c43ed120a9efe181621add6e6832418538dd78fa185164d", "Source": "", "Destination": "/dev/shm", "Driver": "local", "Mode": "", "RW": true, "Propagation": "" }, { "Type": "bind", "Source": "/opt/containers/rac_host_file", "Destination": "/etc/hosts", "Mode": "", "RW": true, "Propagation": "rprivate" }, { "Type": "bind", "Source": "/sys/fs/cgroup", "Destination": "/sys/fs/cgroup", "Mode": "ro", "RW": false, "Propagation": "rprivate" }, { "Type": "bind", "Source": "/boot", "Destination": "/boot", "Mode": "ro", "RW": false, "Propagation": "rprivate" } ], "Config": { "Hostname": "racnode1", "Domainname": "", "User": "grid", "AttachStdin": true, "AttachStdout": true, "AttachStderr": true, "Tty": true, "OpenStdin": true, "StdinOnce": true, "Env": [ "NODE_VIP=172.15.1.160", "VIP_HOSTNAME=racnode1-vip", "PRIV_IP=192.168.17.150", "PRIV_HOSTNAME=racnode1-priv", "PUBLIC_IP=172.15.1.150", "PUBLIC_HOSTNAME=racnode1", "SCAN_NAME=racnode-scan", "SCAN_IP=172.15.1.70", "OP_TYPE=INSTALL", "DOMAIN=example.com", "ASM_DEVICE_LIST=/dev/asm_disk1,/dev/asm_disk2,/dev/asm_disk3,/dev/asm_disk4,/dev/asm_disk5", "ORACLE_PWD=Oracle_12c", "ASM_DISCOVERY_DIR=/dev", "OS_PASSWORD=Oracle_12c", "PATH=/bin:/usr/bin:/sbin:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "SETUP_LINUX_FILE=setupLinuxEnv.sh", "INSTALL_DIR=/opt/scripts", "GRID_BASE=/u01/app/grid", "GRID_HOME=/u01/app/12.2.0/grid", "INSTALL_FILE_1=linuxx64_12201_grid_home.zip", "GRID_INSTALL_RSP=grid.rsp", "GRID_SETUP_FILE=setupGrid.sh", "FIXUP_PREQ_FILE=fixupPreq.sh", "INSTALL_GRID_BINARIES_FILE=installGridBinaries.sh", "INSTALL_GRID_PATCH=applyGridPatch.sh", "INVENTORY=/u01/app/oraInventory", "CONFIGGRID=configGrid.sh", "ADDNODE=AddNode.sh", "ADDNODE_RSP=grid_addnode.rsp", "SETUPSSH=setupSSH.expect", "GRID_PATCH=p27383741_122010_Linux-x86-64.zip", "PATCH_NUMBER=27383741", "SETUPDOCKERORACLEINIT=setupdockeroracleinit.sh", "DOCKERORACLEINIT=dockeroracleinit", "GRID_USER_HOME=/home/grid", "SETUPGRIDENV=setupGridEnv.sh", "DB_BASE=/u01/app/oracle", "DB_HOME=/u01/app/oracle/product/12.2.0/dbhome_1", "INSTALL_FILE_2=linuxx64_12201_database.zip", "DB_INSTALL_RSP=db_inst.rsp", "DBCA_RSP=dbca.rsp", "DB_SETUP_FILE=setupDB.sh", "PWD_FILE=setPassword.sh", "RUN_FILE=runOracle.sh", "STOP_FILE=stopOracle.sh", "ENABLE_RAC_FILE=enableRAC.sh", "CHECK_DB_FILE=checkDBStatus.sh", "USER_SCRIPTS_FILE=runUserScripts.sh", "REMOTE_LISTENER_FILE=remoteListener.sh", "INSTALL_DB_BINARIES_FILE=installDBBinaries.sh", "FUNCTIONS=functions.sh", "COMMON_SCRIPTS=/common_scripts", "CHECK_SPACE_FILE=checkSpace.sh", "EXPECT=/usr/bin/expect", "BIN=/usr/sbin", "container=true", "INSTALL_SCRIPTS=/opt/scripts/install", "SCRIPT_DIR=/opt/scripts/startup", "GRID_PATH=/u01/app/12.2.0/grid/bin:/u01/app/12.2.0/grid/OPatch/:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "DB_PATH=/u01/app/oracle/product/12.2.0/dbhome_1/bin:/u01/app/oracle/product/12.2.0/dbhome_1/OPatch/:/usr/sbin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin", "GRID_LD_LIBRARY_PATH=/u01/app/12.2.0/grid/lib:/usr/lib:/lib", "DB_LD_LIBRARY_PATH=/u01/app/oracle/product/12.2.0/dbhome_1/lib:/usr/lib:/lib" ], "Cmd": [ "/usr/sbin/oracleinit" ], "ArgsEscaped": true, "Image": "oracle/database-rac:12.2.0.1", "Volumes": { "/common_scripts": {}, "/dev/shm": {} }, "WorkingDir": "/home/grid", "Entrypoint": null, "OnBuild": null, "Labels": {} }, "NetworkSettings": { "Bridge": "", "SandboxID": "7343d36a5b07cd58303c13c5c6886b1652607df723537bbeca4e4b9da8ce9cd3", "HairpinMode": false, "LinkLocalIPv6Address": "", "LinkLocalIPv6PrefixLen": 0, "Ports": {}, "SandboxKey": "/var/run/docker/netns/7343d36a5b07", "SecondaryIPAddresses": null, "SecondaryIPv6Addresses": null, "EndpointID": "", "Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "IPAddress": "", "IPPrefixLen": 0, "IPv6Gateway": "", "MacAddress": "", "Networks": { "rac_priv1_nw": { "IPAMConfig": { "IPv4Address": "192.168.17.150" }, "Links": null, "Aliases": [ "1b11b69235ed" ], "NetworkID": "3bd1f5ee2c8310ce940defc113d57804328b0db9c27e6f061d7d8f678a7c533a", "EndpointID": "efa4908cf1f511190cd138c1535e8f9048f756a2fa1b1b8dd7b1b8b4c8d23602", "Gateway": "192.168.17.1", "IPAddress": "192.168.17.150", "IPPrefixLen": 24, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:c0:a8:11:96", "DriverOpts": null }, "rac_pub1_nw": { "IPAMConfig": { "IPv4Address": "172.15.1.150" }, "Links": null, "Aliases": [ "1b11b69235ed" ], "NetworkID": "92fbb95efdccf2e54d12236e0950842dacc186135ff0ede5cf0c77d1b97c59bf", "EndpointID": "d5bb65c7a37b40ac4c52c8ad0d720fabc0d2fcc26c096f584316ee2e602f10ac", "Gateway": "172.15.1.1", "IPAddress": "172.15.1.150", "IPPrefixLen": 24, "IPv6Gateway": "", "GlobalIPv6Address": "", "GlobalIPv6PrefixLen": 0, "MacAddress": "02:42:ac:0f:01:96", "DriverOpts": null } } } } ] [root@docker-host ~]#

[root@docker-host ~]# cat /proc/cpuinfo processor : 0 vendor_id : GenuineIntel cpu family : 6 model : 61 model name : Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz stepping : 4 cpu MHz : 2295.817 cache size : 3072 KB physical id : 0 siblings : 2 core id : 0 cpu cores : 2 apicid : 0 initial apicid : 0 fpu : yes fpu_exception : yes cpuid level : 20 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase avx2 invpcid rdseed bugs : cpu_meltdown spectre_v1 spectre_v2 bogomips : 4589.38 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management:

processor : 1 vendor_id : GenuineIntel cpu family : 6 model : 61 model name : Intel(R) Core(TM) i5-5300U CPU @ 2.30GHz stepping : 4 cpu MHz : 2295.817 cache size : 3072 KB physical id : 0 siblings : 2 core id : 1 cpu cores : 2 apicid : 1 initial apicid : 1 fpu : yes fpu_exception : yes cpuid level : 20 wp : yes flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology nonstop_tsc pni pclmulqdq ssse3 cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt aes xsave avx rdrand hypervisor lahf_lm abm 3dnowprefetch invpcid_single pti fsgsbase avx2 invpcid rdseed bugs : cpu_meltdown spectre_v1 spectre_v2 bogomips : 4589.38 clflush size : 64 cache_alignment : 64 address sizes : 39 bits physical, 48 bits virtual power management:

[root@docker-host ~]# [root@docker-host ~]# free -g total used free shared buff/cache available Mem: 7 2 0 4 5 0 Swap: 15 1 14

psaini79 commented 6 years ago

It seems you have only 2 CPUs? Are they virtual pr physical cores? Also, How many RAC containers have you created? Do you have anything else running on the host?

donjoxie commented 6 years ago

Yes, I created 2 CPUs and 10G memory on VirtualBox Env to testing 2 nodes RAC on a docker container.