rancher / community-catalog

Catalog entries contributed by the community
385 stars 641 forks source link

add support of prometeus in centos hosts #800

Closed behroozam closed 4 years ago

behroozam commented 6 years ago

cadvisor bug in latest build dont let prometeus properly work in centos servers. its a proper solution for fix this bug

rawmind0 commented 6 years ago

Hi @behroozam . Thanks for the PR.

Could you please attach cadvisor bug you are referencing?? Is it a question of running parameters?? https://github.com/google/cadvisor/blob/master/docs/running.md#centos-fedora-and-rhel

behroozam commented 6 years ago

hi @rawmind0 no its not about --privileged=true or location of cgroup , in downgraded version its work properly. others have same problem here https://github.com/google/cadvisor/issues/1850

rawmind0 commented 6 years ago

Thanks for the info @behroozam ...

I tried to reproduce the error but i couldn't... I tried with centos 7.3, 7.4 and 7.5 but i get no error on cadvisor..

14/06/2018 12:09:49I0614 10:09:49.520021       1 storagedriver.go:50] Caching stats in memory for 2m0s
14/06/2018 12:09:49I0614 10:09:49.520432       1 manager.go:151] cAdvisor running in container: "/sys/fs/cgroup/cpu,cpuacct"
14/06/2018 12:09:49I0614 10:09:49.612843       1 fs.go:139] Filesystem UUIDs: map[]
14/06/2018 12:09:49I0614 10:09:49.612870       1 fs.go:140] Filesystem partitions: map[tmpfs:{mountpoint:/dev major:0 minor:87 fsType:tmpfs blockSize:0} /dev/mapper/cl-root:{mountpoint:/var/lib/docker major:253 minor:0 fsType:xfs blockSize:0} /dev/xvda1:{mountpoint:/rootfs/boot major:202 minor:1 fsType:xfs blockSize:0} shm:{mountpoint:/rootfs/var/lib/docker/containers/fe21b5b5794f69a12c207ab30aa1c034e27013cb333c5f5f3a1151bd1b641175/mounts/shm major:0 minor:39 fsType:tmpfs blockSize:0}]
14/06/2018 12:09:49I0614 10:09:49.623139       1 manager.go:225] Machine: {NumCores:1 CpuFrequency:2394453 MemoryCapacity:1038225408 HugePages:[{PageSize:2048 NumPages:0}] MachineID:90a603b0ca53435aa67bd6e0bfa48adc SystemUUID:EC22D1C7-D379-1DBC-AA43-E9C89B1176FF BootID:ea70323e-9688-4575-bd05-5c71ba39d025 Filesystems:[{Device:shm DeviceMajor:0 DeviceMinor:39 Capacity:67108864 Type:vfs Inodes:126736 HasInodes:true} {Device:overlay DeviceMajor:0 DeviceMinor:74 Capacity:6641680384 Type:vfs Inodes:3248128 HasInodes:true} {Device:tmpfs DeviceMajor:0 DeviceMinor:87 Capacity:67108864 Type:vfs Inodes:126736 HasInodes:true} {Device:/dev/mapper/cl-root DeviceMajor:253 DeviceMinor:0 Capacity:6641680384 Type:vfs Inodes:3248128 HasInodes:true} {Device:/dev/xvda1 DeviceMajor:202 DeviceMinor:1 Capacity:1063256064 Type:vfs Inodes:524288 HasInodes:true}] DiskMap:map[253:0:{Name:dm-0 Major:253 Minor:0 Size:6652166144 Scheduler:none} 253:1:{Name:dm-1 Major:253 Minor:1 Size:859832320 Scheduler:none} 202:0:{Name:xvda Major:202 Minor:0 Size:17179869184 Scheduler:deadline}] NetworkDevices:[{Name:eth0 MacAddress:02:73:f6:95:8d:9e Speed:0 Mtu:9001}] Topology:[{Id:0 Memory:1073336320 Cores:[{Id:0 Threads:[0] Caches:[{Size:32768 Type:Data Level:1} {Size:32768 Type:Instruction Level:1} {Size:262144 Type:Unified Level:2}]}] Caches:[{Size:31457280 Type:Unified Level:3}]}] CloudProvider:AWS InstanceType:t2.micro InstanceID:i-070d868272efc41c3}
14/06/2018 12:09:49I0614 10:09:49.625373       1 manager.go:231] Version: {KernelVersion:3.10.0-862.3.2.el7.x86_64 ContainerOsVersion:Alpine Linux v3.4 DockerVersion:18.03.1-ce DockerAPIVersion:1.37 CadvisorVersion:v0.28.3 CadvisorRevision:1e567c2}
14/06/2018 12:09:49I0614 10:09:49.646425       1 factory.go:356] Registering Docker factory
14/06/2018 12:09:51I0614 10:09:51.646888       1 factory.go:54] Registering systemd factory
14/06/2018 12:09:51I0614 10:09:51.648350       1 factory.go:86] Registering Raw factory
14/06/2018 12:09:51I0614 10:09:51.650081       1 manager.go:1178] Started watching for new ooms in manager
14/06/2018 12:09:51W0614 10:09:51.650112       1 manager.go:313] Could not configure a source for OOM detection, disabling OOM events: open /dev/kmsg: no such file or directory
14/06/2018 12:09:51I0614 10:09:51.652307       1 manager.go:329] Starting recovery of all containers
14/06/2018 12:09:51I0614 10:09:51.843296       1 manager.go:334] Recovery completed
14/06/2018 12:09:51I0614 10:09:51.976703       1 cadvisor.go:162] Starting cAdvisor version: v0.28.3-1e567c2 on port 8080

Just trying to figure out if another solution is possible... Like mount cgroups or adding privilege if os is centos or amazon linux..... what do you think??

cjellick commented 4 years ago

Closing this due to staleness. Feel free to reopen or open a new PR if there's still a relevant change to be made. Thanks!