Open cdalexndr opened 3 years ago
Pinging @elastic/integrations-services (Team:Services)
@cdalexndr So, just to be clear, you're running docker inside WSL? What other metrics are you looking at to get the correct disk usage?
Yes, docker has WSL2 based engine enabled, with Alpine image. This invalid disk usage is reported on my developer machine. I can see the correct disk usage in windows explorer (outside of docker).
but used size seems to be only docker related files
So, if you want to monitor a filesystem from inside docker, it needs to be mounted inside the container. There's an example here: https://www.elastic.co/guide/en/beats/metricbeat/current/running-on-docker.html#monitoring-host
Added hostfs
and now the total space has doubled (invalid). Used space still invalid.
docker compose service:
metricbeat:
image: store/elastic/metricbeat:7.9.3
mem_limit: 64m
depends_on:
- kibana #to register dashboards
volumes:
- metricbeat-data:/usr/share/metricbeat/data
- ./config/metricbeat.yml:/usr/share/metricbeat/metricbeat.yml:ro
- /var/run/docker.sock:/var/run/docker.sock:ro
- /:/hostfs:ro
user: root #docker sock access
cap_add:
- SYS_PTRACE
- DAC_READ_SEARCH
labels:
app: metricbeat
command: --strict.perms=false -e -system.hostfs=/hostfs
ES metricbeat doc:
{
"_index": "metricbeat-7.9.3-2020.12.29-000002",
"_type": "_doc",
"_id": "I3Z_F3cBDXAhja8VSPy9",
"_version": 1,
"_score": null,
"_source": {
"@timestamp": "2021-01-18T21:55:52.796Z",
"host": {
"name": "ff2cdac595b4"
},
"agent": {
"name": "ff2cdac595b4",
"type": "metricbeat",
"version": "7.9.3",
"hostname": "ff2cdac595b4",
"ephemeral_id": "5796531f-290a-474e-a1ff-106dc7eeed87",
"id": "0d988131-c303-49c5-9e64-ed8caa5ed884"
},
"system": {
"fsstat": {
"total_size": {
"free": 519909797888,
"used": 19788922880,
"total": 539698720768
},
"count": 13,
"total_files": 33554432
}
},
"event": {
"duration": 3637300,
"dataset": "system.fsstat",
"module": "system"
},
"metricset": {
"name": "fsstat",
"period": 60000
},
"service": {
"type": "system"
},
"ecs": {
"version": "1.5.0"
}
},
"fields": {
"@timestamp": [
"2021-01-18T21:55:52.796Z"
]
},
"highlight": {
"metricset.name": [
"@kibana-highlighted-field@fsstat@/kibana-highlighted-field@"
],
"service.type": [
"@kibana-highlighted-field@system@/kibana-highlighted-field@"
]
},
"sort": [
1611006952796
]
}
Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)
Metricbeat reports invalid free size. Total size reported is total disk size (correct), but used size seems to be only docker related files, not rest of files on disk. So the resulting free space is very large, but in reality, only ~70GB are free.
ES document:
metricbeat.yml:
docker service:
For confirmed bugs, please report: