Open edenon opened 4 months ago
Hi,
many thanks for the great docker file. I think I see the same or similar issue - which is blocking it to start up. I use Portianer to create a stack.
failed to deploy a stack: Network netprobe_light_custom_network Creating Network netprobe_light_custom_network Created Volume "netprobe_light_prometheus_data" Creating Volume "netprobe_light_prometheus_data" Created Volume "netprobe_light_grafana_data" Creating Volume "netprobe_light_grafana_data" Created Container netprobe-presentation Creating Container netprobe-redis Creating Container netprobe-grafana Creating Container netprobe-probe Creating Container netprobe-prometheus Creating Container netprobe-redis Created Container netprobe-probe Created Container netprobe-prometheus Created Container netprobe-presentation Created Container netprobe-grafana Created Container netprobe-redis Starting Container netprobe-presentation Starting Container netprobe-probe Starting Container netprobe-prometheus Starting Container netprobe-grafana Starting Container netprobe-redis Started Container netprobe-presentation Started Container netprobe-probe Started Container netprobe-grafana Started Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/compose/45/config/prometheus/prometheus.yml" to rootfs at "/etc/prometheus/prometheus.yml": mount /data/compose/45/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Non docker or portainer works.
Regards
El vie, 3 de may de 2024, 05:58, j64bit @.***> escribió:
Hi,
many thanks for the great docker file. I think I see the same or similar issue - which is blocking it to start up. I use Portianer to create a stack.
failed to deploy a stack: Network netprobe_light_custom_network Creating Network netprobe_light_custom_network Created Volume "netprobe_light_prometheus_data" Creating Volume "netprobe_light_prometheus_data" Created Volume "netprobe_light_grafana_data" Creating Volume "netprobe_light_grafana_data" Created Container netprobe-presentation Creating Container netprobe-redis Creating Container netprobe-grafana Creating Container netprobe-probe Creating Container netprobe-prometheus Creating Container netprobe-redis Created Container netprobe-probe Created Container netprobe-prometheus Created Container netprobe-presentation Created Container netprobe-grafana Created Container netprobe-redis Starting Container netprobe-presentation Starting Container netprobe-probe Starting Container netprobe-prometheus Starting Container netprobe-grafana Starting Container netprobe-redis Started Container netprobe-presentation Started Container netprobe-probe Started Container netprobe-grafana Started Error response from daemon: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/data/compose/45/config/prometheus/prometheus.yml" to rootfs at "/etc/prometheus/prometheus.yml": mount /data/compose/45/config/prometheus/prometheus.yml:/etc/prometheus/prometheus.yml (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
— Reply to this email directly, view it on GitHub https://github.com/plaintextpackets/netprobe_lite/issues/29#issuecomment-2092592155, or unsubscribe https://github.com/notifications/unsubscribe-auth/BCYO37KZQTBNYNONZPXCHY3ZANGSDAVCNFSM6AAAAABHEYWO3CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJSGU4TEMJVGU . You are receiving this because you authored the thread.Message ID: @.***>
Portainer works for you?
I haven't tested it with portainer not sure if anyone else has
j64bit mention it but do not understand if he made it work. I'm still trying getting error: WARN[0000] /home/NNN---/netprobe_lite/compose.yml: version is obsolete [+] Running 5/0 ✔ Container netprobe-probe Running 0.0s ✔ Container netprobe-redis Running 0.0s ✔ Container netprobe-prometheus Running 0.0s ✔ Container netprobe-grafana Recreated 0.0s ✔ Container netprobe-presentation Running 0.0s Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Any idea about this?
Just to clarify: it does NOT work for me. See my error message above.
Using tag v1.2.1 I get the same error as @j64bit under Portainer.
One can map the parent directly as a separate volume, then map the "file" but this merely appeases docker-compose so it will start the stack, but the "file" map will again be a directory unless it already exists everywhere as a file.
I even tried using the bind
volume type for each individual config file, e.g.:
...
volumes:
- type: bind
source: ./config/prometheus/prometheus.yml
target: /etc/prometheus/prometheus.yml
- prometheus_data:/prometheus # Persistent local storage for Prometheus data
...
but it just thew more errors:
Failed to deploy a stack: Network netprobe_netprobe-net Creating Network netprobe_netprobe-net Created Volume "netprobe_grafana_data" Creating Volume "netprobe_grafana_data" Created Volume "netprobe_prometheus_data" Creating Volume "netprobe_prometheus_data" Created Container netprobe-grafana Creating Container netprobe-prometheus Creating Container netprobe-redis Creating Container netprobe-probe Creating Container netprobe-presentation Creating Error response from daemon: invalid mount config for type "bind": bind source path does not exist: /data/compose/85/config/grafana/datasources/automatic.yml
Without every config file and their leading parent directories already existing at config time (e.g. docker-compose validation pre-run time), I don't think there is a solution for us.
Instead of specifying actual config files directly within the compose file, I often see image creators instead redirect the location of config and data files for their app(s) explicitly into a common directory they can then map those directories as a volumes, e.g.:
- ./config:/myapp/config
- ./data:/myapp/data
…then populate all their config and data files and directories into those locations at first startup.
One place to start troubleshooting might be to run docker compose config -fq
within a fresh and unpopulated project folder… my hope is you will see a bunch of the same errors we are seeing 🤷
I'm having the same issue as j64bit was reporting. I tried changing the file '/netprobe_lite/config/grafana/dashboards/netprobe.json' to the same path but with 'automatic.yml' instead of 'netprobe.json' and it would then load and start. The contents of Grafana would be incorrect, but at least this was a hint as to what might be going wrong. Any help would be appreciated. Thanks for the work that you all put into this.
Same issue here even with absolute path location of volumes. Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis, netprobe-speedtest Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
For those running this on Linux I found a solution from this post: https://stackoverflow.com/questions/72695311/failure-starting-docker-container-failed-to-create-shim-task-oci-runtime-crea
I used chmod +x on the home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json file.
chmod +x home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json
This allowed the installation to complete.
I also did this for automatic.yml, and main.yml although I don't think these were necessary.
For those running this on Linux I found a solution from this post: https://stackoverflow.com/questions/72695311/failure-starting-docker-container-failed-to-create-shim-task-oci-runtime-crea
I used chmod +x on the home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json file.
chmod +x home/ionut/netprobe_lite/config/grafana/dashboards/netprobe.json
This allowed the installation to complete.
I also did this for automatic.yml, and main.yml although I don't think these were necessary.
This doesn't work on my Ubuntu setup. Don't know why. Not even without Portainer.
NNN---@NNN-TEST:~/netprobe_lite$ docker compose up WARN[0000] /home/NNN---/netprobe_lite/compose.yml:
version
is obsolete [+] Running 5/0 ✔ Container netprobe-presentation Created 0.0s ✔ Container netprobe-grafana Created 0.0s ✔ Container netprobe-probe Created 0.0s ✔ Container netprobe-redis Created 0.1s ✔ Container netprobe-prometheus Created 0.0s Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:573 level=info msg="No time or size retention was set so using the default time retention" duration=15d netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:617 level=info msg="Starting Prometheus Server" mode=server version="(version=2.51.2, branch=HEAD, revision=b4c0ab52c3e9b940ab803581ddae9b3d9a452337)" netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:622 level=info build_context="(go=go1.22.2, platform=linux/amd64, user=root@b63f02a423d9, date=20240410-14:05:54, tags=netgo,builtinassets,stringlabels)" netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:623 level=info host_details="(Linux 5.4.0-177-generic #197-Ubuntu SMP Thu Mar 28 22:45:47 UTC 2024 x86_64 b856a8d8dbca (none))" netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:624 level=info fd_limits="(soft=1048576, hard=1048576)" netprobe-prometheus | ts=2024-05-03T02:41:39.181Z caller=main.go:625 level=info vm_limits="(soft=unlimited, hard=unlimited)" netprobe-prometheus | ts=2024-05-03T02:41:39.184Z caller=web.go:568 level=info component=web msg="Start listening for connections" address=0.0.0.0:9090 netprobe-prometheus | ts=2024-05-03T02:41:39.185Z caller=main.go:1129 level=info msg="Starting TSDB ..." netprobe-prometheus | ts=2024-05-03T02:41:39.186Z caller=dir_locker.go:77 level=warn component=tsdb msg="A lockfile from a previous execution already existed. It was replaced" file=/prometheus/lock netprobe-prometheus | ts=2024-05-03T02:41:39.192Z caller=tls_config.go:313 level=info component=web msg="Listening on" address=[::]:9090 netprobe-prometheus | ts=2024-05-03T02:41:39.192Z caller=tls_config.go:316 level=info component=web msg="TLS is disabled." http2=false address=[::]:9090 netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:616 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:698 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.95µs netprobe-prometheus | ts=2024-05-03T02:41:39.194Z caller=head.go:706 level=info component=tsdb msg="Replaying WAL, this may take a while" netprobe-prometheus | ts=2024-05-03T02:41:39.225Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=3 netprobe-prometheus | ts=2024-05-03T02:41:39.226Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=3 netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=2 maxSegment=3 netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:778 level=info component=tsdb msg="WAL segment loaded" segment=3 maxSegment=3 netprobe-prometheus | ts=2024-05-03T02:41:39.227Z caller=head.go:815 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=95.788µs wal_replay_duration=32.827096ms wbl_replay_duration=161ns total_replay_duration=32.974262ms netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1150 level=info fs_type=EXT4_SUPER_MAGIC netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1153 level=info msg="TSDB started" netprobe-prometheus | ts=2024-05-03T02:41:39.230Z caller=main.go:1335 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=main.go:1372 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=2.440331ms db_storage=1.321µs remote_storage=1.21µs web_handler=592ns query_engine=1.12µs scrape=2.11521ms scrape_sd=38.265µs notify=795ns notify_sd=1.333µs rules=1.361µs tracing=6.411µs netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=main.go:1114 level=info msg="Server is ready to receive web requests." netprobe-prometheus | ts=2024-05-03T02:41:39.233Z caller=manager.go:163 level=info component="rule manager" msg="Starting rule manager..." netprobe-redis | 1:C 03 May 2024 02:41:39.248 # WARNING Memory overcommit must be enabled! Without it, a background save or replication may fail under low memory condition. Being disabled, it can also cause failures without low memory condition, see https://github.com/jemalloc/jemalloc/issues/1328. To fix this issue add 'vm.overcommit_memory = 1' to /etc/sysctl.conf and then reboot or run the command 'sysctl vm.overcommit_memory=1' for this to take effect. netprobe-redis | 1:C 03 May 2024 02:41:39.248 oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo netprobe-redis | 1:C 03 May 2024 02:41:39.248 Redis version=7.2.4, bits=64, commit=00000000, modified=0, pid=1, just started netprobe-redis | 1:C 03 May 2024 02:41:39.248 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf netprobe-redis | 1:M 03 May 2024 02:41:39.249 monotonic clock: POSIX clock_gettime netprobe-redis | 1:M 03 May 2024 02:41:39.250 Running mode=standalone, port=6379. netprobe-redis | 1:M 03 May 2024 02:41:39.251 Server initialized netprobe-redis | 1:M 03 May 2024 02:41:39.251 Ready to accept connections tcp netprobe-probe | /netprobe_lite/helpers/network_helper.py:25: SyntaxWarning: invalid escape sequence '|' netprobe-probe | ping = subprocess.getoutput(f"ping -n -i 0.1 -c {count} {site} | grep 'rtt|loss'") Error response from daemon: driver failed programming external connectivity on endpoint netprobe-grafana (588de0b3fe6d13d712538a22086b31bbbe5bd9f4c0ce4753f92b700e8aafff26): Bind for 0.0.0.0:3001 failed: port is already allocated NNN---@NNN-TEST:~/netprobe_lite$ sudo vim compose.yml NNN---@NNN-TEST:~/netprobe_lite$ docker compose up WARN[0000] /home/NNN---/netprobe_lite/compose.yml:version
is obsolete [+] Running 5/0 ✔ Container netprobe-probe Running 0.0s ✔ Container netprobe-redis Running 0.0s ✔ Container netprobe-prometheus Running 0.0s ✔ Container netprobe-grafana Recreated 0.0s ✔ Container netprobe-presentation Running 0.0s Attaching to netprobe-grafana, netprobe-presentation, netprobe-probe, netprobe-prometheus, netprobe-redis Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "/home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json" to rootfs at "/var/lib/grafana/dashboards/netprobe.json": mount /home/NNN---/netprobe_lite/config/grafana/dashboards/netprobe.json:/var/lib/grafana/dashboards/netprobe.json (via /proc/self/fd/6), flags: 0x5000: not a directory: unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type