PermissionDenied when attempting to run pg-ferret-all-in-one

[kjcsb1]

Thanks for you work with pg-ferret.

I get PermissionDenied when attempting to run pg-ferret-all-in-one whether as root, under sudo, or any user:

docker run -it   -e POSTGRES_DB=mydb   -e POSTGRES_USER=myuser   -e POSTGRES_PASSWORD=mypass   --privileged -p 5432:5432 -p 3000:3000   cbellew/pg-ferret-all-in-one:latest
Starting Grafana...
Starting Tempo...
[pg-ferret] [2024-07-20T04:18:58Z INFO  userspace_collector] Starting pg-ferret userspace collector
[pg-ferret] [2024-07-20T04:18:58Z WARN  userspace_collector::bpf] failed to initialize eBPF logger: log event array AYA_LOGS doesn't exist
[pg-ferret] thread 'main' panicked at userspace-collector/src/main.rs:26:30:
[pg-ferret] called `Result::unwrap()` on an `Err` value: LoadError { io_error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }, verifier_log: 0:

Docker version 27.0.3, build 7d4bcd8 Kernel 5.4.0-187-generic

Any suggestions appreciated.

[ChrisBellew]

Thanks for trying pg-ferret and sorry you are having issues.

I've tried a few systems to see if I can replicate:

• Desktop Ubuntu 22.04 x86 (6.5.0-44-generic)
• AWS EC2 Ubuntu 22.04 x86 (6.5.0-1022-aws)
• AWS EC2 Ubuntu 24.04 x86 (6.8.0-1008-aws)

Admittedly they have newer kernel versions but I don't easily have an older kernel version close to hand.

Are you spinning up a cloud box to test this or is it a local machine? Just wondering if you can easily share a way to arrive at the same setup if it's a cloud machine.

[ChrisBellew]

Actually I just tried with AWS EC2 Ubuntu 20.04 x86 (5.15.0-1064-aws) and it fails with the same error you are experiencing.

pg-ferret internally uses Aya to load the eBPF program and I've just noticed a note about issues with running on Ubuntu 20.04

Running on Ubuntu 20.04 LTS (Focal)? If you're running on Ubuntu 20.04, there is a bug with bpftool and the default kernel installed by the distribution. To avoid running into it, you can install a newer bpftool version that does not include the bug with:

https://aya-rs.dev/book/start/development/

Can you please confirm you are using Ubuntu 20.04? If so, I'll take a look into what would be involved to add the workaround, but I can't promise anything soon. Is using Ubuntu 22.04 or 24.04 an option?

[kjcsb1]

Many thanks for the prompt response.

I was trying on my laptop: Linux Mint 20.3

I tried sudo apt install linux-tools-5.8.0-63-generic but that didn't help.

I also just tried on a local physical server: Ubuntu 22.04.4 LTS 5.15.0-105-generic

Starting Grafana...
Starting Tempo...
[pg-ferret] [2024-07-21T08:19:21Z INFO  userspace_collector] Starting pg-ferret userspace collector
[pg-ferret] [2024-07-21T08:19:21Z WARN  userspace_collector::bpf] failed to initialize eBPF logger: log event array AYA_LOGS doesn't exist
[tempo] level=warn ts=2024-07-21T08:19:21.607691993Z caller=modules.go:192 msg="Worker address is empty in single binary mode. Attempting automatic worker configuration. If queries are unresponsive consider configuring the worker explicitly." address=127.0.0.1:9095
[tempo] level=warn ts=2024-07-21T08:19:21.608315979Z caller=modules.go:170 msg="metrics-generator is not configured." err="no metrics_generator.storage.path configured, metrics generator will be disabled"
[postgres] The files belonging to this database system will be owned by user "postgres".
[postgres] This user must also own the server process.
[postgres]
[postgres] The database cluster will be initialized with locale "en_US.utf8".
[tempo] level=warn ts=2024-07-21T08:19:21.610160209Z caller=wal.go:112 msg="unowned file entry ignored during wal replay" file=blocks err=null
[postgres] The default database encoding has accordingly been set to "UTF8".
[postgres] The default text search configuration will be set to "english".
[tempo] level=warn ts=2024-07-21T08:19:21.611015742Z caller=rescan_blocks.go:22 msg="failed to open search wal directory" err="open /var/tempo/wal/search: no such file or directory"
[postgres]
[postgres] Data page checksums are disabled.
[postgres]
[postgres] fixing permissions on existing directory /var/lib/postgresql/data ... ok
[postgres] creating subdirectories ... ok
[postgres] selecting dynamic shared memory implementation ... posix
[postgres] selecting default max_connections ... 100
[pg-ferret] thread 'main' panicked at userspace-collector/src/main.rs:26:30:
[pg-ferret] called `Result::unwrap()` on an `Err` value: LoadError { io_error: Os { code: 13, kind: PermissionDenied, message: "Permission denied" }, verifier_log: 0: (bf) r6 = r1

Now that I know it's probably environmental and not something I'm doing wrong, I'll try to find a machine running a 6.x kernel and let you know how I get on.

Thanks