search

immauss / openvas

Containers for running the Greenbone Vulnerability Manager. Run as a single container with all services or separate single applications containers via docker-compose.
GNU Affero General Public License v3.0
360 stars 102 forks source link

[arm] no report #46

Closed the-jojo-jj closed 3 years ago

the-jojo-jj commented 3 years ago

Hi I've tried arm version with docker and got no report, this is the log:

_openvas_logs.txt

thanks.

immauss commented 3 years ago

The arm version is woefully out of date at the moment. I've finally started working on building an arm version. I'll post here once it's uploaded.

the-jojo-jj commented 3 years ago

Thanks again in advance for your time and effort!

immauss commented 3 years ago

OK .... I'm trying something new, to hopefully save me some time/pain. buildx. Building for multiple architectures and pushing to docker hub in a single tag, all of the architectures.

Please give this one a try: immauss/openvas:multi

I must stress.... This is BETA!! I've not done any testing on this yet. There should be an arm64 image and an arm/v7 image with that tag. If I can get this to work, I'll start using buildx as my default to rebuild weekly and fresh multi-arch images.

Thanks, Scott

immauss commented 3 years ago

Oh ... and this is the latest 21.04 build.

the-jojo-jj commented 3 years ago

Hi Scoot, I've tried immauss/openvas:multi on rpi4 with Linux raspberrypi 5.10.17-v8+ #1414 SMP PREEMPT Fri Apr 30 13:23:25 BST 2021 aarch64 GNU/Linux and Docker version 20.10.7, build f0df350

it wont start with following logs:

standard_init_linux.go:228: exec user process caused: exec format error standard_init_linux.go:228: exec user process caused: exec format error

thanks.

immauss commented 3 years ago

Yes. I'm getting this too. I understood that arm64 would run on aarch64, but that seems to not be so. Needs a little more research it would seem. I'll let you know what I find. Thanks, Scott

immauss commented 3 years ago

OK ... there is now an "arm64v8" tag. Can you give that one a try? It will be a while before I can get to my RPi to try it out.

-Scott

the-jojo-jj commented 3 years ago

Hi everybody..

I've tried both arm64v8 and aarch64, and still got no luck with this same logs:

standard_init_linux.go:228: exec user process caused: exec format error

thanks.

immauss commented 3 years ago

OK ... so I woke up this morning and it dawned on me that my intermediate image, which I was originally creating to try and keep the overall image size down, only exists in amd64. Since the Dockerfile referenced that image, despite the --platform option, every build was still amd64.

I'm building a new multi arch image now. Let's see where that one goes. It should be done and on docker hub within the hour as immauss/openvas:multi.21.04.01 .

Of course, it will be this evening for me before I can test on my RPi ...

immauss commented 3 years ago

OK .. .there is finally a tag that works on my RPi. aarch64

As of right now, I can only verify that it runs without giving me the exec format error.

-Scott

the-jojo-jj commented 3 years ago

I've tried immauss/openvas:aarch64, but still got some error logs

_aarch64_logs.txt

the-jojo-jj commented 3 years ago

hi Scoot, @immauss, i've found also this: error during connect: Post http://docker:2375/v1.40/images/create?fromImage=immauss%2Fopenvas&tag=multi: dial tcp: lookup docker on 192.168.1.1:53: no such host

from immauss/openvas:multi

thanks.

the-jojo-jj commented 3 years ago

Sun Jun 20 14:09:25 BST 2021, i've tried latest immauss/openvas:multi and immauss/openvas:aarch64 and still got same error.

thanks.

the-jojo-jj commented 3 years ago

hi @immauss, i've tested latest aarch64.

and got: gsad gmp:WARNING:2021-06-24 10h06.03 utc:985: Failed to connect to server at /usr/local/var/run/gvmd.sock: No such file or directory gsad gmp:WARNING:2021-06-24 10h06.03 utc:985: Authentication failure for 'admin' from 192.168.1.16. Status was 1.

thanks.

immauss commented 3 years ago

Can you give me so details on how you are starting? docker-compose.yml script or command line? volume / no volume ?

I'm running the latest on RPi 4 w/ 4G running Kali 2021.2 and not seeing any issues. Even generates PDF reports.

Let me know.

-Scott

the-jojo-jj commented 3 years ago

Hi @immauss Scott,

This is my CLI:

mkdir /var/openvas

docker run --privileged -d -p 9392:9392 -p 9390:9390 -p 80:80 -e GMP=9390 --volume openvas:/data -e PUBLIC_HOSTNAME=192.168.1.8 --name aarch64 immauss/openvas:aarch64

immauss commented 3 years ago

try the "latest" tag. It has the most recent aarch64 image.

the-jojo-jj commented 3 years ago

I've tried also latest, with following error: root@raspberrypi:~# docker run --privileged -d -p 9392:9392 -p 9390:9390 -p 80:80 -e GMP=9390 --volume openvas:/data -e PUBLIC_HOSTNAME=192.168.1.8 --name latest immauss/openvas:latest Unable to find image 'immauss/openvas:latest' locally latest: Pulling from immauss/openvas docker: no matching manifest for linux/arm/v7 in the manifest list entries. See 'docker run --help'.

the-jojo-jj commented 3 years ago

also for armv7, here is the logs:

8:C 28 Jun 2021 03:47:28.900 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

8:C 28 Jun 2021 03:47:28.900 # Redis version=5.0.7, bits=32, commit=00000000, modified=0, pid=8, just started

8:C 28 Jun 2021 03:47:28.900 # Configuration loaded

Wait for redis socket to be created...

Testing redis status...  Redis ready.

Fixing Database folder...

Fixing local/var/lib ...

cp: cannot stat '/usr/local/var/lib/*': No such file or directory

8:C 28 Jun 2021 03:50:44.608 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

8:C 28 Jun 2021 03:50:44.608 # Redis version=5.0.7, bits=32, commit=00000000, modified=0, pid=8, just started

8:C 28 Jun 2021 03:50:44.608 # Configuration loaded

Wait for redis socket to be created...

Testing redis status...  Redis ready.

Fixing local/var/lib ...

cp: cannot stat '/usr/local/var/lib/*': No such file or directory

immauss commented 3 years ago

What Arm platform are you using?

I guess I need to pull out an older RPi and see if it works for me.

How much memory does it have?

Thanks, -Scott

the-jojo-jj commented 3 years ago

Hi @immauss, using Linux raspberrypi 5.10.17-v8+ #1421 SMP PREEMPT Thu May 27 14:01:37 BST 2021 aarch64 GNU/Linux and have 4GB RAM.

thanks for your time and effort.

the-jojo-jj commented 3 years ago

HI @immauss, I've tried 21.04.03

Using default tag: latest

error during connect: Post http://docker:2375/v1.40/images/create?fromImage=immauss%2F21.04.03&tag=latest: dial tcp: lookup docker on 192.168.1.1:53: no such host

Using default tag: latest

error during connect: Post http://docker:2375/v1.40/images/create?fromImage=immauss%2F21.04.03&tag=latest: dial tcp: lookup docker on 192.168.1.1:53: no such host

also tag latest

error during connect: Post http://docker:2375/v1.40/images/create?fromImage=immauss%2Fopenvas&tag=latest: dial tcp: lookup docker on 192.168.1.1:53: no such host

error during connect: Post http://docker:2375/v1.40/images/create?fromImage=immauss%2Fopenvas&tag=latest: dial tcp: lookup docker on 192.168.1.1:53: no such host

immauss commented 3 years ago

Yeah ... sorry about that ... 21.04.03 had some dependency issues and I pulled it. ospd-openvas was failing to run and the container would crash. I was expecting to replace it already, but the build processes take entirely too long for the multi-arch builds. I'll let you know when there is a working tag on docker.

immauss commented 3 years ago

The latest tag now should be good to go. Let me know if you have any issues.

the-jojo-jj commented 3 years ago

Hi @immauss from the latest tag, i've got: root@raspberrypi:~# docker run --privileged -d -p 9392:9392 -p 9390:9390 -p 80:80 -e GMP=9390 --volume openvas:/data -e PUBLIC_HOSTNAME=192.168.1.8 --name latest immauss/openvas:latest Unable to find image 'immauss/openvas:latest' locally latest: Pulling from immauss/openvas docker: no matching manifest for linux/arm/v7 in the manifest list entries. See 'docker run --help'.

Im using: root@raspberrypi:~ uname -a Linux raspberrypi 5.10.17-v8+ #1421 SMP PREEMPT Thu May 27 14:01:37 BST 2021 aarch64 GNU/Linux root@raspberrypi:~

thanks.

immauss commented 3 years ago

Why is your docker trying to pull arm/v7 when your uname is showing aarch64 ?

The arm/v7 image is constantly causing problems, but you should be pulling the arm64 image, which is available and working great.

the-jojo-jj commented 3 years ago

i've trying aarch64 also with following logs:

gsad gmp:WARNING:2021-07-05 08h32.12 utc:953: Failed to connect to server at /usr/local/var/run/gvmd.sock: No such file or directory

gsad gmp:WARNING:2021-07-05 08h32.12 utc:953: Authentication failure for 'admin' from 192.168.1.16. Status was 1.

gsad gmp:WARNING:2021-07-05 08h32.19 utc:953: Failed to connect to server at /usr/local/var/run/gvmd.sock: No such file or directory

gsad gmp:WARNING:2021-07-05 08h32.19 utc:953: Authentication failure for 'admin' from 192.168.1.16. Status was 1.

thanks.

immauss commented 3 years ago

Can you try a basic run and send me the full logs.

docker pull immauss/openvas:latest docker run -d -e SKIPSYNC=true --name basic immauss/openvas:latest

wait about 10minutes .. then:

docker logs basic > file-to-send.log

the-jojo-jj commented 3 years ago

hi @immauss

docker run -d -e SKIPSYNC=true --name basic immauss/openvas:latest WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested 4269f722d84c6137bafd1339bd03d870a51d1b6fa43b8687575543a3ff700744

and

docker run -d -e SKIPSYNC=true --name basic immauss/openvas:latest --platform linux/arm64 WARNING: The requested image's platform (linux/arm64) does not match the detected host platform (linux/arm/v7) and no specific platform was requested dc70f529f7fcb14b100bb82ce92362b0c7b4dc82cc17fb1870ca4816430a4a5b docker: Error response from daemon: OCI runtime create failed: container_linux.go:380: starting container process caused: exec: "--platform": executable file not found in $PATH: unknown.

immauss commented 3 years ago

Is this the same system? with the uname of : Linux raspberrypi 5.10.17-v8+ #1421 SMP PREEMPT Thu May 27 14:01:37 BST 2021 aarch64 GNU/Linux ??

can you please send me the output of : docker info

the-jojo-jj commented 3 years ago

Hi @immauss, yes this is from the same machine.

root@raspberrypi:~ uname -a Linux raspberrypi 5.10.17-v8+ #1421 SMP PREEMPT Thu May 27 14:01:37 BST 2021 aarch64 GNU/Linux root@raspberrypi:~ docker info Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)

Server: Containers: 1 Running: 1 Paused: 0 Stopped: 0 Images: 2 Server Version: 20.10.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runtime.v1.linux runc io.containerd.runc.v2 Default Runtime: runc Init Binary: docker-init containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 5.10.17-v8+ Operating System: Debian GNU/Linux 10 (buster) OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 3.706GiB Name: raspberrypi ID: PVTS:LIHI:KGOU:HH5H:3OZQ:D74B:TPLF:ZELB:QWB2:ET6S:OF4R:PS4A Docker Root Dir: /docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No memory limit support WARNING: No swap limit support WARNING: No kernel memory TCP limit support WARNING: No oom kill disable support

immauss commented 3 years ago

So there is defintely something wonky with you docker setup. Everything is reporting your system as arm64/aarch64, but when you try to pull the image, it is looking for the arm/v7, which even if it was there, is not compatible with that kernel.

Did you do a "docker pull immauss/openvas:latest" before running "docker run -d -e SKIPSYNC=true --name basic immauss/openvas:latest"

Maybe go through and make sure you delete any openvas images in your local image store and then do a new pull: I'm grasping at straws here because this well outside my level of XP with docker. Never seen anything like this before.

the-jojo-jj commented 3 years ago

i tried to os reload my pi with official 32 bit version, re-install docker with this following info:

root@raspberrypi:/docker# uname -a Linux raspberrypi 5.10.17-v7l+ #1414 SMP Fri Apr 30 13:20:47 BST 2021 armv7l GNU/Linux

root@raspberrypi:/docker# docker info Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)

Server: Containers: 2 Running: 1 Paused: 0 Stopped: 1 Images: 2 Server Version: 20.10.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 5.10.17-v7l+ Operating System: Raspbian GNU/Linux 10 (buster) OSType: linux Architecture: armv7l CPUs: 4 Total Memory: 3.786GiB Name: raspberrypi ID: TK7Z:RANY:CTSK:XFP6:SAQA:MXOD:C6IM:4VBT:VGCJ:P7SK:QE2C:KOMP Docker Root Dir: /docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No memory limit support WARNING: No swap limit support WARNING: No kernel memory TCP limit support WARNING: No oom kill disable support

tried the armv7 tags with following logs: 8:C 07 Jul 2021 09:05:31.582 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

8:C 07 Jul 2021 09:05:31.582 # Redis version=5.0.7, bits=32, commit=00000000, modified=0, pid=8, just started

8:C 07 Jul 2021 09:05:31.582 # Configuration loaded

Wait for redis socket to be created...

Testing redis status...  Redis ready.

Creating Data and database folder...

Fixing Database folder...

Fixing local/var/lib ...

cp: cannot stat '/usr/local/var/lib/*': No such file or directory

8:C 07 Jul 2021 09:06:23.236 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

8:C 07 Jul 2021 09:06:23.236 # Redis version=5.0.7, bits=32, commit=00000000, modified=0, pid=8, just started

8:C 07 Jul 2021 09:06:23.236 # Configuration loaded

Wait for redis socket to be created...

Testing redis status...  Redis ready.

Fixing local/var/lib ...

cp: cannot stat '/usr/local/var/lib/*': No such file or directory

something in your mind, @immauss?

immauss commented 3 years ago

Well... none of the arm/v7 tags work properly at the moment, and the build process, which works fine for arm64 & amd64, fails for the arm/v7. So I can't update it either.

Why would you want the 32bit kernel if the Pi you have will run the 64?

the-jojo-jj commented 3 years ago

hi @immauss,

I've reload the os again with: root@raspberrypi:~# uname -a Linux raspberrypi 5.4.42-v8+ #1319 SMP PREEMPT Wed May 20 14:18:56 BST 2020 aarch64 GNU/Linux

with latest docker: root@raspberrypi:~# docker info Client: Context: default Debug Mode: false Plugins: app: Docker App (Docker Inc., v0.9.1-beta3) buildx: Build with BuildKit (Docker Inc., v0.5.1-docker)

Server: Containers: 2 Running: 2 Paused: 0 Stopped: 0 Images: 2 Server Version: 20.10.7 Storage Driver: overlay2 Backing Filesystem: extfs Supports d_type: true Native Overlay Diff: true userxattr: false Logging Driver: json-file Cgroup Driver: cgroupfs Cgroup Version: 1 Plugins: Volume: local Network: bridge host ipvlan macvlan null overlay Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog Swarm: inactive Runtimes: io.containerd.runc.v2 io.containerd.runtime.v1.linux runc Default Runtime: runc Init Binary: docker-init containerd version: d71fcd7d8303cbf684402823e425e9dd2e99285d runc version: b9ee9c6314599f1b4a7f497e1f1f856fe433d3b7 init version: de40ad0 Security Options: seccomp Profile: default Kernel Version: 5.4.42-v8+ Operating System: Debian GNU/Linux 10 (buster) OSType: linux Architecture: aarch64 CPUs: 4 Total Memory: 3.709GiB Name: raspberrypi ID: 64HW:BGD6:EU7N:Z2YT:YO7K:RU7B:JGN4:VTZZ:INIG:QEVD:66OP:JJJU Docker Root Dir: /docker Debug Mode: false Registry: https://index.docker.io/v1/ Labels: Experimental: false Insecure Registries: 127.0.0.0/8 Live Restore Enabled: false

WARNING: No memory limit support WARNING: No swap limit support WARNING: No kernel memory TCP limit support WARNING: No oom kill disable support WARNING: No cpu cfs quota support WARNING: No cpu cfs period support

and try the latest tag: docker volume create openvas docker run --privileged -d -p 9392:9392 -p 9390:9390 -p 80:80 -e GMP=9390 --volume openvas:/data -e PUBLIC_HOSTNAME=192.168.1.8 --name latest immauss/openvas:latest

with following error:

==> /usr/local/var/log/gvm/gsad.log <==

gsad gmp:WARNING:2021-07-08 10h38.36 utc:1187: Failed to connect to server at /usr/local/var/run/gvmd.sock: No such file or directory

gsad gmp:WARNING:2021-07-08 10h38.36 utc:1187: Authentication failure for 'admin' from 192.168.1.16. Status was 1.

gsad gmp:WARNING:2021-07-08 10h38.46 utc:1187: Failed to connect to server at /usr/local/var/run/gvmd.sock: No such file or directory

gsad gmp:WARNING:2021-07-08 10h38.46 utc:1187: Authentication failure for 'admin' from 192.168.1.16. Status was 1.

i cant login to web.

-e PASSWORD="1234567890" also doesn't work.

thanks.

immauss commented 3 years ago

It looks like gvmd is not starting.

I keep forgetting to ask, why are you adding "--privileged" ? This should not be needed. I can't say it's causing the problems, but not needed. Also, there is nothing listening on port 80, -p 80:80 is not needed either. And unless you are planning to use the GMP server with some extra tools, you do not need that either. As for the password, it's likely being set in this scenario, but since gsa can't talk to gvmd, it can't login.

immauss commented 3 years ago

OK. The new "armv7" tag is available and seems to be operational. It seems to work on hardware, but it took me a while to realize that postgres just does not like to be run with qemu on the armv7.
I'm closing all of the current armv7 issues since since this is basically a complete rebuild. If you have new (or the same ) problems, please open a new issue.

Thanks, Scott