Closed sdaaish closed 5 years ago
drwetter
commented
5 years ago hmm.. works for me, also with the odd cmdline ;-) : --csv and --csvfile is contradictory, similar: would be with JSON.
What exactly did you want as an output file? If I run this thing, I have a file named reports which contained JSON + CSV -- and this is the only file being generated.
Maybe testssl.sh -U --csv --json www.linux.com is what you wanted?
sdaaish
commented
5 years ago I just noticed that the CSV-exports generates errors but if I use JSON it doesn't. Even with errors it generates a CSV-file that looks OK.
So I'm more curious about what the error means since it does what it is supposed to.
I should also mention I run this in docker on windows, so maybe this has to do with escaping of filepath. But I only noticed this för csv and not any other format with the same input- and output-path.
And if I remove the extra cmd-line option I get the same result for csv. Json still works the same.
/usr/local/bin/testssl.sh --warnings=batch -BB --csvfile reports https://www.linux.com:443
/usr/local/bin/testssl.sh: line 894: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 895: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 896: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 897: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 898: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 899: reports/www.linux.com_p443-20190328-1030.csv: No such file or directory
/usr/local/bin/testssl.sh: line 904: reports/www.linux.com_p443-20190328-1030.csv: No such file or directoryI should also mention I run this in docker on windows, so maybe this has to do with escaping of filepath.
In a container that shouldn't matter.
And if I remove the extra cmd-line option I get the same result for csv.
Can't still reproduce it in a container. Files within reports are created properly
drwetter
commented
5 years ago Q: Which container are you using and are you running everything from inside of the container (using a shell)?
sdaaish
commented
5 years ago I dont have the container published, can do that tomorrow. I'm running almost the same config (from my memory) on a linux host and don't get the error either.
It's nothing special, based on your Dockerfile. Everything runs inside the container I only feed it an input file and an output directory.
Maybe this can be related to windows and docker. If the container reads a filename with qoutation marks this might fail on windows. It happens sometimes with WSL writing to the windows filesystem.
I gonna try some more tests tomorrow if I can get the time.
drwetter
commented
5 years ago It's nothing special, based on your Dockerfile. Everything runs inside the container I only feed it an input file and an output directory.
don't know exactly how I should read that but if you try to write on a windows file reports/www.linux.com_p443-20190328-1030.csv:probably would not work.
sdaaish
commented
5 years ago I tested again and made a test-file to show the scenario. The assumption is that testssl.sh works in docker for windows except when it tries to create .CSV-files.
I made a Gist of this test here: https://gist.github.com/sdaaish/9c57968d1d987f675ed226ed12f3325f This contains the source, result and output log from docker-compose. https://gist.github.com/sdaaish/9c57968d1d987f675ed226ed12f3325f#file-result-log
The reason for using docker-compose is that I don't get volume-mounts to work under windows but I this works in docker-compose. The same image but something works differently. I'm not really that familiar with docker to know why.
I had to move around files to make them available in the gist, they should be in input and results respectively.
So my theory is that this error triggers when I run testssl.sh in a docker-container on Windows and tries to create csv-files. Which is pretty specific and might have nothing to do with testssl. But since I can create json, html and log-files whitout the error message there might be something in testssl that may be broken.
This line creates different formats but only complains about csv:
/usr/local/bin/testssl.sh --warnings=batch -H --jsonfile reports --logfile reports --htmlfile reports --csvfile reports https://www.freebsd.com:443
/usr/local/bin/testssl.sh: line 894: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 895: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 896: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 897: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 898: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 899: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
/usr/local/bin/testssl.sh: line 904: reports/www.freebsd.com_p443-20190401-1308.csv: No such file or directory
To me this is a bug but maybe not the most important one. Something with filenames that are not compatible on Windows.
I think I did a test that proves my point, not sure how to actually find the problem though. Probably need to add some debug to print the filename that the sh-script tries to create.
drwetter
commented
5 years ago Can you try to run against a host with a single IP like testssl.sh?
drwetter
commented
5 years ago Could you do a fresh pull pls and try again.
If it works, please let me know the bash version (bash --version)
sdaaish
commented
5 years ago So I destroyd all images an containers and ran it again. The build pulls the latest version from git, see first example. I realize that I must have forgotten to rebuild the container earlier if you made changes.
Now I can run testssl with one or multiple ip's without error when creating csv-files. This is on the same computer that I ran all other tests. So it seems that I no longer can reproduce the error.
Included version of testssl and bash for this test.
Let me now if you want me to do some other test to verify this.
Regards Stig
-H --jsonfile reports --logfile reports --htmlfile reports --csvfile reports --append https://testssl.sh:443-H --csvfile reports https://linux.com:443This builds a new image and pulls the latest stuff from github.
$ docker-compose run testssl.sh --file input/testssl.sh
Building testssl.sh
Step 1/8 : FROM alpine:latest
---> 196d12cf6ab1
Step 2/8 : RUN apk update && apk upgrade && apk add bash procps drill git coreutils && apk add --no-cache curl
---> Running in 6a67ffb82183
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
v3.8.4-8-g4a349ec470 [http://dl-cdn.alpinelinux.org/alpine/v3.8/main]
v3.8.4-4-gc27a9a0149 [http://dl-cdn.alpinelinux.org/alpine/v3.8/community]
OK: 9549 distinct packages available
(1/5) Upgrading busybox (1.28.4-r1 -> 1.28.4-r3)
Executing busybox-1.28.4-r3.post-upgrade
(2/5) Upgrading libressl2.7-libcrypto (2.7.4-r0 -> 2.7.5-r0)
(3/5) Upgrading libressl2.7-libssl (2.7.4-r0 -> 2.7.5-r0)
(4/5) Upgrading libressl2.7-libtls (2.7.4-r0 -> 2.7.5-r0)
(5/5) Upgrading ssl_client (1.28.4-r1 -> 1.28.4-r3)
Executing busybox-1.28.4-r3.trigger
OK: 4 MiB in 13 packages
(1/20) Installing ncurses-terminfo-base (6.1_p20180818-r1)
(2/20) Installing ncurses-terminfo (6.1_p20180818-r1)
(3/20) Installing ncurses-libs (6.1_p20180818-r1)
(4/20) Installing readline (7.0.003-r0)
(5/20) Installing bash (4.4.19-r1)
Executing bash-4.4.19-r1.post-install
(6/20) Installing libattr (2.4.47-r7)
(7/20) Installing libacl (2.2.52-r5)
(8/20) Installing coreutils (8.29-r2)
(9/20) Installing ldns (1.7.0-r0)
(10/20) Installing drill (1.7.0-r0)
(11/20) Installing ca-certificates (20171114-r3)
(12/20) Installing nghttp2-libs (1.32.0-r0)
(13/20) Installing libssh2 (1.8.2-r0)
(14/20) Installing libcurl (7.61.1-r2)
(15/20) Installing expat (2.2.5-r0)
(16/20) Installing pcre2 (10.31-r0)
(17/20) Installing git (2.18.1-r0)
(18/20) Installing libintl (0.19.8.1-r2)
(19/20) Installing libproc (3.3.15-r0)
(20/20) Installing procps (3.3.15-r0)
Executing busybox-1.28.4-r3.trigger
Executing ca-certificates-20171114-r3.trigger
OK: 30 MiB in 33 packages
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/1) Installing curl (7.61.1-r2)
Executing busybox-1.28.4-r3.trigger
OK: 30 MiB in 34 packages
Removing intermediate container 6a67ffb82183
---> 4c48d98a6397
Step 3/8 : RUN addgroup testssl && adduser -G testssl -g "testssl user" -s /bin/bash -D testssl && ln -s /home/testssl/testssl.sh /usr/local/bin/
---> Running in b8228f86c2d8
Removing intermediate container b8228f86c2d8
---> 00e3b4c07d46
Step 4/8 : USER testssl
---> Running in 2bb24a295ae3
Removing intermediate container 2bb24a295ae3
---> 3f4db89c90e0
Step 5/8 : WORKDIR /home/testssl/
---> Running in b3cc1f330b3e
Removing intermediate container b3cc1f330b3e
---> 2aa23c16f9fd
Step 6/8 : RUN git clone --depth=1 https://github.com/drwetter/testssl.sh.git .
---> Running in 024d6bc23ee4
Cloning into '.'...
Removing intermediate container 024d6bc23ee4
---> 74fedd3c41b0
Step 7/8 : ENTRYPOINT ["testssl.sh"]
---> Running in 250cefee46bd
Removing intermediate container 250cefee46bd
---> 50ca449a1da5
Step 8/8 : CMD ["--help"]
---> Running in 8e21bcdbe959
Removing intermediate container 8e21bcdbe959
---> 4985897a0628
Successfully built 4985897a0628
Successfully tagged testsslsh_testssl.sh:latest
WARNING: Image for service testssl.sh was built because it did not already exist. To rebuild this image you must use `docker-compose build` or `docker-compose up --build`.
###########################################################
testssl.sh 3.0rc4 from https://testssl.sh/dev/
(5b1fdfa 2019-04-02 09:29:13 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.0.2-chacha (1.0.2k-dev)" [~183 ciphers]
on testssl:$PWD/bin/openssl.Linux.x86_64
(built: "Jan 18 17:12:17 2019", platform: "linux-x86_64")
====== Running in file batch mode with file="input/testssl.sh" ======
========================================
/usr/local/bin/testssl.sh --warnings=batch -H --jsonfile reports --logfile reports --htmlfile reports --csvfile reports --append https://testssl.sh:443
Start 2019-04-02 08:06:51 -->> 81.169.166.184:443 (testssl.sh) <<--
Further IP addresses: 2a01:238:4308:a920:1000::e571:51
rDNS (81.169.166.184): --
Service detected: HTTP
Testing for heartbleed vulnerability
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
Done 2019-04-02 08:06:55 [ 6s] -->> 81.169.166.184:443 (testssl.sh) <<--
this reuses the same image.
$ docker-compose run testssl.sh --file input/linux.com.txt
###########################################################
testssl.sh 3.0rc4 from https://testssl.sh/dev/
(5b1fdfa 2019-04-02 09:29:13 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.0.2-chacha (1.0.2k-dev)" [~183 ciphers]
on testssl:$PWD/bin/openssl.Linux.x86_64
(built: "Jan 18 17:12:17 2019", platform: "linux-x86_64")
====== Running in file batch mode with file="input/linux.com.txt" ======
========================================
/usr/local/bin/testssl.sh --warnings=batch -H --csvfile reports https://linux.com:443
Testing all IPv4 addresses (port 443): 151.101.193.5 151.101.129.5 151.101.1.5 151.101.65.5
-----------------------------------------------------
Start 2019-04-02 08:09:06 -->> 151.101.193.5:443 (linux.com) <<--
Further IP addresses: 151.101.129.5 151.101.1.5 151.101.65.5
rDNS (151.101.193.5): --
Service detected: HTTP
Testing for heartbleed vulnerability
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
Done 2019-04-02 08:09:14 [ 9s] -->> 151.101.193.5:443 (linux.com) <<--
-----------------------------------------------------
Start 2019-04-02 08:09:14 -->> 151.101.129.5:443 (linux.com) <<--
Further IP addresses: 151.101.193.5 151.101.1.5 151.101.65.5
rDNS (151.101.129.5): --
Service detected: HTTP
Testing for heartbleed vulnerability
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
Done 2019-04-02 08:09:17 [ 12s] -->> 151.101.129.5:443 (linux.com) <<--
-----------------------------------------------------
Start 2019-04-02 08:09:17 -->> 151.101.1.5:443 (linux.com) <<--
Further IP addresses: 151.101.193.5 151.101.129.5 151.101.65.5
rDNS (151.101.1.5): --
Service detected: HTTP
Testing for heartbleed vulnerability
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
Done 2019-04-02 08:09:20 [ 15s] -->> 151.101.1.5:443 (linux.com) <<--
-----------------------------------------------------
Start 2019-04-02 08:09:20 -->> 151.101.65.5:443 (linux.com) <<--
Further IP addresses: 151.101.193.5 151.101.129.5 151.101.1.5
rDNS (151.101.65.5): --
Service detected: HTTP
Testing for heartbleed vulnerability
Heartbleed (CVE-2014-0160) not vulnerable (OK), no heartbeat extension
Done 2019-04-02 08:09:24 [ 19s] -->> 151.101.65.5:443 (linux.com) <<--
-----------------------------------------------------
Done testing now all IP addresses (on port 443): 151.101.193.5 151.101.129.5 151.101.1.5 151.101.65.5
$ docker-compose run testssl.sh --version
###########################################################
testssl.sh 3.0rc4 from https://testssl.sh/dev/
(5b1fdfa 2019-04-02 09:29:13 -- )
This program is free software. Distribution and
modification under GPLv2 permitted.
USAGE w/o ANY WARRANTY. USE IT AT YOUR OWN RISK!
Please file bugs @ https://testssl.sh/bugs/
###########################################################
Using "OpenSSL 1.0.2-chacha (1.0.2k-dev)" [~183 ciphers]
on testssl:$PWD/bin/openssl.Linux.x86_64
(built: "Jan 18 17:12:17 2019", platform: "linux-x86_64")$ docker-compose run --entrypoint /bin/bash testssl.sh --version
GNU bash, version 4.4.19(1)-release (x86_64-alpine-linux-musl)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Hi Stig,
great! There was a sloppy file creation statement for CSV files which seem to have worked in other cases but probably not with Alpine Linux' bash -- which is also used by the docker image @ dockerhub.
Thanks for the feedback and details provided!
Cheers, Dirk
There seem to be a problem with the CSV-generation. It creates a csv-file but it complains about No such file or directory. Running the same command but with JSON as output works without error. The resulting CSV-file looks OK though.
Runs in a docker container, based on the Dockerfile provided on the testssl site.
The error
Commands below, same scan with csv and json as output.
Linux version
SSL
The input-file