Open Dexus opened 2 years ago
Actually if I try to show a report in any format (CSV, PDF, XML, TXT, etc) all the reports are empty.
Is this related to the same bug?
I'm not sure I use in general only PDF Format. Will check on Monday if there are other known issues about this topic. But to be clear, no it should not as far as I know.
I'm not sure I use in general only PDF Format. Will check on Monday if there are other known issues about this topic.
Yes I use PDF only as well, previously with the fix in SSC #249
#!/bin/sh
echo " --> updmap-sys --syncwithtrees"
yes 2> /dev/null | updmap-sys --syncwithtrees > /dev/null 2>&1 > /dev/null
echo " --> updmap-sys"
updmap-sys > /dev/null 2>&1 > /dev/null
echo " --> mktexlsr"
mktexlsr > /dev/null 2>&1 > /dev/null
echo " --> texhash"
texhash > /dev/null 2>&1 > /dev/null
echo " --> fmtutil-sys --all"
fmtutil-sys --all > /dev/null 2>&1 > /dev/null
echo " --> updmap-sys --force"
updmap-sys --force > /dev/null 2>&1 > /dev/null
exit 0
This fixed the PDF reports only, the other reports were working fine. But now all the reports are empty, so I'm wondering if it was related to this upstream bug.
Thanks for checking!
Are you using the default config on the webpage? Or did you set some different filters?
Are you using the default config on the webpage? Or did you set some different filters?
Defaults config.
apply_overrides=1 levels=hml rows=100 min_qod=70 first=1 sort-reverse=severity
Includes Notes and Overrides.
Should I run the /opt/setup/script/reportFix.sh?
Should I run the /opt/setup/script/reportFix.sh?
Never used this script, and have no idea if this will help or for which exact problem it was ever created.
Never used this script, and have no idea if this will help or for which exact problem it was ever created.
Its a GVM Reporting fix for lines over a certain amount. It says:
"This tool allows you to modify the max_rows_per_page setting. A larger number will allow you to export more data, but it will make the web UI load much slower. Any scan with more than 15000 results should be broken into multiple scans."
Won't run this then.
Have been testing since early this morning. All report formats result in 0byte downloads.
Notifications that are supposed to attach reports don't have any attachments.
My docker run has OPT_PDF="1" which has installed the necessary texlive deps:
ii texlive-base 2020.20210202-3 all TeX Live: Essential programs and files
ii texlive-binaries 2020.20200327.54578-7 amd64 Binaries for TeX Live
ii texlive-fonts-recommended 2020.20210202-3 all TeX Live: Recommended fonts
ii texlive-latex-base 2020.20210202-3 all TeX Live: LaTeX fundamental packages
ii texlive-latex-extra 2020.20210202-3 all TeX Live: LaTeX additional packages
ii texlive-latex-recommended 2020.20210202-3 all TeX Live: LaTeX recommended packages
ii texlive-pictures 2020.20210202-3 all TeX Live: Graphics, pictures, diagrams
https://github.com/Secure-Compliance-Solutions-LLC/GVM-Docker/issues/249 does not fix the situation.
Thanks.
can you move in you container to /var/lib/gvm/gvmd/report_formats/11e2bb5a-5dbf-496f-abe6-7ebb301aae68
and make a ls -alsR
root@36f20adb1648:/var/lib/gvm/gvmd/report_formats/11e2bb5a-5dbf-496f-abe6-7ebb301aae68# ls -alsR
.:
total 32
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 3 gvm gvm 4096 Mar 22 13:20 ..
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 5057e5cc-b825-11e4-9d0e-28d24461215b
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 77bd6c4a-1f62-11e1-abf0-406186ea4fc5
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 a3810a62-1f62-11e1-9219-406186ea4fc5
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 a994b278-1f62-11e1-96ac-406186ea4fc5
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 c1645568-627a-11e3-a660-406186ea4fc5
4 drwxr-xr-x 2 gvm gvm 4096 Apr 1 07:40 c402cc3e-b531-11e1-9163-406186ea4fc5
./5057e5cc-b825-11e4-9d0e-28d24461215b:
total 24
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
8 -rw-r--r-- 1 gvm gvm 6520 Mar 22 13:20 Anonymous_XML.xsl
4 -rwxr-xr-x 1 gvm gvm 904 Mar 22 13:20 generate
4 -rw-r--r-- 1 gvm gvm 394 Mar 22 13:20 report_format.xml
./77bd6c4a-1f62-11e1-abf0-406186ea4fc5:
total 20
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
4 -rwxr-xr-x 1 gvm gvm 1010 Mar 22 13:20 generate
4 -rw-r--r-- 1 gvm gvm 1587 Mar 22 13:20 ITG.xsl
4 -rw-r--r-- 1 gvm gvm 492 Mar 22 13:20 report_format.xml
./a3810a62-1f62-11e1-9219-406186ea4fc5:
total 60
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
4 -rwxr-xr-x 1 gvm gvm 1029 Mar 22 13:20 generate
4 -rw-r--r-- 1 gvm gvm 466 Mar 22 13:20 report_format.xml
44 -rw-r--r-- 1 gvm gvm 41283 Mar 22 13:20 TXT.xsl
./a994b278-1f62-11e1-96ac-406186ea4fc5:
total 16
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
4 -rwxr-xr-x 1 gvm gvm 961 Mar 22 13:20 generate
4 -rw-r--r-- 1 gvm gvm 357 Mar 22 13:20 report_format.xml
./c1645568-627a-11e3-a660-406186ea4fc5:
total 32
4 drwxr-xr-x 2 gvm gvm 4096 Mar 22 13:20 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
16 -rw-r--r-- 1 gvm gvm 15120 Mar 22 13:20 CSV_Results.xsl
4 -rwxr-xr-x 1 gvm gvm 880 Mar 22 13:20 generate
4 -rw-r--r-- 1 gvm gvm 659 Mar 22 13:20 report_format.xml
./c402cc3e-b531-11e1-9163-406186ea4fc5:
total 84
4 drwxr-xr-x 2 gvm gvm 4096 Apr 1 07:40 .
4 drwxr-xr-x 8 gvm gvm 4096 Mar 22 13:20 ..
4 -rwxr-xr-x 1 gvm gvm 1512 Apr 1 07:40 generate
68 -rw-r--r-- 1 gvm gvm 69517 Mar 22 13:20 latex.xsl
4 -rw-r--r-- 1 gvm gvm 482 Mar 22 13:20 report_format.xml
can you move in you container to
/var/lib/gvm/gvmd/report_formats/11e2bb5a-5dbf-496f-abe6-7ebb301aae68
and make als -alsR
I have no such directory.
root@c2fa4cdc630f:~ # cd /var/lib/gvm/gvmd/report_formats/11e2bb5a-5dbf-496f-abe6-7ebb301aae68
bash: cd: /var/lib/gvm/gvmd/report_formats/11e2bb5a-5dbf-496f-abe6-7ebb301aae68: No such file or directory
root@c2fa4cdc630f:~ # cd /var/lib/gvm/gvmd/report_formats
root@c2fa4cdc630f:/var/lib/gvm/gvmd/report_formats # ls
root@c2fa4cdc630f:/var/lib/gvm/gvmd/report_formats # ls -ailhR
.:
total 8.0K
1318923 drwxr-xr-x 2 gvm gvm 4.0K Apr 24 18:04 .
1318920 drwxr-xr-x 4 gvm gvm 4.0K Apr 24 18:37 ..
root@c2fa4cdc630f:/var/lib/gvm/gvmd/report_formats #
Have I done something wrong?
No but there is a difference between the docs and the code. (#15)
Please use the code from https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker/blob/devel/GVMDocker/scripts/sync-all.sh and add it to /opt/setup/scripts/sync-all.sh
to fix it please run also the script.
Need later to release the fixes to the latest images.
Thanks @Dexus, I’m not at my desk at the moment will come back and do it.
I copied the code from https://github.com/DeineAgenturUG/greenbone-gvm-openvas-for-docker/blob/devel/GVMDocker/scripts/sync-all.sh to my container /opt/setup/scripts/sync-all.sh. The only difference between them is:
root@c2fa4cdc630f:/opt/setup/scripts # diff sync-all.sh sync-all.sh.org
2,3c2,3
< set -Eeo pipefail
< if [[ ! "${AUTO_SYNC}" =~ ^(yes|y|YES|Y|true|TRUE)$ ]]; then
---
> set -Eeuo pipefail
> if [[ "${AUTO_SYNC}" != "YES" ]]; then
I ran the script, it did some updates, but my /var/lib/gvm/gvmd/report_formats
is still empty.
In my feed status, it says all the feeds are updating. After updating, I tried to generate different reoprts (TXT, CSV, etc) they are all still 0 bytes.
is the folder still empty?
maybe I found the bug. It looks like latex exits with error 1 because of
/tmp/tmp.g2YW3e2mf9/report.tex:235: Misplaced \noalign.
\rowcolor ->\noalign
{\ifnum 0=`}\fi \global \let \CT@do@color \CT@@do@color...
l.235 \rowcolor
{white}{\verb=Services=}\\
/tmp/tmp.g2YW3e2mf9/report.tex:235: ==> Fatal error occurred, no output PDF file produced!
But this is more a upstream bug, so I'm not able to do here something else as wait... not sure if it is latex bug or one for the greenbone side.
Yes, still empty.
root@c2fa4cdc630f:/var/lib/gvm/gvmd/report_formats # ls -ailh
total 8.0K
1318923 drwxr-xr-x 2 gvm gvm 4.0K Apr 24 18:04 .
1318920 drwxr-xr-x 4 gvm gvm 4.0K Apr 24 18:37 ..
I even restarted the container and let it update again.
what does the gvmd log tells you? /var/log/gvm/gvmd.log`
If there is no error to find, please try it with a complete clean system (all volume should be empty) and change after the feeds are synced and ~10 minutes idle time the database folder to your updated db and restart the container. I see this very very rarely and most when the system was stopped before the sync was done, that files are missing.
I see a whole bunch of these messages in /var/log/gvm/gvmd.log, which corresponds to every time my tasks try to generate a report or I manually try to generate a report of any type (TXT, CSV, PDF, etc).
md manage:WARNING:2022-04-25 02h32.42 +08:3897: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h33.03 +08:3909: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/a3810a62-1f62-11e1-9219-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h34.03 +08:4288: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h37.03 +08:4364: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/5057e5cc-b825-11e4-9d0e-28d24461215b/generate
md manage:WARNING:2022-04-25 02h37.38 +08:4383: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h39.52 +08:4449: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/a3810a62-1f62-11e1-9219-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h40.28 +08:4589: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 02h40.43 +08:4600: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c1645568-627a-11e3-a660-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h07.03 +08:5275: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h07.59 +08:5481: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h24.32 +08:6354: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h24.55 +08:6370: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/a3810a62-1f62-11e1-9219-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h38.20 +08:1118: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h40.07 +08:1219: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h46.56 +08:2610: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h47.05 +08:2671: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 03h59.42 +08:42514: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h03.50 +08:69296: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h07.00 +08:82231: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h07.47 +08:87860: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h07.49 +08:87823: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h19.21 +08:4127: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h47.12 +08:109642: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h47.42 +08:109550: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 04h53.46 +08:109460: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 05h35.43 +08:100097: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 06h00.56 +08:1879: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 06h22.29 +08:92130: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 06h23.53 +08:99018: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 08h04.21 +08:876576: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 09h52.39 +08:93147: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 11h58.50 +08:25464: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 19h57.19 +08:1121283: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 19h58.30 +08:1121508: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/c402cc3e-b531-11e1-9163-406186ea4fc5/generate
md manage:WARNING:2022-04-25 19h59.07 +08:1121531: run_report_format_script: No generate script found at /var/lib/gvm/gvmd/report_formats/27127ef5-e2b9-4d04-a1ed-368e97fd8448/a3810a62-1f62-11e1-9219-406186ea4fc5/generate
Ok will try this after dinner - thank you!
please try it with a complete clean system (all volume should be empty) and change after the feeds are synced and ~10 minutes idle time the database folder to your updated db and restart the container. I see this very very rarely and most when the system was stopped before the sync was done, that files are missing.
Ok will try this after dinner - thank you!
please try it with a complete clean system (all volume should be empty) and change after the feeds are synced and ~10 minutes idle time the database folder to your updated db and restart the container. I see this very very rarely and most when the system was stopped before the sync was done, that files are missing.
maybe a gvmd --rebuild-gvmd-data=report_formats
help there also
gvmd --help
...
--rebuild Remove NVT db, and rebuild it from the scanner.
--rebuild-gvmd-data=<types> Reload all gvmd data objects of a given types from feed. The types must be "all" or a comma-separated of the following: "configs", "port_lists" and "report_formats"
--rebuild-scap Rebuild all SCAP data.
...
I tried gvmd --rebuild-gvmd-data=report_formats
but it didn't work. I even tried gvmd --rebuild
and it didn't work also.
But then, I think I found the issue.
The directory name for report generation directory (/var/lib/gvm/gvmd/report_formats/
I checked my previous version of gvm container and found that the
Thanks for the help!
btw. why did you not reuse the volumes from the old one? only the DB volume need to be changed after the update to match PGSql 13.
btw. why did you not reuse the volumes from the old one? only the DB volume need to be changed after the update to match PGSql 13.
You’re right! Stupid of me. Sorry to waste your time on this issue, it was not intentional. Completely my fault.
No problem.
Describe the bug A upstream bug, that lets not create PDF if you not use the default report settings.
To Reproduce Steps to reproduce the behavior: ...
Expected behavior Create PDF with all kind of settings.
Screenshots
WORKING:
NOT WORKING:
Image in use: current latest
Additional GB Ticket: AP-1987