Import or update the vulnerability information problems

[ZupeiNie]

Hi， Documentation: When you deploy Eclipse Steady using Docker, not only the vulnerability data from project KB is automatically imported, but it is also periodically updated so that any new vulnerabilities are imported automatically into your Eclipse Steady backend. However, in the course of actual use, I found that there were only 124 vulnerabilities in my backend and 747 in the official library, whether there were problems in automatic updates. Second, when I use kaybee to add vulnerability information, I don't know how to value the parameter KB_IMPORTER_PATH. I didn't find kb-importer jar file. Can you help me? Thank you very much！

[serenaponta]

Hi @11111821 , Using the default configuration coming with docker/.env.sample (see [1] ), around 500 vulnerabilities should be imported because of KB_IMPORTER_SKIP_CLONE=True. As you only have 124, could you run docker logs steady-kb-importer to check what went wrong? In fact we are currently working on improving the initial import of vulnerabilities as it takes long (~2h as mentioned at [1]) and, with the images available in docker-hub, if the container is stopped during the initial import a flag needs to be manually removed to have to continue processing the vulnerabilities (removing kb-importer/data/running). Once the new docker images will be published (likely next week) this bug will be fixed.

KB_IMPORTER_PATH should contain the path to the executable jar that you can find in the volume mounted to the steady-kb-importer container, i.e., at kb-importer/data/kb-importer.jar.

[1] https://eclipse.github.io/steady/admin/tutorials/docker/#populatemaintain-the-vulnerability-database

[serenaponta]

Hi @11111821https://github.com/11111821 ,

You correctly used the value of BACKEND_BUGS_TOKEN to configure the variableUSER_TOKEN. By default, steady.sh always will returns the message "Please configure the necessary variables in the script and try again" to remind the user to configure the variables. The comment just above the echo points out that the echo has to be commented once the variables are configured as follows:

  ## COMMENT OUT THE NEXT LINE AND EDIT THE FOLLOWING LINES
#  echo "Please configure the necessary variables in the script and try again" && exit 1

The alternative is to already set the variables and comment out the line with "echo" in the kaybeeconf.yaml configuration file. In this way the echo will be already commented out for all steady.sh scripts you may generate using the corresponding configuration file.

As mentioned in my previous comment, the images of version 3.2.3 will be released next week. In the meantime, if you want to try again with version 3.2.2 you can set VULAS_RELEASE=3.2.2 in docker/.env. With version 3.2.2 is important to not stop the container until the initial import is complete and the daily cron job may encounter some issues. Sorry for the delay in releasing the new images.

From: sudo @.> Sent: mercredi 13 avril 2022 05:50 To: eclipse/steady @.> Cc: PONTA, Serena @.>; Comment @.> Subject: Re: [eclipse/steady] Import or update the vulnerability information problems (Issue #543)

@.*** Thank you very much for your reply With your help, I found the address of KB-importer. Jar and successfully obtained the steady.sh file after running the command Kaybee merge. Then when I continued to run the steady.sh file, I reported an error. Return "Please configure the necessary variables in the script and try again" I found the parameter BACKEND_BUGS_TOKEN in docker/.env and thought it was the value of USER_TOKEN. I can't find the problem. Can you help me?

In addition, after the above failure, I want to redeploy steady to see if the vulnerability will be uploaded automatically, but there is a problem that the image cannot be found. When I deployed steady before, it was still version 3.2.2. I saw that version 3.2.3 was updated on April 4th. Is it that the image of version 3.2.3 was not updated?

Here is a screenshot of my parameters [EFCAD90B-104D-4B59-B551-EB34ADD25F2A]https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F57307064%2F163096069-0823a3d8-9ef1-419d-a7d5-d9dcf8a94f82.png&data=04%7C01%7Cserena.ponta%40sap.com%7C564364b068f34fd6712f08da1d00b5e1%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C637854186139337816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=BXwUTk5%2BjMu7bphWJCgYoqNZQuO7PDEYUnv%2Faw6%2FC8U%3D&reserved=0

https://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fuser-images.githubusercontent.com%2F57307064%2F163096297-86ad4f16-504b-4c6b-ae07-5c6127cf13fd.png&data=04%7C01%7Cserena.ponta%40sap.com%7C564364b068f34fd6712f08da1d00b5e1%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C637854186139337816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=p62jmjJJBJR0PVJuffq8ah%2FIkuZP0FQDc4lXqCdrT4Y%3D&reserved=0

- Reply to this email directly, view it on GitHubhttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Feclipse%2Fsteady%2Fissues%2F543%23issuecomment-1097524761&data=04%7C01%7Cserena.ponta%40sap.com%7C564364b068f34fd6712f08da1d00b5e1%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C637854186139337816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=QFqN7g9%2F9SIBe9l5ohoai%2F%2FV7rciAnn1N1Ug%2BxLAkLE%3D&reserved=0, or unsubscribehttps://eur03.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAKGJZBHTAZJCWFZZREBOJULVEY77FANCNFSM5TGC356Q&data=04%7C01%7Cserena.ponta%40sap.com%7C564364b068f34fd6712f08da1d00b5e1%7C42f7676cf455423c82f6dc2d99791af7%7C0%7C0%7C637854186139337816%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=5Tp1IkEy4d7%2B2utReelXfmiJdJv0RKVdisBqJnTAr2w%3D&reserved=0. You are receiving this because you commented.Message ID: @.**@.>>

[ZupeiNie]

Hi， @serenaponta In the afternoon, I redeployed steady-3.2.2. At this time, the vulnerability information can be imported to the back end. Although the speed is very slow, it is feasible! I look forward to the release of steady-3.2.3, and thank you very much for your detailed reply!

[serenaponta]

@11111821, we just released steady 3.2.4 and published the corresponding docker images. In particular it contains some improvements to make kb-importer restart in case it was stopped before the initialization was done and fixes an issue with the cron job to keep the vulnerability database up to date. The performance improvement is still not part of release 3.2.4 and it's work in progress (#537)

[henrikplate]

Hello @11111821, Can this ticket be closed? I suggest that you watch the repo to be notified once #537 is completed and a new release is available.

[ZupeiNie]

hi @henrikplate Okay, no problem