search

aquasecurity / vuln-list-update

Apache License 2.0
174 stars 99 forks source link

feat(openEuler): Add module for updating openEuler os cves #284

Closed wjunLu closed 5 months ago

wjunLu commented 7 months ago

Description

What's openEuler?

openEuler is an open source, free Linux distribution platform. The platform provides an open community for global developers to build an open, diversified, and architecture-inclusive software ecosystem. openEuler is also an innovative platform that encourages everyone to propose new ideas, explore new approaches, and practice new solutions.

Learn more, please visit https://www.openeuler.org/en/

Trivy does not support openEuler

We can see that the operating systems currently supported by trivy for security detection does not include openEuler(see https://aquasecurity.github.io/trivy/v0.50/docs/coverage/os/).

image

To support openEuler

Now, openEuler has 2,345,659 users, 18,072 contributors and 1,501 organization members(see https://datastat.openeuler.org/en/overview). It is necessary to support such a very mature open source operating system.

Discussion

Our discussion is here https://github.com/aquasecurity/trivy/discussions/6400

Relatived PRs

https://github.com/aquasecurity/trivy-db/pull/397

CLAassistant commented 7 months ago

CLA assistant check
All committers have signed the CLA.

knqyf263 commented 6 months ago

@DmitriyLewen Could you take a look?

wjunLu commented 6 months ago

@DmitriyLewen Could you take a look?

Thanks! I'm looking forward to your suggestions!

wjunLu commented 6 months ago

@knqyf263 Is the committer busy recently? Do you have any other suggestions?

DmitriyLewen commented 6 months ago

Hello @wjunLu I will try to check this PR today or tomorrow.

wjunLu commented 6 months ago

Hello @wjunLu I will try to check this PR today or tomorrow.

Thank you!

knqyf263 commented 6 months ago

@wjunLu We have other high priority tasks and will take some time to review this PR. Thanks for your patience.

wjunLu commented 6 months ago

Hello @wjunLu Left comments.

Can you also fork https://github.com/aquasecurity/vuln-list and push openEuler advisories to that repository (to make sure the CI/CD is working properly, repository contains all the advisories, etc.)?

OK! I will upload the files to vuln-list

wjunLu commented 6 months ago

Hello @wjunLu Left comments. Can you also fork https://github.com/aquasecurity/vuln-list and push openEuler advisories to that repository (to make sure the CI/CD is working properly, repository contains all the advisories, etc.)?

OK! I will upload the files to vuln-list

https://github.com/aquasecurity/vuln-list/pull/32

wjunLu commented 6 months ago

@DmitriyLewen Could you please check this again? Thank you!

wjunLu commented 6 months ago

Hello @wjunLu Left comments. Can you also fork https://github.com/aquasecurity/vuln-list and push openEuler advisories to that repository (to make sure the CI/CD is working properly, repository contains all the advisories, etc.)?

OK! I will upload the files to vuln-list

aquasecurity/vuln-list#32

I found this PR has been closed by bot

wjunLu commented 6 months ago

@knqyf263 @DmitriyLewen Please test the changes. I'm still looking forward your suggestions!

wjunLu commented 6 months ago

@DmitriyLewen @knqyf263 Hi, thank you for the previous comments, could you please check the new changes? or provide some suggestions for me. Thank you!

DmitriyLewen commented 6 months ago

Hello @wjunLu Unfortunately, we work on other, higher priority tasks. I will check your changes as soon as I have time.

Best regards, Dmitriy

wjunLu commented 6 months ago

Thanks very much!

2024年5月13日 10:50，DmitriyLewen @.***> 写道：

Hello @wjunLu https://github.com/wjunLu Unfortunately, we work on other, higher priority tasks. I will check your changes as soon as I have time.

Best regards, Dmitriy

— Reply to this email directly, view it on GitHub https://github.com/aquasecurity/vuln-list-update/pull/284#issuecomment-2106538864, or unsubscribe https://github.com/notifications/unsubscribe-auth/BAKVXQ7PJY6VA63AH2TVTXLZCAS5ZAVCNFSM6AAAAABF4PHYRGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCMBWGUZTQOBWGQ. You are receiving this because you were mentioned.

DmitriyLewen commented 6 months ago

I found this PR has been closed by bot

It's not a problem. I only asked about this so that I could see the changes. CI/CD will add these files after merging this PR.

wjunLu commented 6 months ago

Hello @wjunLu Left some comments.

Also tests are broken:

2024/05/15 12:28:20 failed to fetch CVRF data from repo.openEuler.org, err: [failed to fetch URL: HTTP error. status code: 404, url: http://127.0.0.1:54377/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1202.xml failed to fetch URL: HTTP error. status code: 404, url: http://127.0.0.1:54377/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1480.xml]

@DmitriyLewen Thank you very much! I have modified the codes based on your comments, and the testing result is OK!

=== RUN   TestConfig_Update
=== RUN   TestConfig_Update/positive_test_openEuler
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/15 17:21:04 Fetching openEuler CVRF data...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/15 17:21:04 Fetching openEuler CVRF 2021 data into openeuler/2021 ...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/15 17:21:04 Fetching openEuler CVRF 2023 data into openeuler/2023 ...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/15 17:21:04 Fetching openEuler CVRF 2024 data into openeuler/2024 ...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/15 17:21:04 Fetching openEuler CVRF 2022 data into openeuler/2022 ...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
=== RUN   TestConfig_Update/invalid_filesystem_write_read_only_path
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/15 17:21:04 Fetching openEuler CVRF data...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
=== RUN   TestConfig_Update/empty_file_format
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/15 17:21:04 Fetching openEuler CVRF data...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/15 17:21:04 empty CVRF xml
2024/05/15 17:21:04 Fetching openEuler CVRF 2021 data into openeuler/2021 ...
0 [_____________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________] ?% ? p/s
=== RUN   TestConfig_Update/invalid_file_format
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/15 17:21:04 Fetching openEuler CVRF data...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
=== RUN   TestConfig_Update/broken_XML
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/15 17:21:04 Fetching openEuler CVRF data...
1 / 1 [--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
--- PASS: TestConfig_Update (0.09s)
    --- PASS: TestConfig_Update/positive_test_openEuler (0.07s)
    --- PASS: TestConfig_Update/invalid_filesystem_write_read_only_path (0.01s)
    --- PASS: TestConfig_Update/empty_file_format (0.00s)
    --- PASS: TestConfig_Update/invalid_file_format (0.00s)
    --- PASS: TestConfig_Update/broken_XML (0.01s)
PASS
ok      github.com/aquasecurity/vuln-list-update/openeuler      1.852s
wjunLu commented 5 months ago

I don't have permission to update your PR. So i left 3 small refactoring comments to make it easier to maintain your changes.

Hi @DmitriyLewen , thank you very much! I made some changes based on your comments. Especially, the types struct now reproduces all information from vendor.

All testcases are passed as below:

=== RUN   TestConfig_Update
=== RUN   TestConfig_Update/positive_test_openEuler
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/17 16:28:12 Fetching openEuler CVRF data...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/17 16:28:12 Fetching openEuler CVRF 2021 data into openeuler/2021 ...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/17 16:28:12 Fetching openEuler CVRF 2023 data into openeuler/2023 ...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/17 16:28:12 Fetching openEuler CVRF 2024 data into openeuler/2024 ...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/17 16:28:12 Fetching openEuler CVRF 2022 data into openeuler/2022 ...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s=== RUN   TestConfig_Update/invalid_filesystem_write_read_only_path
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/17 16:28:12 Fetching openEuler CVRF data...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
=== RUN   TestConfig_Update/empty_file_format
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/17 16:28:12 Fetching openEuler CVRF data...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
2024/05/17 16:28:12 empty CVRF xml
2024/05/17 16:28:12 Fetching openEuler CVRF 2021 data into openeuler/2021 ...
0 [___________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________________] ?% ? p/s
=== RUN   TestConfig_Update/invalid_file_format
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/17 16:28:12 Fetching openEuler CVRF data...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
=== RUN   TestConfig_Update/broken_XML
    openeuler_test.go:87: http ready to start...
    openeuler_test.go:99: http started!
    openeuler_test.go:108: updating...
2024/05/17 16:28:12 Fetching openEuler CVRF data...
1 / 1 [------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------] 100.00% ? p/s
--- PASS: TestConfig_Update (0.08s)
    --- PASS: TestConfig_Update/positive_test_openEuler (0.06s)
    --- PASS: TestConfig_Update/invalid_filesystem_write_read_only_path (0.01s)
    --- PASS: TestConfig_Update/empty_file_format (0.00s)
    --- PASS: TestConfig_Update/invalid_file_format (0.00s)
    --- PASS: TestConfig_Update/broken_XML (0.01s)
PASS
ok      github.com/aquasecurity/vuln-list-update/openeuler      1.815s

Maybe I need to push the new data to https://github.com/aquasecurity/vuln-list again?

DmitriyLewen commented 5 months ago

I made some changes based on your comments. Especially, the types struct now reproduces all information from vendor.

Great! Thanks!

wjunLu commented 5 months ago

Sorry, let fix the CI issue!

wjunLu commented 5 months ago

LGTM now. @wjunLu Thanks for your work and patience!

@knqyf263 take a look, when you have time

@knqyf263 @DmitriyLewen Thank you very much!

wjunLu commented 5 months ago

Hi @knqyf263 , could you please merge this PR or give me some suggestions when you have time? Thank you very much!

wjunLu commented 5 months ago

Hi @DmitriyLewen, is @knqyf263 still busy with other higher-priority works? I sent an email but didn't receive a reply from him/her. What should I next to merge this PR?

knqyf263 commented 5 months ago

Hi @DmitriyLewen, is @knqyf263 still busy with other higher-priority works?

Yes

wjunLu commented 5 months ago

Hi @DmitriyLewen, is @knqyf263 still busy with other higher-priority works?

Yes

OK! Maybe I should use the patches first. Thank you!

knqyf263 commented 5 months ago

Thanks for your patience. Let's merge this PR and see how it goes.

wjunLu commented 5 months ago

Thanks for your patience. Let's merge this PR and see how it goes.

Thank you very much! The related PRs for using the openeuler-vuln-list are https://github.com/aquasecurity/trivy-db/pull/397 and https://github.com/aquasecurity/trivy/pull/6475, please review and comment when you have time.

knqyf263 commented 5 months ago

The data looks good. https://github.com/aquasecurity/vuln-list/tree/main/openeuler

I'll find some time and review those PRs. Thanks.