openeuler-riscv / oerv-team

OERV 实习生工作中心
11 stars 43 forks source link

验证软件包 cve-ease 的功能 #105

Open jiewu9823 opened 10 months ago

jiewu9823 commented 10 months ago

验证软件包 cve-ease 的功能 参考 https://gitee.com/openeuler/cve-ease

  1. 验证结果以及遇到的问题,请提交在本issue的评论里
  2. 如果遇到问题,请将问题同步到仓库 https://gitee.com/openeuler/cve-ease 的 issues中
TriangleABCD commented 8 months ago

验证 cve-ease 功能

0. 验证环境

  1. openEuler 23.09 x86_64 on VMWare Workstation Pro 17
  2. openEuler 23.09 RISC-V on Licheepi 4A

1. 安装验证

1.0 直接安装

按照步骤安装后尝试运行均出现同一问题,以 23.09 x86_64 的报错信息为例:

Traceback (most recent call last):
  File "/usr/bin/cve-ease", line 18, in <module>
    import cve_ease as ease
  File "/usr/local/lib/python3.11/site-packages/cve_ease/__init__.py", line 14, in <module>
    from .sql import SQLSession, activate_session, purge_db
  File "/usr/local/lib/python3.11/site-packages/cve_ease/sql.py", line 13, in <module>
    from cve_ease.models import DBModel
  File "/usr/local/lib/python3.11/site-packages/cve_ease/models/__init__.py", line 11, in <module>
    from .base import DBModel
  File "/usr/local/lib/python3.11/site-packages/cve_ease/models/base.py", line 11, in <module>
    from sqlalchemy.ext.declarative import declarative_base
ModuleNotFoundError: No module named 'sqlalchemy'

缺少了 python 中的 sqlalchemy 库。尝试使用 pip 安装之后,陆续发现一共缺少的 python 依赖库有:

1.1 容器安装

make run-in-docker 失败,出现如下报错信息:

Package systemd-243-31.oe1.x86_64 is already installed.
Package python3-3.7.9-6.oe1.x86_64 is already installed.
No match for argument: python3-defusedxml
Error: Unable to find a match: python3-defusedxml
The command '/bin/sh -c yum makecache     && yum install -y systemd make python3 python3-requests python3-flask             python3-sqlalchemy python3-pandas python3-dbus python3-defusedxml     && pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple flask_sqlalchemy     && ln -svf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime     && rm -f /etc/profile.d/system-info.sh' returned a non-zero code: 1

1.2 rpm包安装

按照文档步骤可以正常安装,只是在最后一步 rpm -ivh *.rpm 之前,需要提前安装所需的 python 依赖:

sudo dnf install python3-defusedxml python3-flask python3-pandas python3-sqlalchemy python3-requests

2. 基础配置验证

$ sudo systemctl status cve-ease.timer
● cve-ease.timer - CTyunOS cve-ease Project
     Loaded: loaded (/usr/lib/systemd/system/cve-ease.timer; enabled; preset: disabled)
     Active: active (running) since Wed 2024-01-31 21:55:37 CST; 10s ago
      Until: Wed 2024-01-31 21:55:37 CST; 10s ago
    Trigger: n/a
   Triggers: ● cve-ease.service
       Docs: https://gitee.com/openeuler/cve-ease

1月 31 21:55:37 localhost.localdomain systemd[1]: Started CTyunOS cve-ease Project.
$ sudo systemctl status cve-ease.service
○ cve-ease.service - CTyunOS cve-ease project
     Loaded: loaded (/usr/lib/systemd/system/cve-ease.service; enabled; preset: disabled)
     Active: inactive (dead) since Wed 2024-01-31 21:56:15 CST; 1min 41s ago
TriggeredBy: ● cve-ease.timer
       Docs: https://gitee.com/openeuler/cve-ease
    Process: 4340 ExecStart=/usr/bin/cve-ease daemon (code=exited, status=0/SUCCESS)
   Main PID: 4340 (code=exited, status=0/SUCCESS)

1月 31 21:55:37 localhost.localdomain systemd[1]: Starting CTyunOS cve-ease project...
1月 31 21:56:15 localhost.localdomain systemd[1]: cve-ease.service: Deactivated successfully.
1月 31 21:56:15 localhost.localdomain systemd[1]: Finished CTyunOS cve-ease project.

3. basic 类命令验证

3.0 config 配置相关子命令

$ sudo cve-ease config -r
 config file is in /etc/cve-ease/cve-ease.cfg
# 注意: 默认token仅为测试使用,您在使用时需根据里播报方式不同申请token 进行替换
[main]
pid_file_path = /var/log/cve-ease/cve-ease.pid
lock_file_path = /var/log/cve-ease/cve-ease.lock

# log configuration
log_file_path = /var/log/cve-ease/cve-ease.log
## debug/ error(default) / warn
log_level = error
log_maxbytes = 10240
log_backup_num = 30

# sql configuration
db_type = sqlite
db_file_path = /usr/share/cve-ease/cve-ease.db
......

3.1 service 相关子命令

$ sudo cve-ease service -s
 * On-boot status cve-ease.timer: enabled
 * Active cve-ease.timer: active waiting
$ sudo cve-ease service -k
 * Stop cve-ease.timer: /org/freedesktop/systemd1/job/1086
 * Disable cve-ease.timer: dbus.Array([dbus.Struct((dbus.String('unlink'), dbus.String('/etc/systemd/system/timers.target.wants/cve-ease.timer'), dbus.String('')), signature=None)], signature=dbus.Signature('(sss)'))
$ sudo cve-ease service -s
 * On-boot status cve-ease.timer: disabled
 * Active cve-ease.timer: dead
$ sudo cve-ease service -r
 * Stop cve-ease.timer: /org/freedesktop/systemd1/job/1087
 * Disable cve-ease.timer: dbus.Array([], signature=dbus.Signature('(sss)'))
 * Enable cve-ease.timer: (dbus.Boolean(True), dbus.Array([dbus.Struct((dbus.String('symlink'), dbus.String('/etc/systemd/system/timers.target.wants/cve-ease.timer'), dbus.String('/usr/lib/systemd/system/cve-ease.timer')), signature=None)], signature=dbus.Signature('(sss)')))
 * Start cve-ease.timer: /org/freedesktop/systemd1/job/1172
$ sudo cve-ease service -s
 * On-boot status cve-ease.timer: enabled
 * Active cve-ease.timer: active running

4. info 类命令验证

4.0 cve 子命令:爬取openEuler社区CVE公告信息

$ sudo cve-ease cve -r
 * scrapy from OpenEuler done
 * total record num: 6867
{
    "code": 0,
    "msg": "",
    "result": {
        "totalCount": 6867,
        "securityNoticeList": [],
        "cveDatabaseList": [
            {
                "id": 23912,
                "affectedProduct": "",
                "announcementTime": "2024-01-30",
                "attackComplexityNVD": "",
                "attackComplexityOE": "",
                "attackVectorNVD": "",
                "attackVectorOE": "",
                "availabilityNVD": "",
                "availabilityOE": "",
                "confidentialityNVD": "",
                "confidentialityOE": "",
                "cveId": "CVE-2024-21733",
                "cvsssCoreNVD": "5.3",
                "cvsssCoreOE": "7.5",
......
$ sudo cve-ease cve -m
 * scrapy from OpenEuler done
 * total record num: 6867
 * makecache done
$ sudo cve-ease cve -t
cve total record : 6867
$ sudo cve-ease cve -l
CVE-2018-8050   afflib-3.7.16-9.               6.5        2020-06-16 11:18:14
CVE-2017-6827   audiofile-0.3.6-24             7.8        2020-06-16 11:18:14
CVE-2017-6837   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
CVE-2017-6836   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
CVE-2017-6835   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
CVE-2017-6834   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
CVE-2017-6833   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
CVE-2017-6832   audiofile-0.3.6-24             5.5        2020-06-16 11:18:14
......

4.1 sa 子命令:爬取openEuler社区安全公告信息

$ sudo cve-ease sa -r
 * scrapy from OpenEuler done
 * total record num: 2403
{
    "code": 0,
    "msg": "",
    "result": {
        "totalCount": 2403,
        "securityNoticeList": [
            {
                "id": 4152,
                "affectedComponent": "tomcat",
                "affectedProduct": "openEuler-20.03-LTS-SP1;openEuler-20.03-LTS-SP4;openEuler-22.03-LTS;openEuler-22.03-LTS-SP1;openEuler-22.03-LTS-SP2;openEuler-22.03-LTS-SP3",
                "announcementTime": "2024-01-30",
                "cveId": "CVE-2024-21733;",
                "description": "",
                "introduction": "",
                "packageName": "",
                "referenceDocuments": "",
                "revisionHistory": "",
......
$ sudo cve-ease sa -m
 * scrapy from OpenEuler done
 * total record num: 2403
 * makecache done
$ sudo cve-ease sa -t
security notice total record : 2403
$ sudo cve-ease sa -l
openEuler-SA-2020-1001  2020-05-13 01:33:04  kernel                         CVE-2019-20636
openEuler-SA-2020-1002  2020-05-13 01:33:04  httpd-2.4.34-18                CVE-2019-10082
openEuler-SA-2020-1003  2020-05-13 01:33:04  gnutls                         CVE-2020-11501
openEuler-SA-2020-1004  2020-06-17 08:49:01  java-1.8.0-openjdk             CVE-2020-2803; CVE-2020-2805; CVE-2020-2781; CVE-2020-2830; CVE-2020-2800; CVE-2020-2754; CVE-2020-2755; CVE-2020-2773; CVE-2020-2756; CVE-2020-2757;
openEuler-SA-2020-1005  2020-06-17 08:33:41  lxc-4.0.1-2020052701           CVE-2019-5736
......

4.2 cvrf 子命令

$ sudo cve-ease cvrf -m
[ 6/2275 ] Download https://repo.openeuler.org/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1006.xml save to: /usr/share/cve-ease/cvrf/2021/cvrf-openEuler-SA-2021-1006.xml
[ 11/2275 ] Download https://repo.openeuler.org/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1011.xml save to: /usr/share/cve-ease/cvrf/2021/cvrf-openEuler-SA-2021-1011.xml
[ 18/2275 ] Download https://repo.openeuler.org/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1018.xml save to: /usr/share/cve-ease/cvrf/2021/cvrf-openEuler-SA-2021-1018.xml
[ 29/2275 ] Download https://repo.openeuler.org/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1029.xml save to: /usr/share/cve-ease/cvrf/2021/cvrf-openEuler-SA-2021-1029.xml
[ 35/2275 ] Download https://repo.openeuler.org/security/data/cvrf/2021/cvrf-openEuler-SA-2021-1035.xml save to: /usr/share/cve-ease/cvrf/2021/cvrf-openEuler-SA-2021-1035.xml
......
$ sudo cve-ease cvrf -t
cvrf total record : 2275
$ sudo cve-ease cvrf -l
SecurityNoticeNo: openEuler-SA-2021-1001
PackageName: dnsmasq
CVEID: CVE-2020-25681;CVE-2020-25685;CVE-2020-25682;CVE-2020-25683;CVE-2020-25684;CVE-2020-25687;CVE-2020-25686;
Affected Product: openEuler-20.03-LTS-SP1;openEuler-20.03-LTS;

SecurityNoticeNo: openEuler-SA-2021-1002
PackageName: sudo
CVEID: CVE-2021-3156;CVE-2021-23240;CVE-2021-23239;
Affected Product: openEuler-20.03-LTS-SP1;openEuler-20.03-LTS;

SecurityNoticeNo: openEuler-SA-2021-1003
PackageName: kernel
CVEID: CVE-2020-14351;CVE-2020-29661;CVE-2020-12352;CVE-2020-27777;CVE-2020-29660;CVE-2020-29569;CVE-2020-27815;CVE-2020-25656;CVE-2020-27830;CVE-2020-27675;
Affected Product: openEuler-20.03-LTS-SP1;openEuler-20.03-LTS;
......

4.3 rpm 子命令

$ sudo cve-ease rpm -l
NAME                           VERSION              EPOCH  RELEASE              ARCH            SOURCE(SNAME)
kexec-tools                    2.0.26               0     1.oe2309             x86_64          kexec-tools-2.0.26-1.oe2309.src.rpm(kexec-tools)
perl-B-Debug                   1.26                 0     6.oe2309             noarch          perl-B-Debug-1.26-6.oe2309.src.rpm(perl-B-Debug)
tzdata                         2022g                0     5.oe2309             noarch          tzdata-2022g-5.oe2309.src.rpm(tzdata)
ncurses-base                   6.4                  0     6.oe2309             noarch          ncurses-6.4-6.oe2309.src.rpm(ncurses)
libreport-filesystem           2.17.10              0     1.oe2309             noarch          libreport-2.17.10-1.oe2309.src.rpm(libreport)
fonts-filesystem               4.0.2                0     2.oe2309             noarch          fonts-rpm-macros-4.0.2-2.oe2309.src.rpm(fonts-rpm-macros)
abattis-cantarell-fonts        0.303.1              0     3.oe2309             noarch          abattis-cantarell-fonts-0.303.1-3.oe2309.src.rpm(abattis-cantarell-fonts)
dnf-data                       4.16.2               0     1.oe2309             noarch          dnf-4.16.2-1.oe2309.src.rpm(dnf)
......

4.4 repodata 子命令

$ sudo cve-ease repodata -m
 * Using product: openEuler-20.03-LTS-SP1
makecache https://mirrors.tuna.tsinghua.edu.cn/openeuler/openEuler-20.03-LTS-SP1/everything/x86_64/
makecache https://mirrors.tuna.tsinghua.edu.cn/openeuler/openEuler-20.03-LTS-SP1/update/x86_64/
[     1/28575 ] Cache OK! CUnit-2.1.3-22.oe1.x86_64.rpm
[     2/28574 ] Cache OK! CUnit-devel-2.1.3-22.oe1.x86_64.rpm
[     3/28573 ] Cache OK! CUnit-help-2.1.3-22.oe1.noarch.rpm
[     4/28572 ] Cache OK! CreateImage-0.0.5-31.oe1.oe1.noarch.rpm
[     5/28571 ] Cache OK! GConf2-3.2.6-24.oe1.x86_64.rpm
[     6/28570 ] Cache OK! GConf2-devel-3.2.6-24.oe1.x86_64.rpm
[     7/28569 ] Cache OK! GeoIP-1.6.12-5.oe1.x86_64.rpm
[     8/28568 ] Cache OK! GeoIP-GeoLite-data-2018.06-3.oe1.noarch.rpm
[     9/28567 ] Cache OK! GeoIP-devel-1.6.12-5.oe1.x86_64.rpm
[    10/28566 ] Cache OK! GeoIP-help-1.6.12-5.oe1.noarch.rpm
[    11/28565 ] Cache OK! GraphicsMagick-1.3.30-9.oe1.x86_64.rpm
[    12/28564 ] Cache OK! GraphicsMagick-c++-1.3.30-9.oe1.x86_64.rpm
[    13/28563 ] Cache OK! GraphicsMagick-c++-devel-1.3.30-9.oe1.x86_64.rpm
......
$ sudo cve-ease repodata --osv ctyunos2 -p openEuler-22.03-LTS -c
 * Using product: openEuler-22.03-LTS
 * Setup OSV : ctyunos2
check done!

4.5 logger 子命令

$ sudo cve-ease logger -t
cve record num: 0
sa record num: 0
log expiration days: 14

4.6 db 子命令

$ sudo cve-ease db -p
Danger! This operations is very dangerous!
Please make sure three times, if you still want to purge db, press 'yes' :

Purge/Recreate database done!
$ sudo cve-ease db -s
database file  /usr/share/cve-ease/cve-ease.db
CVE record num: 0
SA record num: 0
CVELOG record num: 0
SALOG record num: 0

5. notifier 类命令验证

5.0 wecom企业微信群聊机器人

$ sudo cve-ease wecom -t -c "helloworld"
watcher all_pkg_watcher wecom_notifier send done!
watcher all_pkg_watcher wecom_notifier send done!
watcher kernel_pkg_watcher wecom_notifier send done!
watcher kernel_pkg_watcher wecom_notifier send done!
watcher hotpkg_watcher wecom_notifier send done!
watcher hotpkg_watcher wecom_notifier send done!
watcher higher_score_watcher wecom_notifier send done!
watcher higher_score_watcher wecom_notifier send done!

5.1 dingding钉钉群聊机器人

$ sudo cve-ease dingding -t -c "helloworld"
2024-02-29 23:17:10,507 [ERROR] cve-ease: url:
https://oapi.dingtalk.com/robot/send?access_token=81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb
2024-02-29 23:17:10,509 [ERROR] cve-ease: request:
{
    "msgtype": "text",
    "text": {
        "content": "Msg from CVE-EASE: \n\u53d1\u5e03\u65f6\u95f4: 2024-02-29 23:17:09\ndingding_notifier\u6d4b\u8bd5\nIP: 114.222.98.43\nWatcher: all_pkg_watcher\n\u529f\u80fd\u63cf\u8ff0:\n \u7528\u4e8e\u76d1\u6d4b\u6240\u6709CVE\u4e0eSA\uff08\u5b89\u5168\u516c\u544a\uff09\u66f4\u65b0\u64ad\u62a5\u3001\u72b6\u6001\u64ad\u62a5\n\u9644\u52a0\u4fe1\u606f:\nhelloworld"
    }
}
2024-02-29 23:17:10,510 [ERROR] cve-ease: response:
{"errcode":400102,"errmsg":"description:机器人已经停用或者未启用;solution:请让企业管理员前往开放平台后台启用对应机器人 :https://open-dev.dingtalk.com/#/"}
2024-02-29 23:17:10,510 [ERROR] cve-ease: [-] dingding_notifier 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb send mesg failed!
watcher all_pkg_watcher dingding_notifier send done!

5.2 feishu飞书群聊机器人

$ sudo cve-ease feishu  -t -c "helloworld"
watcher all_pkg_watcher feishu_notifier send done!
watcher all_pkg_watcher feishu_notifier send done!
watcher kernel_pkg_watcher feishu_notifier send done!
watcher kernel_pkg_watcher feishu_notifier send done!
watcher hotpkg_watcher feishu_notifier send done!
watcher hotpkg_watcher feishu_notifier send done!
watcher higher_score_watcher feishu_notifier send done!
watcher higher_score_watcher feishu_notifier send done!

5.3 mail163邮箱

$ sudo cve-ease mail163 -t -c "helloworld"
watcher all_pkg_watcher mail163_notifier send done!
watcher kernel_pkg_watcher mail163_notifier send done!
watcher hotpkg_watcher mail163_notifier send done!
watcher higher_score_watcher mail163_notifier send done!

5.4 mailqq邮箱

$ sudo cve-ease mailqq -t -c "helloworld"
watcher all_pkg_watcher mailqq_notifier send done!
watcher kernel_pkg_watcher mailqq_notifier send done!
watcher hotpkg_watcher mailqq_notifier send done!
watcher higher_score_watcher mailqq_notifier send done!
TriangleABCD commented 8 months ago

问题汇总

1. 直接安装和 rpm 安装缺少依赖

直接安装时按照步骤安装后尝试运行均出现同一问题,以 23.09 x86_64 的报错信息为例:

Traceback (most recent call last):
  File "/usr/bin/cve-ease", line 18, in <module>
    import cve_ease as ease
  File "/usr/local/lib/python3.11/site-packages/cve_ease/__init__.py", line 14, in <module>
    from .sql import SQLSession, activate_session, purge_db
  File "/usr/local/lib/python3.11/site-packages/cve_ease/sql.py", line 13, in <module>
    from cve_ease.models import DBModel
  File "/usr/local/lib/python3.11/site-packages/cve_ease/models/__init__.py", line 11, in <module>
    from .base import DBModel
  File "/usr/local/lib/python3.11/site-packages/cve_ease/models/base.py", line 11, in <module>
    from sqlalchemy.ext.declarative import declarative_base
ModuleNotFoundError: No module named 'sqlalchemy'

缺少了 python 中的 sqlalchemy 库。尝试安装之后,陆续发现一共缺少的 python 依赖库有:

rpm 安装时按照文档步骤可以正常安装,只是在最后一步 rpm -ivh *.rpm 之前,需要提前安装所需的 python 依赖:

sudo dnf install python3-defusedxml python3-flask python3-pandas python3-sqlalchemy python3-requests

2. docker 安装报错

make run-in-docker 失败,出现如下报错信息:

Package systemd-243-31.oe1.x86_64 is already installed.
Package python3-3.7.9-6.oe1.x86_64 is already installed.
No match for argument: python3-defusedxml
Error: Unable to find a match: python3-defusedxml
The command '/bin/sh -c yum makecache     && yum install -y systemd make python3 python3-requests python3-flask             python3-sqlalchemy python3-pandas python3-dbus python3-defusedxml     && pip3 install -i https://pypi.tuna.tsinghua.edu.cn/simple flask_sqlalchemy     && ln -svf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime     && rm -f /etc/profile.d/system-info.sh' returned a non-zero code: 1

3. 钉钉群聊机器人的测试 token 已经无法使用

$ sudo cve-ease dingding -t -c "helloworld"
2024-02-29 23:17:10,507 [ERROR] cve-ease: url:
https://oapi.dingtalk.com/robot/send?access_token=81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb
2024-02-29 23:17:10,509 [ERROR] cve-ease: request:
{
    "msgtype": "text",
    "text": {
        "content": "Msg from CVE-EASE: \n\u53d1\u5e03\u65f6\u95f4: 2024-02-29 23:17:09\ndingding_notifier\u6d4b\u8bd5\nIP: 114.222.98.43\nWatcher: all_pkg_watcher\n\u529f\u80fd\u63cf\u8ff0:\n \u7528\u4e8e\u76d1\u6d4b\u6240\u6709CVE\u4e0eSA\uff08\u5b89\u5168\u516c\u544a\uff09\u66f4\u65b0\u64ad\u62a5\u3001\u72b6\u6001\u64ad\u62a5\n\u9644\u52a0\u4fe1\u606f:\nhelloworld"
    }
}
2024-02-29 23:17:10,510 [ERROR] cve-ease: response:
{"errcode":400102,"errmsg":"description:机器人已经停用或者未启用;solution:请让企业管理员前往开放平台后台启用对应机器人 :https://open-dev.dingtalk.com/#/"}
2024-02-29 23:17:10,510 [ERROR] cve-ease: [-] dingding_notifier 81907155a6cc88004e1ed6bcdd86c68d5b21565ed59d549ca031abc93d90d9cb send mesg failed!
watcher all_pkg_watcher dingding_notifier send done!

4. 163 邮箱和 qq 邮箱的邮件发送测试未收到邮件

TriangleABCD commented 8 months ago

已在对应码云仓库提交issue : https://gitee.com/openeuler/cve-ease/issues/I94VFB?from=project-issue