对on_command事件响应器的测试与预期不符

[zhanbao2000]

描述

1. 在对 on_command 事件响应器进行测试时，无论 .env 文件中 COMMAND_START 设置如何，均需要在测试时对给定的测试消息添加 / 前缀才能正常进行测试，否则 should_call_send、should_finished 等测试均不符合预期。

2. 在已对测试消息添加 / 前缀的情况下继续测试，若被 handle 装饰器装饰的函数（我称其为处理函数）设置了 event 形参，并通过 typing hint 将 event 形参的类型指定为 MessageEvent，那么 should_call_send、should_finished 等测试均依然不符合预期。

具体不符合预期的表现详见复现步骤和结果。

版本

• nonebot2: 2.0.0rc3
• nonebug: 0.2.3
• pytest: 7.2.1
• pytest-asyncio: 0.16.0

复现

插件：src/plugins/bug/__init__.py

from nonebot import on_command
from nonebot.adapters.onebot.v11 import Event, MessageEvent

t1 = on_command('t1')
t2 = on_command('t2')
t3 = on_command('t3')
t4 = on_command('t4')

@t1.handle()
async def _():
    await t1.finish("test1 ok")

@t2.handle()
async def _(event):
    await t2.finish("test2 ok")

@t3.handle()
async def _(event: Event):
    await t3.finish("test3 ok")

@t4.handle()
async def _(event: MessageEvent):
    await t4.finish("test4 ok")

单元测试：unittest/plugins/test_bug.py

import pytest
from nonebot.adapters.onebot.v11 import Message, MessageEvent
from nonebot.plugin import Plugin
from nonebug import App

@pytest.fixture
def load_plugin(nonebug_init: None) -> Plugin:
    import nonebot
    return nonebot.load_plugin('src.plugins.bug')

# 使用 / 前缀，并且处理函数没有加任何参数
@pytest.mark.asyncio
async def test_t1_prefix(app: App, load_plugin):
    from src.plugins.bug import t1

    async with app.test_matcher(t1) as ctx:
        bot = ctx.create_bot()

        msg = Message("/t1")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test1 ok", True)
        ctx.should_finished()

# 不使用 / 前缀，并且处理函数没有加任何参数
@pytest.mark.asyncio
async def test_t1(app: App, load_plugin):
    from src.plugins.bug import t1

    async with app.test_matcher(t1) as ctx:
        bot = ctx.create_bot()

        msg = Message("t1")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test1 ok", True)
        ctx.should_finished()

# 使用 / 前缀，处理函数添加 event 形参但不使用 typing hint
@pytest.mark.asyncio
async def test_t2(app: App, load_plugin):
    from src.plugins.bug import t2

    async with app.test_matcher(t2) as ctx:
        bot = ctx.create_bot()

        msg = Message("/t2")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test2 ok", True)
        ctx.should_finished()

# 使用 / 前缀，处理函数添加 event 形参并指定类型为 Event
@pytest.mark.asyncio
async def test_t3(app: App, load_plugin):
    from src.plugins.bug import t3

    async with app.test_matcher(t3) as ctx:
        bot = ctx.create_bot()

        msg = Message("/t3")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test3 ok", True)
        ctx.should_finished()

# 使用 / 前缀，处理函数添加 event 形参并指定类型为 MessageEvent
@pytest.mark.asyncio
async def test_t4(app: App, load_plugin):
    from src.plugins.bug import t4

    async with app.test_matcher(t4) as ctx:
        bot = ctx.create_bot()

        msg = Message("/t4")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test4 ok", True)
        ctx.should_finished()

环境：.env

HOST=...
PORT=...

SUPERUSERS=...
COMMAND_START=["","!","/"]
SESSION_EXPIRE_TIMEOUT=60

USERS_DB=data/users.db
GROUPS_DB=data/groups.db
TEMP=temp

结果

预期

5 个测试均成功

实际

测试用例	测试消息	处理函数的形参声明	结果
test_t1_prefix	/t1	async def _():	通过
test_t1	t1	async def _():	不通过
test_t2	/t2	async def _(event):	通过
test_t3	/t3	async def _(event: Event):	通过
test_t4	/t4	async def _(event: MessageEvent):	不通过

具体表现为：在未通过的两个测试中，无论测试用例填写什么预期结果，should_finished 方法都总是能通过。而 should_finished 方法则总是无法通过。

日志

============================= test session starts =============================
collecting ... collected 5 items

plugins/test_bug.py::test_t1_prefix 
plugins/test_bug.py::test_t1 02-03 00:21:23 [SUCCESS] nonebot | NoneBot is initializing...
02-03 00:21:23 [INFO] nonebot | Current Env: prod
02-03 00:21:24 [SUCCESS] nonebot | Succeeded to import "src.plugins.bug"
PASSED                               [ 20%]02-03 00:21:24 [INFO] nonebot | Matcher(type='message', module=src.plugins.bug) running complete
02-03 00:21:24 [SUCCESS] nonebot | NoneBot is initializing...
02-03 00:21:24 [INFO] nonebot | Current Env: prod
02-03 00:21:24 [SUCCESS] nonebot | Succeeded to import "src.plugins.bug"
FAILED                                      [ 40%]
plugins\test_bug.py:32 (test_t1)
app = <nonebug.app.App object at 0x0000028D9C27E100>
load_plugin = Plugin(name='bug', module=<module 'src.plugins.bug' from 'I:\\Developer\\Python\\project\\mokabot2\\src\\plugins\\bug\...c.plugins.bug), Matcher(type='message', module=src.plugins.bug)}, parent_plugin=None, sub_plugins=set(), metadata=None)

    @pytest.mark.asyncio
    async def test_t1(app: App, load_plugin):
        from src.plugins.bug import t1

        async with app.test_matcher(t1) as ctx:
            bot = ctx.create_bot()

            msg = Message("t1")
            event = MessageEvent(
                user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
                message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
            )

            ctx.receive_event(bot, event)
            ctx.should_call_send(event, "test1 ok", True)
>           ctx.should_finished()

plugins\test_bug.py:48: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\venv\lib\site-packages\nonebug\base.py:17: in __aexit__
    await self.run()
..\venv\lib\site-packages\nonebug\base.py:24: in run
    await self.run_test()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <nonebug.mixin.process.MatcherContext object at 0x0000028D9C27E430>

    async def run_test(self):
        from nonebot.rule import Rule, TrieRule
        from nonebot.matcher import current_handler
        from nonebot.exception import (
            PausedException,
            FinishedException,
            RejectedException,
        )

        with self.monkeypatch.context() as m:
            while self.action_list:
                # prepare for next event
                stack = AsyncExitStack()
                dependency_cache: T_DependencyCache = {}
                # fake event received
                receive_event = self.action_list.pop(0)
>               assert isinstance(
                    receive_event, ReceiveEvent
                ), f"Unexpected model {receive_event} expected ReceiveEvent"
E               AssertionError: Unexpected model Finished() expected ReceiveEvent
E               assert False
E                +  where False = isinstance(Finished(), ReceiveEvent)

..\venv\lib\site-packages\nonebug\mixin\process\__init__.py:107: AssertionError
02-03 00:21:24 [SUCCESS] nonebot | NoneBot is initializing...
02-03 00:21:24 [INFO] nonebot | Current Env: prod
02-03 00:21:24 [SUCCESS] nonebot | Succeeded to import "src.plugins.bug"
PASSED                                      [ 60%]02-03 00:21:24 [INFO] nonebot | Matcher(type='message', module=src.plugins.bug) running complete
02-03 00:21:24 [SUCCESS] nonebot | NoneBot is initializing...
02-03 00:21:24 [INFO] nonebot | Current Env: prod
02-03 00:21:25 [SUCCESS] nonebot | Succeeded to import "src.plugins.bug"
PASSED                                      [ 80%]02-03 00:21:25 [INFO] nonebot | Matcher(type='message', module=src.plugins.bug) running complete
02-03 00:21:25 [SUCCESS] nonebot | NoneBot is initializing...
02-03 00:21:25 [INFO] nonebot | Current Env: prod
02-03 00:21:25 [SUCCESS] nonebot | Succeeded to import "src.plugins.bug"
FAILED                                      [100%]02-03 00:21:25 [INFO] nonebot | Matcher(type='message', module=src.plugins.bug) running complete

plugins\test_bug.py:89 (test_t4)
app = <nonebug.app.App object at 0x0000028D9E439FA0>
load_plugin = Plugin(name='bug', module=<module 'src.plugins.bug' from 'I:\\Developer\\Python\\project\\mokabot2\\src\\plugins\\bug\...c.plugins.bug), Matcher(type='message', module=src.plugins.bug)}, parent_plugin=None, sub_plugins=set(), metadata=None)

    @pytest.mark.asyncio
    async def test_t4(app: App, load_plugin):
        from src.plugins.bug import t4

        async with app.test_matcher(t4) as ctx:
            bot = ctx.create_bot()

            msg = Message("/t4")
            event = MessageEvent(
                user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
                message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
            )

            ctx.receive_event(bot, event)
            ctx.should_call_send(event, "test4 ok", True)
>           ctx.should_finished()

plugins\test_bug.py:105: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
..\venv\lib\site-packages\nonebug\base.py:17: in __aexit__
    await self.run()
..\venv\lib\site-packages\nonebug\base.py:24: in run
    await self.run_test()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <nonebug.mixin.process.MatcherContext object at 0x0000028D9E5AF8B0>

    async def run_test(self):
        from nonebot.rule import Rule, TrieRule
        from nonebot.matcher import current_handler
        from nonebot.exception import (
            PausedException,
            FinishedException,
            RejectedException,
        )

        with self.monkeypatch.context() as m:
            while self.action_list:
                # prepare for next event
                stack = AsyncExitStack()
                dependency_cache: T_DependencyCache = {}
                # fake event received
                receive_event = self.action_list.pop(0)
>               assert isinstance(
                    receive_event, ReceiveEvent
                ), f"Unexpected model {receive_event} expected ReceiveEvent"
E               AssertionError: Unexpected model Finished() expected ReceiveEvent
E               assert False
E                +  where False = isinstance(Finished(), ReceiveEvent)

..\venv\lib\site-packages\nonebug\mixin\process\__init__.py:107: AssertionError

plugins/test_bug.py::test_t2 
plugins/test_bug.py::test_t3 
plugins/test_bug.py::test_t4 

========================= 2 failed, 3 passed in 2.22s =========================

额外补充

使用

@pytest.fixture
def load_plugin(nonebug_init: None) -> Plugin:
    import nonebot
    print(nonebot.get_driver().config)
    return nonebot.load_plugin('src.plugins.bug')

查看 Nonebug 的配置文件，可以得到：

driver='~fastapi' host=IPv4Address('127.0.0.1') port=8080 log_level='INFO' api_timeout=30.0 superusers=set() nickname=set() command_start={'/'} command_sep={'.'} session_expire_timeout=datetime.timedelta(seconds=120)

这与给定的 .env 明显不符，这应该是导致第一个问题的原因

[yanyongyu]

所有nonebot相关import全都要在函数内部进行

[yanyongyu]

第一个问题.env配置加载，你确定你的测试工作目录里是否有你要加载的.env？.env需要在你pytest启动的目录

[zhanbao2000]

第一个问题.env配置加载，你确定你的测试工作目录里是否有你要加载的.env？.env需要在你pytest启动的目录

.env文件确实不在pytest的启动目录下，而是在整个项目的根目录下，应该就是这个原因导致.env没有被加载。

我的项目结构大概这样：

│  .env
│  bot.py
│
├─src
│  ├─plugins
│  │  └─calculator
│  │          main.py
│  │          __init__.py
│  │
│  ├─plugins_preload
│  └─utils
└─unittest
    │  helper.py
    │
    ├─plugins
    │      test_calculator.py
    │
    ├─plugins_preload
    └─utils

稍后我尝试一下有没有合适的方法可以在测试的时候令nonebug载入给定的.env，如果你有好的见解也希望可以告诉我，谢谢

[zhanbao2000]

所有nonebot相关import全都要在函数内部进行

已经了解了，经过尝试，若将 test_t4 改成这样就能通过

@pytest.mark.asyncio
async def test_t4(app: App, load_plugin):
    from src.plugins.bug import t4
    from nonebot.adapters.onebot.v11 import Message, MessageEvent

    async with app.test_matcher(t4) as ctx:
        bot = ctx.create_bot()

        msg = Message("/t4")
        event = MessageEvent(
            user_id=1, message=msg, message_id=1, time=1, self_id=1, post_type="message",
            message_type="private", sub_type="friend", raw_message=str(msg), font=1, sender={}
        )

        ctx.receive_event(bot, event)
        ctx.should_call_send(event, "test4 ok", True)
        ctx.should_finished()

[zhanbao2000]

稍后我尝试一下有没有合适的方法可以在测试的时候令nonebug载入给定的.env

直接在 bot 项目目录下添加 run_test.py 即可

│  .env
│  bot.py
│  run_test.py
│
├─src
│  ├─plugins
│  │  └─calculator
│  │          main.py
│  │          __init__.py
│  │
│  ├─plugins_preload
│  └─utils
└─unittest
    │  helper.py
    │
    ├─plugins
    │      test_calculator.py
    │
    ├─plugins_preload
    └─utils

run_test.py：

import pytest

if __name__ == '__main__':
    pytest.main(['-v', 'unittest'])

[yanyongyu]

直接在项目目录运行pytest unittest就可以

[zhanbao2000]

直接在项目目录运行pytest unittest就可以

谢谢，学到了