search

bytedance / Fastbot_iOS

About Fastbot(2.0) is a model-based testing tool for modeling GUI transitions to discover app stability problems
Other
556 stars 112 forks source link

Where can I get the test report? #2

Open jaderYang opened 3 years ago

jaderYang commented 3 years ago

Very appreciate your great tools. The tool is very convenient to use, too. I tried to create some crashes in my app, and then test it with Fastbot for two minutes. The app did crash, but Fastbot will restart it. When the time runs out, it will show that the test was successful. How can I get a report of how many times my app crashed during the test time and where did the crash occur?

jaderYang commented 3 years ago

也理我一下呀

Maxxxx24 commented 3 years ago

+1

geron-cn commented 3 years ago

一般都是通过被测试 App 预设的日志/crash 上报工具（例如 bugly）， 或者可以通过 xcode 或者idevicecrashreport收集异常。

另外， 工具集成报告这块我们也正在考虑集成进去🙏。

jaderYang commented 3 years ago

好的，理解了。 考虑到想把它集成进CI/CD，结果能及时确定发布流程是否能继续。 希望后面能实现这个feature吧。
再次感谢~

geron-cn commented 3 years ago

update：idevicecrashreport for windows https://github.com/libimobiledevice-win32/libimobiledevice-vs

defnngj commented 2 years ago

目前我们是通过 tidevice 运行 fastbot-iSO 进行Monkey 测试。这种方式可以通过 udid 指定设备，从而可以同时在多个设备上运行。

但是在终端运行的过程中日志量很大。

[D 211222 18:25:49 _device:917] logProcess: [fastbot] : state first visited, get reward  reward is 3.000000
    [fastbot] : action first visited, get reward 4.000000
    [fastbot] : got reward: 73.3714
    [fastbot] : build state cost: 0.001s action cost: 0.000s total cost 0.001s
[D 211222 18:25:49 _instruments:608] Ignore notification from server: 4093, 0x2, ('outputReceived:fromProcess:atTime:', ['[fastbot] : state first visited, get reward  reward is 3.000000 \n[fastbot] : action first visited, get reward 4.000000\n[fastbot] : got reward: 73.3714\n[fastbot] : build state cost: 0.001s action cost: 0.000s total cost 0.001s\n', 6338, 5398457728177])
[D 211222 18:25:49 _instruments:594] DTXMessage: expects_reply:1 flags:2 ('_XCT_logDebugMessage:', ['18:25:49.744 FastbotRunner-Runner[6338:807511] Handling kAXUserTestingNotification\n'])
[D 211222 18:25:49 _instruments:594] DTXMessage: expects_reply:0 flags:2 ('outputReceived:fromProcess:atTime:', ['2021-12-22 18:25:49.743771+0800 FastbotRunner-Runner[6338:806337] state g0s307 action SCROLL_LEFT_RIGHT pos[169,480,170,534] throttle 0.30\n', 6338, 5398461445941])
[D 211222 18:25:49 _device:917] logProcess: 2021-12-22 18:25:49.743771+0800 FastbotRunner-Runner[6338:806337] state g0s307 action SCROLL_LEFT_RIGHT pos[169,480,170,534] throttle 0.30

而且，tidevice 打印的这样的日志我不确定是否能捕捉到崩溃。

在运行过程，就算看到app 被重启了。那么会有一下情况。

  1. 有可能app 遍历结束，重新启动遍历，因为设置的时长未到。
  2. 在app 遍历过程中会切换到 别的app 操作，例如 微信、短信、浏览器等，然后又重新切回 App 。
  3. 真的碰到崩溃了，重新启动运行。

所以，从fastbot 工具端 如何发现崩溃，当前是比较迷惑。

当然，App 本身是有集成firebase 的，如果发现了崩溃会上报到 firebase平台，但从开发了解到的情况，他们有时很难通过firebase 去定位问题，因为不知道用户操作场景，就很难模拟。

想请教 @geron-cn , 1. 你们是如何收集崩溃信息的。 2. 如果 fastbot 工具集成报告的话，是否有进度，思路是什么？

geron-cn commented 2 years ago

update: release AnyTrace, the Fastbot test management assistant: supports one-click Fastbot test start, crash analysis, etc. AnyTrace User Manual

jeniva commented 2 years ago

windows上跑Fastbot_iOS的日志怎么收集呢，之前mac上我是挂载获取沙盒里面的app的日志和idevicecrashreport获取崩溃日志。现在换到Windows上来我不知道要咋获取日志了，我看https://github.com/libimobiledevice-win32/libimobiledevice-vs这个编译不成功，有没有现成可以用的办法？

feng12345678 commented 1 year ago

因为苹果升级到13.3系统，强制升级Xcode到14以上，导致无法运行Fastbot 项目。作者能更新一下兼容性吗