search

wechaty / friday

Friday is an Assistant BOT Built on Wechaty for Serving our Community
http://friday.chatie.io:5200
Apache License 2.0
83 stars 20 forks source link

make batch test for qna maker service in friday-bot #49

Open wj-Mcat opened 4 years ago

wj-Mcat commented 4 years ago

When we update the trian data for qna maker, we should keep qa accuracy for qna maker service. So, test data is essential.

We can follow the example journey from microsoft. But, the testing tools is based on .exe file, we can write a testing script which send restful generateAnswer request for every line testing data. And finnaly collect and analyze the test result. @huan How do you think about it ?

huan commented 4 years ago

Thanks for creating this discussion thread for our QnAMaker validation tests!

Basically, I have two ideas about this task:

  1. We should generalize the QnAMaker testing tasks so that it could be easy managing in the future
  2. We can use CI tools with Windows os to run win32 programs, or even we can use wine under Linux.

BTW: I'm also ok with a RESTFUL solution.

I'd like to suggest that you can write a project and try to solve this problem with some scripts, after that we can discuss based on your solution.

wj-Mcat commented 4 years ago

@huan Yes, I have create a simple qna-maker test script in qna-maker-batch-test repo. Let's talk about it and make it more suitable for qna-maker test

huan commented 4 years ago

Great!

Please feel free to go-ahead start improving our QnAMaker KB and adding some tests to your repository with the power of GitHub Actions.

Let's find the best practice by using it / practicing.