search

fossasia / susi_server

SUSI.AI server backend - the Artificial Intelligence server for personal assistants https://susi.ai
GNU Lesser General Public License v2.1
2.5k stars 1.08k forks source link

Add integration tests for basic skill features #1398

Open spotlesscoder opened 5 years ago

spotlesscoder commented 5 years ago

As discovered on CCCamp 2019, there might be issues with server side components sometimes.

When fixing those or adding new features to the SUSI.AI skill language, it should be made sure that existing functionality is not broken.

For that, there should be a CI system which executes common skills using basic functionality such as basic conversations, calling external APIs (which should be mocked during the test), and checks the results.

norbusan commented 4 years ago

I think that can be done. We have a python api client that can be cloned during ci testing, and send of a set of test queries and check for correct answers.

I try to look into it ...

norbusan commented 4 years ago

@mariobehling @Orbiter THere seems to be something broken in the .travis.yml file:

      env:
        global:
          - GH_REF: github.com/fossasia/susi_server.git
          - secure: ...

that doesn't work, it needs to be GH_REF=... etc.

Then for tests, there are obviously some test run

      after_success:
        - if [ -e ./gradlew ]; then ./gradlew test jacocoTestReport;else gradle test jacocoTestReport;fi

but obviously these tests don't really do anything useful, at least the travis-ci output doesn't indicate any activity.

I guess that is the place that needs fixing, because the test directory already contains quite some test cases.