kiwitcms / Kiwi

open source test management system with over 2 million downloads!
https://kiwitcms.org
GNU General Public License v2.0
977 stars 292 forks source link

RFC: Start automated tests from webIU [figure out architecture details] #914

Open atodorov opened 5 years ago

atodorov commented 5 years ago

While the preferred way to integrate automation test results is from the bottom-up, that is via plugins which read the results from the test runner or automation framework and send them to Kiwi TCMS there is a valid use-case for the opposite as well.

A few people have asked for a "BUTTON" inside TestRun and/or individual TestExecutions which will ping back their test infrastructure and schedule new execution. In this context the Script and Arguments field of TestCase may be used to point to specific scripts to be executed. Environment from the TestRun may also be needed.

This RFC is to discuss possible implementations. The following comes to my mind:

1) Use signals: this is what Django does in such cases. That will require modification of settings and possibly custom code to attach your own signal handler. Will likely require to rebuild the standard docker image.

Signals in Django are synchronous so you will either have to fork threads inside the signal handler or send messages (e.g. to a RabbitMQ server).

2) Directly skip the signals part and use Celery to send to a message bus. Will also need you to configure the message bus connection via settings. In this scenario I can see how quickly someone will also request that Kiwi TCMS responds to messages, e.g. for setting status, which I don't feel is what we want to do.

3) Combine 1) and 2) with an add-on implementation that responds to the signal and sends the messages over to Celery.

4) Use web-hooks, multiple entries. Kiwi TCMS will ping all the configured web hooks, possibly without waiting for an answer (so it doesn't block for long periods of time) or forking into a separate thread to call the web hooks. This can be made configurable via Admin Panel.

Other ideas, suggestions, comments ?

atodorov commented 2 years ago

@sandeepV2 I believe you posted this comment earlier:

we have Jenkins job that is configured to run test cases. Is there an option in TCMS where i click "test run" button and it invokes the respective Jenkin job. Can it be achieved ?, if yes how?

The functionality you are looking for is discussed in this issue. You can subscribe to it to follow the updates.