RedHatQE / polarize

GNU General Public License v3.0
2 stars 2 forks source link

Support the metadata required for automated metrics data collection #54

Open rarebreed opened 7 years ago

rarebreed commented 7 years ago

One of the initiatives for Platform QE is to automatically submit metadata about TestRuns above and beyond what is required in Polarion. The additional data they need looks something like this:

{"component": "test_run_id",
 "trigger": "manual",
 "tests": {"executor": "jenkins",
           "subtest": "",
           "arch": "x8664",  
           "executed": 15,
           "failed": 1,
           "passed": 14},
 "jenkins_job_url": "",
 "jenkins_build_url": "build_url",
 "logstash_url": "",
 "CI_tier": "3",
 "base_distro": "RHEL 7.4",
 "compose_id": "",
 "brew_task_id": "",
 "create_time": "",
 "completion_time": "",
 "CI_infra_failure": "",
 "CI_infra_failure_desc": "",
 "job_name": "stoner-scratch",
 "build_type": "",
 "team": "RHSM",
 "note": "",
 "recipients": "[\"stoner\", \"jsefler\", \"redakken\", \"shwetha\", \"jstavel\"]"}

There are two possibilities to do this:

  1. Add new elements under the element in the xunit file
  2. Publish a message with this JSON format to the appropriate topic on their Message Bus

If #1 is chosen, then the generateReport method in XUnitReporter will need to be modified to add all this metadata. If #2 is chosen, then the means to publish this to the Message bus will be required (CIBusPublisher along with new CLI tool).

Both methods will require grabbing some extra metadata and storing it into the polarize.properties file as shown by the required keys in the JSON message.

https://docs.engineering.redhat.com/display/CI/Automated+CI+metrics+data+collection

rarebreed commented 7 years ago

So, I am going to use the second approach where I will send the JSON message to the CI Message Bus. This may actually be marginally harder than the first approach, but I also think it will be more flexible. It will also be easier for other (java) teams to utilize. Finally, if we ever do the real-time reporting route, it will be better to do this approach (since if we go with the first approach, we wouldnt have all the data available yet anyway).

I am going to split out the solution to this problem outside of polarize, so that other java teams may download the jars separately.

rarebreed commented 7 years ago

I think the first step here is to create a java class that can (de)serialize this JSON type. The second step will be to split out the com.github.redhatqe.polarize.messagebus package into its own jar (which will then be a required dependency for polarize).

rarebreed commented 7 years ago

I have separated out the messaging part of polarize into its own library called polarize-bus. It is making use of a configuration library called byzantine. The goal is to eventually make polarize have a dependency on polarize-bus (the reason this was separated out was so that other teams could make use of it without having to install all of polarize along with all the configuration setup).

rarebreed commented 7 years ago

Right now polarize-bus can send a simple TextMessage type message to the main CI Bus server. It also uses RxJava2 and since there is a MessageListener attached to the ActiveMQ server, polarize-bus will react to events that are sent.

There's still some work to be done. Right now, polarize-bus will find messages (using the selector it was set with), but other than parsing and printing out the received message, it's not saved anywhere. In fact, I kind of dont like how it is saving some internal state, but I couldn't figure out another way to do it with RxJava2 (since the onNext handler returns void, so I had to save the state somewhere)