salabs / TestArchiver

Tools for serialising test results to SQL database
Apache License 2.0
34 stars 22 forks source link

[Question - RobotFramework Database listener] - how to map log from 'log_message' table to keyword in 'keyword_tree' #68

Open sokol1412 opened 1 year ago

sokol1412 commented 1 year ago

Hi!

Firstly I need to mention that I really love work you've done. I'm now using the RobotFramework live listener which writes the data to my PostgreSQL database.

I have one issue which I'm having hard time to solve. I would like to somehow get an information on the log messages linked to a particular instance of keyword for test result.

In the current schema I do not see any connection between log_message and keyword_tree tables (I would do some join on execution_path column, but this column is missing from keyword_tree table)

Do you have any ideas how I could achieve my goal? Thanks in advance!

Muusssi commented 1 year ago

Sorry for being super slow to answer.

Keywords are stored in a way that is supposed to optimise storage while still being able to reconstruct the execution tree. This way of storage is quite efficient as repeated test executions tend to contain the same execution trees again and again.

The draw side is that there is no explicit information on the place of the keyword call in the execution tree in the tables. The keyword tree would need to be reconstructed first somehow to be able to do the matching based on the execution_path.

I have attempted to do the reconstruction using (recursive) database queries but those are unfeasibly inefficient or at least beyond my query writing skills.

Unfortunately this matching is very difficult to do in the database in single query. I might recommend looking to Epimetheus backend API: https://testmanagerfront.azurewebsites.net/data/doc#/Keyword%20trees/get_data_keyword_tree__fingerprint___ There is already a python implementation that builds the keyword trees (using multiple queries) and those can be matched to log messages. See: keyword_tree() in BaseHandler: https://github.com/salabs/Epimetheus/blob/master/backend_server/server.py#L419