robots-from-jupyter / robotframework-jupyterlibrary

A Robot Framework library for testing Jupyter end-user applications and extensions
https://robotframework-jupyterlibrary.rtfd.io
BSD 3-Clause "New" or "Revised" License
22 stars 9 forks source link

Add coverage keywords #70

Open bollwyvl opened 1 year ago

bollwyvl commented 1 year ago

Elevator Pitch

Capture client, server, and kernel coverage with robot tests.

Motivation

Being able to gather fine-grained coverage of JS, python, and maybe other languages from a single test increases the value of executed robot tests.

Design Ideas

Client coverage:

Server coverage:

Kernel coverage could be gathered by copyinh a custom kernel.json, and adding some more data:

{
 "argv": [
  "$PREFIX/bin/python",
  "-m",
  "coverage", "run", "-m", // ... add this line, with more args
  "ipykernel_launcher",
  "-f",
  "{connection_file}"
 ],
 "display_name": "Python 3 (ipykernel)", // add the context name?
 "language": "python",
 "metadata": {
  "debugger": true
 }
}

Finally, some reporting keywords could be added, optionally embedding the reports inside the robot HTML if they can be generated as a single file.