search

xvzcf / tls-interop-runner

Interoperability testing of TLS implementations.
Other
10 stars 13 forks source link

Methods for debugging when tests fail? #33

Open dmcardle opened 3 years ago

dmcardle commented 3 years ago

Currently, each run_endpoint.sh script clones the impl's source inside the container. If we fetched sources on the host and shared source dirs with the containers, we'd at least be able to insert printfs.

I'm no Docker expert, so do you folks have a better workflow? Any hope of using GDB or RR debugger?

xvzcf commented 3 years ago

I'm opposed to cloning the sources outside the container, and off the top of my head can't come up with anything better than adding debugging tools to the Dockerfile and entering running containers (also, I think this is fine locally, but probably should not be present in the Dockerfiles that get committed to main).

I'll think more about what can be done to make this easier.

dmcardle commented 3 years ago

Sounds good to me. Thanks for considering the question :)

xvzcf commented 3 years ago

This idea might be a bit out there; I don't have the time to properly investigate it at the moment, but I thought I'd note it down here in case someone else is interested in looking into it.

We could wrap the endpoints in Nix packages/derivations and distribute the corresponding files. A user would then just have to install Nix, which would handle the dependencies and assemble a reproducible build environment for each endpoint that is also isolated from the rest of the system: if, for example, one endpoint needs Perl 5 to build, and another needs Perl 6, Nix would handle that without impacting the broader system (or the other endpoints). Nix profiles could be used to further aid debugging.

These same Nix files would also be used to build the Docker containers, perhaps with the Docker nixos/nix image. Isolated and reproducible builds for the endpoints would be cool to have in general.

One caveat is that Nix has a steep learning curve (I'm not all that familiar with it myself).