freebsd / poudriere

Port/Package build and test system
https://github.com/freebsd/poudriere/wiki
BSD 2-Clause "Simplified" License
393 stars 162 forks source link

ports-mgmt/portgraph generated visualizations could be helpful. #1136

Open gessel opened 8 months ago

gessel commented 8 months ago

Prerequisites

What is your proposal?

Add an option to support ports-mgmt/portgraph in the web interface of poudriere, which might, say, make the Package column a clickable element to instigate generation of the graph into an accessible sub-directory and then load that file into a new tab in the browser, or, perhaps add a column in the ports lists like the log file, say "graph" that would say "generate" and, if generated, change to "show."

I like the visualization. Others might find it useful too.

What is the existing behavior, if any?

The current Dependency Graph is a fine text file, but lacks the graphic panache of portgraph, which produces very nice svg files that are fully compatible with modern browsers.

What is the motivation / use case for the change?

I would find it helpful in tracking down port dependencies in browser and for quickly validating port option changes that might add or subtract dependencies.

Did you consider any alternatives?

Loading the Dependency Graph text file into an editor like VSCodium is a usable solution for tracking down dependencies in pairwise fashion, but far slower than simply seeing them mapped out in a lovely graph.

Is this really a ports feature request?

It is a presentation layer request, somewhat superficial but pretty coolio. ## Example ![poudriere](https://github.com/freebsd/poudriere/assets/3670635/e86fbac3-77b3-4970-94f8-42a64bf28274) On the other hand, it does have a lot of dependencies, so it is not a "light" addition, but pretty cool that you can easily see how that is true. ![portgraph](https://github.com/freebsd/poudriere/assets/3670635/f5bfdff2-3291-46c5-bb2e-41d64e84e9d6) ## Additional context It is perhaps a somewhat superfluous request, but it is a nice visualization and the live-linking of nodes to freshports is pretty snazzy.
lbartoletti commented 8 months ago

As I'm involved in the poudriere activity and the author of portgraph, I allow myself to intervene in this proposal.

Firstly, thank you for the interest in this tool. Your proposal aligns with the reason why I created this tool in the first place. Originally, it was to address a "bug": why did a database package need to install Qt? This led me to discover the option that installed Qt and propose an alternative. Subsequently, it became about the interest in visualizing dependencies between ports.

I had plans to create a website akin to freshports or something similar to better visualize these dependencies, but time flies and life sometimes has other priorities. The software was left as it was for a few years (the documentation indicates py36...), but recently Kenneth Raplee did a good refactor, and I'll be releasing a new version in a few days.

All of this is to say that portgraph serves this purpose, but...

Behind it, you should see that portgraph is essentially just make -C xxx-depends-list ..., which is already done more or less by poudriere to get the list of packages to compile, in my humble opinion. The difference is that portgraph will generate... a graph in dot format for visualization.

Furthermore, portgraph is written in Python, which would require installing Python with poudriere. Personally, this isn't a problem for me as I use it regularly, but I understand that it might not be feasible here. I would even say it should be rejected. I've considered rewriting it in shell multiple times to avoid this Python dependency.

In a sense, I would say keep the idea - but perhaps more towards FreshPorts - but it would need to be rewritten in shell.

gessel commented 8 months ago

Dude, awesome bit of code in a lot of ways. I'm a fan!

You're absolutely right about the dependencies, though I'm not fully convinced they're a show stopper or really need to be mitigated to be useful. I can't speak for anyone else, but I find the graphic representation easy to understand and navigate than searching through the pair-wise, non-hierarchical dependency structure that the Dependency Graph export generates, though that's useful too.

I'd find it a useful addition to the already useful and nicely organized Poudriere web interface and I'd love to be able to click on a port and get the dependency graph.

The questions I most often want to answer are:

"why is "port" being built?"

"what cruft will be dragged along if I install "port"?"

To my mind, it'd be very useful to generate the graph visualization within the build jail using pkg info -r "pkg" and pkg info -d "pkg" but this command is only a single step and does not recurse through the results which is kinda the key to making the graph useful and interesting.

Looking through the code, it does some pretty cool things: detecting abandoned ports, unmaintained ports, very cool.

I converted the output of the Dependency Graph using graphviz and the whole thing, even on my little server, it's a bit much but kinda cool looking. ports dot

Very nice code and I'm glad it exists. Thanks for a useful tool!

(here's the dotformat parameters that yielded the graph, just for anyone's interest).

`digraph portgraph { layout=fdp overlap=false splines=true inputscale=0 sep=.0 maxiter=200000

zip7 -> ccache;
zip7 -> gmake;
zip7 -> libsysinfo;
zip7 -> pkg;
ImageMagick7 -> Imath;
ImageMagick7 -> cairo;

etc...`

I also had to simplify the port naming structure to simplify conversion to dotformat, so 7zip became zip7 etc.