scala / scala-jenkins-infra

A Chef cookbook that manages Scala's CI infrastructure.
https://scala-ci.typesafe.com
Apache License 2.0
14 stars 17 forks source link

GraphViz errors in release-main #190

Closed retronym closed 6 years ago

retronym commented 8 years ago

https://scala-ci.typesafe.com/job/scala-2.12.0-integrate-bootstrap/111/consoleFull

[warn] 7 warnings found
[info] Main Scala API documentation to /home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/compiler...
[info] Compiling 292 Scala sources to /home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/quick/classes/compiler...
Graphviz dot encountered an error when generating the diagram for:
scala.collection.parallel.IterableSplitter.Appended
These are usually spurious errors, but if you notice a persistent error on
a diagram, please use the -diagrams-debug flag and report a bug with the output.
Graphviz will be restarted...

Graphviz dot encountered an error when generating the diagram for:
scala.collection.parallel.mutable.ParHashSet.ParHashSetIterator
These are usually spurious errors, but if you notice a persistent error on
a diagram, please use the -diagrams-debug flag and report a bug with the output.
Graphviz will be restarted...

Graphviz dot encountered an error when generating the diagram for:
scala.collection.parallel.mutable.ParArray.ParArrayIterator
These are usually spurious errors, but if you notice a persistent error on
a diagram, please use the -diagrams-debug flag and report a bug with the output.
Graphviz will be restarted...

Graphviz dot encountered an error when generating the diagram for:
scala.collection.parallel.immutable.ParRange.ParRangeIterator
These are usually spurious errors, but if you notice a persistent error on
a diagram, please use the -diagrams-debug flag and report a bug with the output.
Graphviz will be restarted...

Graphviz dot encountered an error when generating the diagram for:
scala.collection.parallel.immutable.ParHashMap.ParHashMapIterator
These are usually spurious errors, but if you notice a persistent error on
a diagram, please use the -diagrams-debug flag and report a bug with the output.

Known problem? I assume this means we'll have degraded ScalaDoc for our releases.

szeiger commented 8 years ago

The known spurious "errors" from graphviz (which are really warnings) can be avoided with by overriding the dot command in the path with this:

#!/bin/bash
/usr/local/bin/dot "$@" 2> >( grep -v "CoreText performance note:" >&2 )

If these messages still show up with such a script in place, it's probably a real error.

retronym commented 8 years ago

Thanks! I forgot to mention that later on in the log I noticed:

**********************************************************************
Diagrams will be disabled for this run because the graphviz dot tool
has malfunctioned too many times. These scaladoc flags may help:

-diagrams-debug  Show debugging information for the diagram creation process.
-diagrams-dot-path <path>  The path to the dot executable used to generate the inheritance diagrams. Eg: /usr/bin/dot
-diagrams-dot-restart <n>  The number of times to restart a malfunctioning dot process before disabling diagrams (default: 5)
-diagrams-dot-timeout <n>  The timeout before the graphviz dot util is forcefully closed, in seconds (default: 10)

Please note that graphviz package version 2.26 or above is required.
szeiger commented 8 years ago

Let's get some details. I've kicked off a build with -diagrams-debug: https://scala-ci.typesafe.com/view/scala-2.12.0/job/scala-2.12.0-integrate-bootstrap/112/

szeiger commented 8 years ago

This looks like an actual problem with the diagrams:

**********************************************************************
The graphviz dot diagram tool has malfunctioned and will be restarted.

The following is the log of the failure:
  DOT <error console>: Warning: Unable to reclaim box space in spline routing for edge "node0" -> "node3". Something is probably seriously wrong.
  Input thread in scala.collection.parallel.IterableSplitter.Appended: Exception: java.lang.InterruptedException

  Cleanup: Last template: scala.collection.parallel.IterableSplitter.Appended
  Cleanup: Last dot input: 
    digraph G {
    compound="true";
    rankdir="TB";
    node [margin="0.08,0.01", fontsize="8.00", style="filled,rounded", height="0.0", penwidth="1", shape="rect", width="0.0", fontname="Source Code Pro"];
    edge [arrowsize="0.7", fontcolor="#aaaaaa", fontsize="9.00", color="#d4d4d4", fontname="Source Code Pro"];
    subgraph clusterAll {
    style="invis"
    subgraph clusterOutgoing {
    style="invis"
    node4 [label=<<TABLE BORDER="0" CELLBORDER="0"><TR><TD><IMG SCALE="TRUE" SRC="/home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/library/lib/class_diagram.png" /></TD><TD VALIGN="MIDDLE">CollectionsHaveToParArray[Appended[U, PI], T] </TD></TR></TABLE>>,fontcolor="#7d7d7d", URL="package$$CollectionsHaveToParArray.html#inheritance-diagram", color="#ababab", id="implicit-outgoing class|graph1_4", fillcolor="#e1e1e1", tooltip="scala.collection.parallel.IterableSplitter.Appended can be implicitly converted to CollectionsHaveToParArray[Appended[U, PI], T] by the implicit method CollectionsHaveToParArray in package scala.collection.parallel"];
    node3 [label=<<TABLE BORDER="0" CELLBORDER="0"><TR><TD><IMG SCALE="TRUE" SRC="/home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/library/lib/class_diagram.png" /></TD><TD VALIGN="MIDDLE">MonadOps[U] </TD></TR></TABLE>>,fontcolor="#7d7d7d", URL="../TraversableOnce$$MonadOps.html#inheritance-diagram", color="#ababab", id="implicit-outgoing class|graph1_3", fillcolor="#e1e1e1", tooltip="scala.collection.parallel.IterableSplitter.Appended can be implicitly converted to MonadOps[U] by the implicit method MonadOps in object scala.collection.TraversableOnce"];
    node3 -> node4 [constraint="false", style="invis", minlen="0.0"];
    }
    subgraph clusterThis {
    style="invis"
    node0 [label=<<TABLE BORDER="0" CELLBORDER="0"><TR><TD><IMG SCALE="TRUE" SRC="/home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/library/lib/class_diagram.png" /></TD><TD VALIGN="MIDDLE">Appended[U, PI] </TD></TR></TABLE>>,fontcolor="#ffffff", URL="#inheritance-diagram", color="#418565", id="this class|graph1_0", fillcolor="#418565", tooltip="scala.collection.parallel.IterableSplitter.Appended (this class)"];
    }

    node0 -> node3 [id="implicit-outgoing|0_3", tooltip="Appended[U, PI] can be implicitly converted to MonadOps[U], CollectionsHaveToParArray[Appended[U, PI], T]", constraint="false", minlen="2", ltail="clusterThis", lhead="clusterOutgoing", label="implicitly"];
    }
    node1 [label=<<TABLE BORDER="0" CELLBORDER="0"><TR><TD><IMG SCALE="TRUE" SRC="/home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/library/lib/class_diagram.png" /></TD><TD VALIGN="MIDDLE">SeqSplitter.Appended[U, PI] </TD></TR></TABLE>>,fontcolor="#ffffff", URL="SeqSplitter$Appended.html#inheritance-diagram", color="#418565", id="class|graph1_1", fillcolor="#418565", tooltip="scala.collection.parallel.SeqSplitter.Appended"];
    node2 [label=<<TABLE BORDER="0" CELLBORDER="0"><TR><TD><IMG SCALE="TRUE" SRC="/home/jenkins/workspace/scala-2.12.0-integrate-bootstrap/build/scaladoc/library/lib/trait_diagram.png" /></TD><TD VALIGN="MIDDLE">IterableSplitter[U] </TD></TR></TABLE>>,fontcolor="#ffffff", URL="IterableSplitter.html#inheritance-diagram", color="#2E6D82", id="trait|graph1_2", fillcolor="#2E6D82", tooltip="scala.collection.parallel.IterableSplitter"];
    node2 -> node0 [id="inheritance|graph1_2_0", tooltip="Appended[U, PI] is a subtype of IterableSplitter[U]", dir="back", arrowtail="empty"];
    node0 -> node1 [id="inheritance|graph1_0_1", tooltip="SeqSplitter.Appended[U, PI] is a subtype of Appended[U, PI]", dir="back", arrowtail="empty"];
    }

  Cleanup: Dot path: dot
  Cleanup: Dot exit code: 143
**********************************************************************
Graphviz will be restarted...
SethTisue commented 8 years ago

see also https://github.com/scala/scala-dev/issues/105. I had intended to look into it, but then went on leave

nafg commented 7 years ago

Is this related? http://www.graphviz.org/mantisbt/view.php?id=2472 What version of graphviz are you using?

SethTisue commented 7 years ago

What version of graphviz are you using?

https://github.com/scala/scala-jenkins-infra/blob/master/attributes/worker.rb#L80-L82 has:

    default['graphviz']['url']      = 'https://dl.dropboxusercontent.com/u/12862572/graphviz_2.28.0-1_amd64.deb'
    default['graphviz']['checksum'] = '76236edc36d5906b93f35e83f8f19a2045318852d3f826e920f189431967c081'
    default['graphviz']['version']  = '2.28.0-1'

there are some notes at https://github.com/scala/scala-jenkins-infra/blob/2453d8c106cf06e0c119a7f977487db623fa3eb9/recipes/_worker-config-debian.rb#L93 about a past failed attempt to use 2.36 or 2.38

nafg commented 7 years ago

@SethTisue FWIW I got the code from "Last dot input" to run locally on 2.38 (after removing IMGs)

adriaanm commented 7 years ago

Do you have a branch to repro? It would be easy to install 2.38 on the worker and try running it against a branch

nafg commented 7 years ago

@adriaanm I just copied that output into a dot file and edited it, that's all. It could be all the other bugs persist.

BTW has anyone thought about replacing graphviz in the build with letting d3 visualize on the fly? Or using a java library to render the image?

VladUreche commented 7 years ago

BTW has anyone thought about replacing graphviz in the build with letting d3 visualize on the fly? Or using a java library to render the image?

Yes, back when I was working with Damien on diagrams, he looked into runtime diagram libraries. But the ones he found weren't very reliable and didn't look very nice either. But that was back in 2012, so things may have changed in the meantime.

SethTisue commented 6 years ago

2.13.x eliminates graphviz (https://github.com/scala/scala/pull/6436). (and on 2.12.x, we've de-facto decided we can live with the status quo.)