sourcegraph / srclib

srclib is a polyglot code analysis library, built for hackability. It consists of language analysis toolchains (currently for Go and Java, with Python, JavaScript, and Ruby in beta) with a common output format, and a CLI tool for running the analysis.
https://srclib.org
Other
942 stars 62 forks source link

Incremental builds fail on following build if unit has no graph op #104

Open xizhao opened 9 years ago

xizhao commented 9 years ago

I have a toolchain that doesn't contain a graph step, but creates a new unit file type. When I try to build with old, stateful data, builds fail with the following error:

rule maker graph: no tool satisfies op "graph" for source unit type "LicenseUnit"

This is what I think is happening.

  1. Srclib tries to copy over old build data and copies over LicenseUnit files
  2. Srclib tries to run graphing LicenseUnit files
  3. My toolchain has no graph step for LicenseUnits so it fails

I see a few solutions:

  1. Make this error non-fatal (build still execute cleanly even if the graph step is missing)
  2. Configure srclib so we can skip certain steps (i.e. graph)

This would be true if I removed the toolchain as well, so I would imagine this bug would manifest if toolchains were removed. Therefore I would also suggest not copying over files that don't match a tc unit type.

samertm commented 9 years ago

Hmm, that doesn't sound right. You'll get this error when a unit type ("LicenseUnit") isn't associated with an op ("graph") in the Srclibtoolchain file in the root directory of the toolchain.

For instance, this is the contents of Srclibtoolchain for srclib-go:

{
  "Tools": [
    {
      "Subcmd": "scan",
      "Op": "scan",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    },
    {
      "Subcmd": "graph",
      "Op": "graph",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    },
    {
      "Subcmd": "depresolve",
      "Op": "depresolve",
      "SourceUnitTypes": [
        "GoPackage"
      ]
    }
  ]
}

Does it build correctly when caching is disabled?

jelmerdereus commented 9 years ago

@xizhao can you give us an update? Can it be replicated?