search

bazelbuild / rules_scala

Scala rules for Bazel
Apache License 2.0
363 stars 278 forks source link

Stricter unused dependency checking for plus one deps mode #867

Open Jamie5 opened 5 years ago

Jamie5 commented 5 years ago

When using plus one deps mode, many unused deps do get marked, but some deps can be left out because they are already the dep of another dep, and it also makes sense to leave them out because they never appear in the source code of the package being compiled, and the only reason the dep is needed is to make scalac happy. Would it be possible to have a unused deps mode for plus one deps mode where a dep is marked as an unused, unless it is explicitly referenced in the source code? (Not sure if this would end up with a number of false positives - not that familiar with scalac's needs)

This is now in progress, the below summarizes the status

Known potential issues

Things to do

Jamie5 commented 4 years ago

Yes. Note even under the previous plan there would have been an off but it would not have been exposed to the user. So even though under the current plan we do expose off to the user it doesn't change anything in that the plugin will continue to not run in the direct case even if strict_deps=error/warning (and unused_deps=off)

Jamie5 commented 4 years ago

We are now using the new ast mode and strict deps with our +1 and things seem to work fine.

@anchlovi / others if you have time to test the process is now much simpler, just updating to latest rules_scala and then adjusting the toolchain as stated in README.md.

Known potential issues

Things to do

ittaiz commented 4 years ago

(@ittaiz what is this period?)

4 weeks?

(@ittaiz do we know how this will work?)

3 months from now if no one complains at all or if we do have issues streaming then a month since the last issue?

Also I like your proposal from before very much

Jamie5 commented 4 years ago

Ok those timelines sound good.

Regarding handling unused deps with +1 when strict-deps definition doesn't work: another possibility is that in the case of plus one deps we simply define the strict deps to be S + UP (S being the ones which appear in the code). The nice thing about this is that like today, every dep is either strict or unused, and every dep which appears in deps is definitely relevant. It does mean more entries in deps potentially, but this is likely quite a rare case and so may be acceptable. This would even mean that we (we not being rules_scala) would probably being fine with a flag about using this exception.

I would prefer to defer this until we get an example of this in real life, which may point to what sort of patterns in which this happens which may point out what approaches are better over others. Rather than just jumping in right now. This is with the awareness that the first examples will not necessarily be representative of all of rules_scala's customers, as it would at least be something more than the guessing of right now. And that this may delay the turning on of ast mode by default I think is reasonable.

ittaiz commented 4 years ago

+1 on deferring until real examples. I really hope people at WixEng will be able to send examples (repros are hard).

S+UP- probably a dumb question but doesn’t it effectively mean no unused deps? I think I missed your point

Jamie5 commented 4 years ago

UP is the targets which we need but which fail under the traditional strict deps definition - that is, they do not appear in the code, and also are not a plus one of anything which appears in the code. So in the example of subclassing A->B->C->D->E, then A is being built, B is a direct dep (and appears in the code, so is a strict dep), C is a +1 dep, while D and E are needed but not covered by (strict deps, and plus one deps of strict deps). Hence D and E are in UP.

Under the alternative proposal (let's call it proposal B), we would need to include both D and E in A's deps, so that the deps entries for A are [B, D, E]. Any other deps would be marked as unused.

One concrete difference is that suppose we have class F subclassing D. Under the original proposal (proposal A, let's say), then it would be valid to have [B, F, E] as the deps. The reason is that since F brings in D and D is in UP, F would not be marked as unused (but also would not be marked as strict).

However under proposal B, we would emit a strict deps error that D must be included, and also an unuse deps error that F is not used. This way the deps makes more sense, rather than F which is kind of unrelated to A being in the deps.

The downside of proposal B is that we need to have [B, D, E] and not just [B, D] even though D already brings in E. This seemed better to me than having strange dependencies that really never get used such as F not be warned about in the deps list.

ittaiz commented 4 years ago

Can we somehow have the best of both worlds? It feels close

Jamie5 commented 4 years ago

One possibility is to take proposal A but change it so that in step 6 For every target in U, report it as unused unless it either is contained in UP, or one its plus one deps is contained in UP. instead becomes For every target in U, report it as unused unless it is contained in UP.

This means that random things like F will still get reported. However the downside is that the workflow in the case of [B, F, E] will be 1. remove F 2. get a compile error 3. read the compile error and realize we have to insert D into the list. Which is no fun for anyone.

Another is to take proposal B, and change it so that the new strict deps definition includes, of UP, only those which are not covered by a different dep in UP with +1. However this analysis will be sketchy since we would only have access to the +1 deps of the things which are listed in deps so this might just be strange and I haven't thought it through to see if it even makes sense.

So hopefully the real examples will help to clarify things.

ittaiz commented 4 years ago

Thanks!

First let me reiterate my agreement about real examples, they’re crucial.

Second the first alternative which trims UP in proposal A sounds really good. Dropping F to get D is no fun but isn’t it the correct thing to do? This what we’re trying to improve in our current discussion. Maybe I’m missing something.

Jamie5 commented 4 years ago

So if we accept the fact that people will receive confusing errors then yes it is fine. Though it isn't a strict dep, I guess we can report the deps which would be orphaned by removing the unused dep so that the user can add them in in the same fell swoop.

There may well be complaints that unused deps checker isn't working, because they removed a dep and it wasn't complained about as strict and also wasn't complained about as unused originally. (This dep would have been pinned by the +1 of some other dep in UP so would be safe to remove).

Implementation wise it is a bit more complex but that seems acceptable since it'll be simple on the outside.

ittaiz commented 4 years ago

hmm, I think I understand. WDYT about writing a summary of the current proposal with its pros and cons? This would help others coming into this issue to not have to read our whole thought stream.

Jamie5 commented 4 years ago

Summary of plus one/unused deps issue and potential solutions.

Note: historical discussion on https://github.com/bazelbuild/rules_scala/pull/991#issuecomment-584311708 and on this thread as well.

Problem

+1 deps combined with strict deps is just an approximation of what deps the scalac compiler actually needs and does not always work.

For example assuming we have the inheritance chain A -> B -> C -> D, each in its own bazel package. For A, its only strict dep is B. B's +1 brings in C. But nothing brings in D as a dep, and scalac does require all of A, B, C, D to compile.

As of right now, if C or D was included in A's deps, it would be reported as an unused dependency.

Note that this problem is limited to +1 mode and does not apply to transitive mode.

Existing mitigations include properly exporting iface deps in the cases in which this happens, and using unused_dependency_checker_ignored_targets. At least one team is fine with those mitigations, and so far have not needed them yet, suggesting the problem is sometimes quite rare.

The plan is to wait for examples in the wild where this is a problem to get more information about what sort of situations this occurs in practice.

Solutions

There are a few potential solutions, which share the initial steps

  1. Compute the set of required targets for compilation, R (will need to figure out how to compute this, but can just improve it slowly)
  2. Compute the strict deps, S (which uses the same computation as today)
  3. Compute the targets that S reaches (i.e. itself and its plus one deps) as P
  4. Let the set of unprovided targets be UP = R - P
  5. Let targets we would report as unused based on the existing algorithm be U

In order to do step 3, we will need to pass the plus one deps of each dep to the plugin. This information already exists in bazel, it just gets merged into a flat list of direct and indirect dependencies today before reaching the plugin.

A note about the computation of R - it need not be perfect. We only need that R + P together include all the deps that are actually needed to compile.

Another note: There may be a desire, depending on the solution, to have this augmented behavior be opt-in.

Solution A

For every target in U, report it as an unused dep unless it appears in UP.

When we report an unused dep, we need to check if it "pins" any deps which are in UP, and if so emit errors to add those deps manually. Otherwise the user will get compile errors when they blindly follow the buildozer command.

Solution B

We keep the overall framework of code the same, but we use S' = S + UP, S' being the set of strict deps we use. So in the common steps above, all of this logic exists in UP. So the logic is still

  1. compute strict deps, S'
  2. report strict deps and unused deps based on S'

Advantages and Disadvantages

One True Deps List

Under solution B, for any state of code, there is only one valid set of deps, and any other set of deps will have either unused deps warnings or compile errors. There aren't situations where there is an dep which you can remove without an error - either that compilation fails, or that a strict dep is missing.

No one will complain that there are false negatives in the unused deps checker, based on the fact that even when they delete the dep, things still compile.

On the other hand, in solution A, this is not the case. For example, in a subclassing chain A -> B -> C -> D -> E, we can have A's deps be [B, D, E] and this will not throw an error. However, removing E will not cause any compilation errors or unused deps errors. While under solution B a strict deps error will be reported for E.

Less deps

Under solution A, we can elide some deps due to the plus one rule. For example in a subclassing chain A -> B -> C -> D -> E, for A the deps [B, D] is enough under solution A while solution B insists we include E in the deps list.

Implementation Complexity

Without thinking too much about it, right now A feels somewhat more complex to implement than B, and both need a decent amount of work to bring in the +1 deps of each direct dep to the plugin.

Computing the true required deps

This is a non-exhaustive list of things that we should do to find true required deps.

Jamie5 commented 4 years ago

@ittaiz From the first post

Once the new AST mode is sufficiently validated [definition is 1 month since no more true errors [the last known one requiring a fix being https://github.com//pull/1008 ], 3 months after Feb 17 [which would be May 17], and also @ittaiz approves] do the following steps sequentially, with enough time in between them to allow people to migrate

Is this accurate, in that May 17 would be the day we could switch over, assuming no further reports come in? AFAIK there are a few potential issues such as the subclassing thing, but no real world examples.

Should we actually only switch the default to be ast for transitive/plus-one, while still giving the user the option to toggle? Which might force more people onto ast by accident and hence maybe generate more bug reports and various issues? If so, should we do that now and then reset the 1 month/3 month clock, and then remove dependency_tracking_method after that waiting period has ended?

ittaiz commented 4 years ago

Sounds good re the extra caution of switching the default. AFAIK AST mode is not working for Wix but @or-shachar and his team need to do the detailed work of repros and such.

Jamie5 commented 4 years ago

Ok. In the near future I will switch the default, and then wait 3 months/1 months thing before removing the default. For any code which is public and reasonably easy to compile I can also help look directly.

ittaiz commented 4 years ago

To make sure- we’re aligned that at any point people will be able to use +1/direct without strict deps or unused deps, yes?

Jamie5 commented 4 years ago

Yes - this removal of options is only for ast vs high-level. (and if someone turns off both strict_deps and unused_deps, then dependency_analyzer wouldn't run anyways)

liucijus commented 4 years ago

A few repros of non-trivial situations:

liucijus commented 4 years ago

One more:

Despite my annoying repros, this features seems to be working very well! Thanks for all this fantastic work, @Jamie5

liucijus commented 4 years ago

Example of how strict deps can create unnecessary coupling on the user side: #1052

Probably this means that there's a lot of value in solutions mentioned https://github.com/bazelbuild/rules_scala/issues/867#issuecomment-588516930

Jamie5 commented 4 years ago

@ittaiz we have passed the time gate that Remove dependency_tracking_method on toolchain and ast becomes the method for transitive/+1, while high-level becomes the method for direct is now open. Do we feel this is a reasonable step to take now? I know there are a few open issues but none of them were significant enough to require fixing at this time from what I could tell.