Support custom lint rules

[trentgrover-wf]

I'd like to be able to specify custom lint rules outside of this package. This is mostly to help enforce consistency in consumption patterns of some of our private libraries that don't apply well to the larger Dart community, but would be very beneficial to our developers.

To give 1 specific example, we've implemented a Disposable interface / mixin to assist with cleaning up streams and other data structures that won't necessarily be garbage collected without some manual intervention (https://github.com/Workiva/w_common). I'd like to be able to perform some linting similar to the existing cancel_subscriptions and close_sinks rules to verify that any Disposable object instantiated is actually disposed.

[pq]

I am going to make the Flutter project more international.

A rule to help here may be general enough to merit an implementation in the linter proper. Consider opening a feature request so we can discuss?

[bwilkerson]

Do you have an example for Option 2?

The only example I'm aware of is the plugin for the built_value package (https://github.com/google/built_value.dart/tree/master/built_value_analyzer_plugin).

[SixSheeppp]

I am going to make the Flutter project more international.

A rule to help here may be general enough to merit an implementation in the linter proper. Consider opening a feature request so we can discuss?

I am going to make the Flutter project more international.

A rule to help here may be general enough to merit an implementation in the linter proper. Consider opening a feature request so we can discuss?

I create a line request. Link: #2376

[Levi-Lesches]

For what it's worth Dart Code Metrics seems like it can help. Their CONTRIBUTING.MD file explains how to add a new rule, and you can also open an issue on their repo.

[bwilkerson]

That's approximately the same amount of work that's required to add a lint to the linter package.

[fzyzcjy]

any updates?

[hpoul]

fwiw, using analyzer_plugin doesn't seem to be that complicated. I was able to hack something working together in a few hours. (borrowed a few things from dart-code-checker and cool_linter.

but it would then be integrated with both IDEs and the command-line analyzer (dart analyze).

although this is unfortunately not correct, as plugins do not work with the CLI right now:

• https://github.com/dart-lang/sdk/issues/38407
• https://github.com/flutter/flutter/issues/28327#issuecomment-532875896

[pro100svitlo]

~4 years but still no result ...

that's quite sad 😢

[btrautmann]

fwiw, using analyzer_plugin doesn't seem to be that complicated. I was able to hack something working together in a few hours. (borrowed a few things from dart-code-checker and cool_linter.

@hpoul your package was helpful -- I was able to almost get our own analyzer plugin stood up, but the IDE highlighting would show up and then randomly disappear. This wasn't the case when messing around with string_literal_finder, so I don't know if there is something about referencing the plugin from source versus pub.dev that would cause that, but I couldn't identify anything else that would've been the culprit. I even verified via the instrumentation logs that the IDE was receiving my analysis error params:

1635363849542:PluginNoti:{"event"::"analysis.errors","params"::{"file"::"/Users/brandontrautmann/src/.../example.dart","errors"::[{"severity"::"WARNING","type"::"LINT","location"::{"file"::"/Users/brandontrautmann/src/.../example.dart","offset"::0,"length"::40,"startLine"::1,"startColumn"::1,"endLine"::1,"endColumn"::41},"message"::"Mocktail import","code"::"no_mocktail_imports"}]}}:

On the CLI front, we were able to use most of linters internal code to get something working so that we can add our own custom rules and invoke our package on CI. The only downside is that developers won't know about their violations until CI runs on their PR 😢

Edit: You can see here how after an issue shows up successfully, removing and replacing the issue does not cause it to be highlighted again...

https://user-images.githubusercontent.com/8343465/139140161-4ef800c6-fb25-42b3-88d7-b5368c0004b6.mov

[DanTup]

I was able to almost get our own analyzer plugin stood up, but the IDE highlighting would show up and then randomly disappear

Are you able to make a small repro you can share that triggers this? (it would make sense to file it on an issue at https://github.com/dart-lang/sdk)

[btrautmann]

Are you able to make a small repro you can share that triggers this? (it would make sense to file it on an issue at https://github.com/dart-lang/sdk)

Yeah, I was thinking the same. I'll dig into doing that tomorrow. Shouldn't be too hard at all!

[btrautmann]

@DanTup I was able to put together a small reproduction project using string_literal_finder and an example dart package. You can find it here. Clone that, and once pub geting you should be able to reproduce something similar to this (note that switching between files causes analysis to be flaky):

https://user-images.githubusercontent.com/8343465/139286169-9959aa7c-76c1-4dc4-a649-ab2c2dedf45a.mov

Here's the diagnostics Plugins page, if it helps:

One additional thing that I noticed, is that when the analysis server creates a directory for the analyzer_plugin, it copies over it's pubspec.yaml but relative paths not made absolute, i.e a bootstrap package that relies on the host package like this:

dependencies:
  better_lint:
    path: ../../

Will not be able to resolve things packages that exist in the host package because that package is not copied over to the temp directory as well. Am I off base/misunderstanding something here? In the case of string_literal_finder its pubspec looks like this:

dependencies:
  string_literal_finder: ^1.0.0-dev.1

If I'm correct, does this mean that you cannot use relative paths when referencing the host package from the bootstrap backage?

PS I didn't want to create an issue quite yet in case you weren't able to reproduce, but let me know if you are and I'd be happy to open one with this information.

[DanTup]

@btrautmann I don't seem to be able to get the plugin working at all. If I enable the analyzer instrumentation log I see this:

1635436489047:PluginErr:RequestErrorCode.PLUGIN_ERROR:IsolateSpawnException:: Unable to spawn isolate:: Users/danny/.pub-cache/hosted/pub.dartlang.org/analyzer_plugin-0.7.0/lib/protocol/protocol_common.dart::71::12:: Error:: The getter 'JenkinsSmiHash' isn't defined for the class 'AddContentOverlay'.
 - 'AddContentOverlay' is from 'package::analyzer_plugin/protocol/protocol_common.dart' ('Users/danny/.pub-cache/hosted/pub.dartlang.org/analyzer_plugin-0.7.0/lib/protocol/protocol_common.dart').
Try correcting the name to the name of an existing getter, or defining a getter or field named 'JenkinsSmiHash'.
    hash = JenkinsSmiHash.combine(hash, 704418402);
           ^^^^^^^^^^^^^^

I believe JenkinsSmiHash has been removed now, so I think maybe there's a mismatch of versions. However I removed all versions from pubspec (and even tried explicitly setting analyzer_plugin 0.8.0) but the error persists (and always has analyer_plugin-0.7.0). I'm curious if you see the same if you enable that log (although I wonder how you'd get any diagnostics from the plugin at all if you do).

It's probably worth moving this to an SDK issue to avoid adding lots of noise here. If it turns out to not be an issue, it can always be closed (and if it's something you've done, having a recorded issue with the solution may be useful for others searching there too).

does this mean that you cannot use relative paths when referencing the host package from the bootstrap backage?

I'm not certain if this is intended or just hasn't come up before. @bwilkerson may be better placed to answer that.

[btrautmann]

I don't seem to be able to get the plugin working at all. If I enable the analyzer instrumentation log I see this:

Ah, interesting -- I was running into this last night in my other project trying to get things working and after I ran a pub upgrade I started seeing this issue. I'll move this over to an sdk issue and we can investigate further there.

[btrautmann]

I don't seem to be able to get the plugin working at all. If I enable the analyzer instrumentation log I see this:

Ah, interesting -- I was running into this last night in my other project trying to get things working and after I ran a pub upgrade I started seeing this issue. I'll move this over to an sdk issue and we can investigate further there.

Update: I do believe, as mentioned in https://github.com/dart-lang/sdk/issues/47567#issuecomment-1125086186, that flakiness was caused by using a relative path within the bootstrap package's pubspec.

As far as this issue is concerned, our team was able to write our own analyzer plugin usable via CI and the IDE based on dart-code-metrics. Obviously this is a heavy-handed approach for every team using dart/Flutter to do. Our team is interested in possibly working on a way to make this easier for the community (dart code metrics has said they don't have interest in supporting custom lint rules), but we're not committed to it yet.

[incendial]

As far as this issue is concerned, our team was able to write our own analyzer plugin usable via CI and the IDE based on dart-code-metrics. Obviously this is a heavy-handed approach for every team using dart/Flutter to do. Our team is interested in possibly working on a way to make this easier for the community (dart code metrics has said they don't have interest in supporting custom lint rules), but we're not committed to it yet.

Dart Code Metrics here. Just to make it clear: out point is that a proper direction would be to have an easier way to create plugins rather than creating some additional package with api that depends on the plugins api. This approach is very fragile. Also, consider that if someone adds a linter rule to linter package or DCM, for instance, there will be no need for others to implement the same rule from scratch. Looks like a huge benefit for the community to me.

dart code metrics has said they don't have interest in supporting custom lint rules

And to point it out clearly: DCM is a set of custom lint rules (among other features) and I'm really curious, what stops you from suggesting a rule and contributing instead of creating a whole new package?

[FMorschel]

And to point it out clearly: DCM is a set of custom lint rules (among other features) and I'm really curious, what stops you from suggesting a rule and contributing instead of creating a whole new package?

For me, the point of being able to create a package would be to develop project-specific rules. For example naming conventions, where the linter would aways tell me or someone in my team not to create Managers, but to create Controllers instead

[incendial]

And to point it out clearly: DCM is a set of custom lint rules (among other features) and I'm really curious, what stops you from suggesting a rule and contributing instead of creating a whole new package?

For me, the point of being able to create a package would be to develop project-specific rules. For example naming conventions, where the linter would aways tell me or someone in my team not to create Managers, but to create Controllers instead

Event though I got the idea, looks like your case can be covered with a generic rule like https://dartcodemetrics.dev/docs/rules/common/ban-name. Or am I missing something?

[FMorschel]

That's basically what I was looking for, thank you!

And to point it out clearly: DCM is a set of custom lint rules (among other features) and I'm really curious, what stops you from suggesting a rule and contributing instead of creating a whole new package?

For me, the point of being able to create a package would be to develop project-specific rules. For example naming conventions, where the linter would aways tell me or someone in my team not to create Managers, but to create Controllers instead

Event though I got the idea, looks like your case can be covered with a generic rule like https://dartcodemetrics.dev/docs/rules/common/ban-name. Or am I missing something?

That's basically what I was looking for, thank you!

[btrautmann]

And to point it out clearly: DCM is a set of custom lint rules (among other features) and I'm really curious, what stops you from suggesting a rule and contributing instead of creating a whole new package?

Yeah, along the same lines as what @FMorschel is saying above, we have project-specific rules based on internal APIs that we'd want to enforce, e.g use MyNewApi and not MyOldApi (and we'd want to do this based on the actual AST nodes themselves, not just naming) -- we even have a rule prohibiting the usage of "smart" (curly) apostrophes.

I definitely apologize if I made it sound like DCM was "unwilling" or "rudely against" a more abstract solution. Definitely not my intention. Your project is awesome and helped us HUGELY with our own implementation. I totally understand a desire not to build out a less-than-ideal solution like the one proposed above. I do think, though, that the "easier analyzer plugin" approach will probably be the only viable one for the foreseeable future. Out of curiosity, assuming you shipped as part of that solution a set of stable APIs that consumers could utilize, what are the exact concerns around fragility? Asking because it could inform (or deter us from) our approach.

[incendial]

I definitely apologize if I made it sound like DCM was "unwilling" or "rudely against" a more abstract solution. Definitely not my intention.

Oh, no worries, I just tried to give an additional context for others.

Your project is awesome and helped us HUGELY with our own implementation.

Thank you!

I totally understand a desire not to build out a less-than-ideal solution like the one proposed above. I do think, though, that the "easier analyzer plugin" approach will probably be the only viable one for the foreseeable future. Out of curiosity, assuming you shipped as part of that solution a set of stable APIs that consumers could utilize, what are the exact concerns around fragility? Asking because it could inform (or deter us from) our approach.

Unfortunately, current plugins API is not that mature and sometimes has unexpected problems (mostly performance, but in early days we also copied some hacky code from Dart Angular plugin just to make ours work). I expect this kind of problems will resolve with time, especially if there will be more plugins (big thanks to the Dart team, they're really investing in making plugins great), but here are few examples of issue I'm talking about: https://github.com/dart-lang/sdk/issues/47851, https://github.com/Dart-Code/Dart-Code/issues/3679, https://github.com/dart-lang/sdk/issues/48682. If anything like that breaks, you might get your users become really unsatisfied with the stability of your package.

On the other hand there are some limitation from the API, like: no way to add custom dart fix contributions, no way to run plugin analysis on dart analyze, etc.

I think that's why investing in capabilities, stability, documentation / examples, and maybe some additional linting API for plugins might be a better approach, IMO.

If nothing of that stops you, take a look at this repo https://github.com/hisaichi5518/dart-linting. This might give you some ideas too.

[bwilkerson]

... no way to add custom dart fix contributions ...

That is, unfortunately, true. It wouldn't be too hard, though, to add a protocol to support that case.

... no way to run plugin analysis on dart analyze ...

dart analyze is built on the analysis server, so it ought to be loading and running plugins. If that's not the case, then that's a bug. (As a reminder to myself: it's possible that dart analyze isn't waiting to get results from plugins before printing the results from server.)

[nilsreichardt]

Invertase published a package to make your custom lint rules: https://invertase.io/blog/announcing-dart-custom-lint 🎉

cc: @rrousselGit

[btrautmann]

Invertase published a package to make your custom lint rules: https://invertase.io/blog/announcing-dart-custom-lint 🎉

cc: @rrousselGit

If I'm understanding how things work under the hood (I haven't looked at the source, but only the announcement blog post, so I could be wrong), this is very similar to what our team was thinking of doing, and probably has the risks that @incendial was warning about regarding building against an unstable API. That said, I think it's a great step in the right direction for the community at large and is the only reasonable next step at this time.

Our team ended up basically doing a pared own fork of dart code metrics, added the ability to baseline, and added the ability to parse/lint YAML files, which I don't think would be possible with Invertase's solution (at least out of the box). So for folks like us that want a more advanced feature set, it probably doesn't make sense to migrate, but for 99% of the community I think this is awesome!

[KKimj]

Invertase published a package to make your custom lint rules: https://invertase.io/blog/announcing-dart-custom-lint 🎉

cc: @rrousselGit

Thanks! What a terrific news! I searched these custom linting functionalites for at least a year.

** Acknowledge invertase

[incendial]

and added the ability to parse/lint YAML files

@brianquinlan this sounds pretty lit 🔥 , we have the same in our roadmap

[rrousselGit]

added the ability to parse/lint YAML files, which I don't think would be possible with Invertase's solution

I don't see why you couldn't. custom_lint does it. When plugins are starting or fail to start, it underlines the plugin name in the project's pubspec.yaml

[btrautmann]

added the ability to parse/lint YAML files, which I don't think would be possible with Invertase's solution

I don't see why you couldn't. custom_lint does it. When plugins are starting or fail to start, it underlines the plugin name in the project's pubspec.yaml

Totally, that's why I suffixed that statement with "(at least out of the box)" 😊

[bartekpacia]

Would love to see some progress on this. It'd be really great if custom lint rules created with custom_lint would be run with dart analyze (currently it's not possible).

[srawlins]

Please follow https://github.com/dart-lang/sdk/issues/53402 for updates on the upcoming analyzer plugin system, which will provide the ability to write custom lint rules.