☂️ New analyzer plugin system

[srawlins]

Tasks needed to deliver a new analyzer plugin system:

Tech preview

• [x] Document requirements. Public doc available
• [x] Design overall system. Public doc available
• [x] Design plugin API. Public doc available
• [x] Prototype system, to measure memory and CPU.
• [x] Enforce analyzer compatibility, by matching SDK and using a pubspec version constraint.
• [ ] Ship analysis_server_plugin package.
• [x] Relative path plugins are normalized.
• [ ] Document the API (taken from the design docs, but checked in)
• [ ] https://github.com/dart-lang/sdk/issues/59646
• [ ] LintCode, LintRule, NodeLintRegistry, and LinterContext are public API. (Best effort; these can honestly be under "General availability", but there is conceivably less churn if they are renamed and made public before GA.)

General availability

• [x] Support priority files.
• [ ] https://github.com/dart-lang/sdk/issues/59647
• [ ] https://github.com/dart-lang/sdk/issues/59644
• [ ] Differing plugin sets in analysis options within an analysis context are at least understood, tested, documented.
• [ ] Design workflow for slow plugins.
• [ ] Design workflow for plugin crash.
• [ ] Design for AOT compilation.
• [ ] Provide a testing story.
• [ ] Provide a debugging story.
• [ ] Change the protocol regarding offsets, lines, and columns.
• [ ] Diagnostics page has info about individual plugins, and pub resolution
• [ ] https://github.com/dart-lang/site-www/issues/5892

Product excellence

• [ ] Support writing Assists
• [ ] Support writing analysis rules for pubspec files and analysis options files.
• [ ] Support writing quick fixes for pubspec files and analysis options files.
• [ ] Support applying plugin-provided quick fixes with dart fix .

[matanlurey]

This is really exciting Sam! If you could use any help trying out prototypes I'd volunteer some time!

[parlough]

@srawlins Would you mind creating go links for your design docs on https://github.com/flutter/website/blob/main/firebase.json? Then you can add them to the top of your docs making them a bit easier to share and discuss publicly.

Community members often follow new go links on flutter.dev as well.

[srawlins]

I'm not good about referencing this issue in my CLs, as all the work is sort of just preparation and grooming the code. So I'll write down some of the under-the-hood changes I've been making. They are largely to support two areas: lint rule APIs and correction producer (mostly quick fix) APIs:

lint rule APIs:

• LinterContext:
  • Deprecate resolveNameInScope (replaced with resolveNameInScope2) (334082) and inTestDir (replaced with extension getter) (365300).
  • Remove analysisOptions, declaredVariables, canBeConst, canBeConstConstructor, inTestDir (replaced with extension getters) (365521), isEnabled (368524).
  • Rename currentUnit to definingUnit (368524).
  • Add isInLibDir, libraryElement (368524).
• LinterContextImpl: remove strictCasts field (344200).
• Mark LintDriver and DartLinter as "only used for linter tools and tests" (334645).
• Remove CamelCaseString and SourceLinter (334645).
• Mark DartLinter.lintPubspecSource as @visibleForTesting (334645).
• LinterOptions: mark enabledLints and analysisOptions final, and remove resourceProvider (334645).
• Move analyzer's top-level function lintFiles, classes ErrorWatchingSink and FileGlobFilter to linter's tool/ directory (334645).
• NodeLintRegistry: remove deprecated addOnClause (366404).
• Remove NodeLintRuleWithContext (366504), Linter, getVisitor, linterVersion (366840), LinterAnalysis.linterUnit, LinterContextUnit2 (366405).
• Simplify LibraryAnalyzer._computeLints (366222).
• Add LinterContextUnit.errorReporter (366405).
• Add LinterVisitor.afterLibrary (368524).
• Move DartLinter out of analyzer package (369562).
• Mark Group.builtin and Group.custom as visible-for-testing (369562).
• Remove all fields from Group (369782).
• Allow each LintRule multiple categories (369861).
• Require each LintRule to either override lintCode or lintCodes (372183).
• Rename LinterContextUnit -> LintRuleUnitContext, LinterContextImpl -> LinterContextWithResolvedResults, LinterContextParsedImpl -> LinterContextWithParsedResults (374861).
• Make LintCode public API (375060).
• Rename Category to LintRuleCategory (377269).
• Move LintRule category declarations to a messages.yaml file (382780).
• Convert the classes AnalysisOptionsFileConfig and PubspecYamlFileConfig each to a single method ().
• Privatize: ExpectedDiagnostic fields diagnosticMatcher, offset, length, and messageContains, ExpectedError field codes, ExpectedLint field lintName, LintRuleTest field collectionIncludedPaths, _ContextResolutionTest field sdkRoot (386580).
• Privatize writeTestPackageAnalysisOptionsFile and writeTestPackagePubspecYamlFile functions (386580).
• Simplify LintConfig and RuleConfig to not have an unnecessary impl class (388960).
• Make many functions on _LintConfig like addAsListOrString both static and private (388960).
• Make all of RuleConfig's fields final and pass them all as parameters into the constructor (388960).
• Simplify FormatterOptions code (389020).
• Remove most use cases of LintFilter and move it to the linter package (389403).
• Use StringSinks more in linter code, simplifying some tools (390665).
• Always propagate exceptions in lint rule tests (391202).
• Move LintDriver into the linter package (391321).

correction producer APIs:

• _AbstractCorrectionProducer:
  • Remove strictCasts (344220), invalidNodes, displayStringForType, mightBeImplicitConstructor, nameOfType, shouldWrapParenthesisBeforeAnd (367363), coveredNode (367761).
  • Make sealed (367761)
• CorrectionProducer:
  • Remove getLinterContext (364964).
  • Convert each of assistArguments, fixArguments, multiFixArguments to be List<String>? (364420).
  • Remove errorMessage (367363).
  • Make configure abstract (367363).
  • Add coveringNode (367761).
  • Make sealed (367761).
  • Remove configure (368760).
  • Add context as a named parameter to the constructor (368760).
• MultiCorrectionProducer: remove typeProvider, typeSystem (367363).
• MultiCorrectionProducer: add context (368760).
• ResolvedCorrectionProducer: remove coreTypeBool (367363).
• CorrectionUtils:
  • Remove patternOfBoolCondition (347781).
  • Remove findPossibleLocalVariableConflicts (358641).
  • Remove getTypeSource, targetClassElement, and targetExecutableElement (358663).
  • Remove prepareNewStatementLocation (360080).
  • Move prepareNewFieldLocation and prepareNewGetterLocation to DartFileEditBuilder, as addFieldInsertion and addGetterInsertion (360302).
  • Move addMethodInsert and addCaseClauseAtEndInsert to DartEditBuilder (360980).
  • Remove getDefaultValueCode (361132).
  • Move last constructor APIs to DartFileEditBuilder (362063).
  • Remove prepareNewClassMemberLocation (362303).
  • Remove addLibraryImports (363402).
  • Remove indentLeft, indentRight (363404).
  • Remove selectionIncludesNonWhitespaceOutsideNode (363501).
• DartFileEditBuilder:
  • Add addConstructorInsertion, a replacement for both CorrectionUtils.prepareEnumNewConstructorLocation and CorrectionUtils.prepareNewConstructorLocation (361624).
  • Rename add___Insertion APIs to insert___ (362001).
  • Add insertIntoUnitMember (362183).
  • Add a parameter to writeOverride called setSelection which controls whether selection should be set (362183).
• CorrectionProducerContext:
  • Make private: selectionOffset, selectionLength, utils, sessionHelper, unitResult, applyingBulkFixes, diagnostic, token (364420).
  • Remove selectionEnd, unit, file, session, workspace, typeProvider; these can be accessed via getters (364420).
  • Make final (367761).
  • Make non-generic (368760).
• Make private AssistProcessor.generators, multiGenerators, .assistContext, .assists (364420).
• FixProcessor:
  • Make private: FixProcessor.fixContext (364420).
  • Add registration of "ignore" producer generators (376126).
  • Registered producer generators use LintCodes over Strings (377267).
• Remove hasFix top-level function(345347), classes TransformOverride (347641), ExpressionCasePattern (347781), CorrectionUtils_InsertDesc (347780), CancelCorrectionException (348083), Baseprocessor (364360).
• Separate FixProcessor and built-in fix producer mappings into two files (345561).
• Combine DartFixContext and DartFixContextImpl and change constructor to use all named parameters (358382).
• Remove static class Flutter. Most methods are refactored as extension methods. Some unused ones are deleted. Some are inlined into their call sites (362300).
• Remove unused class TransformOverrideSet (340280).
• Use an enum to express how widely a CorrectionProducer can be applied (366740).
• Remove unused ParsedCorrectionProducer (368760).
• Re-introduce ParsedCorrectionProducer, and use in 7 producers (369561).
• Make DartFixKindPriority public, use lower-cased values (374742).
• BulkFixProcessor:
  • Privatize BulkFixProcessor.syntacticLintCodes (377267).
  • Privatize four more fields and one method (377622).
  • Mark two fields @visibleForTesting (377622).
  • Privatize 6 fields and 1 getter on IterativeBulkFixProcessor (377622).
• Separate fix generators into their own file (377563).

Plugin server API:

• Introduce the PluginServer (382480).
• Run PluginServer zoned (384140).
• Support overlay changes (384305).
• Add the plugin package generator (384681).

Analysis Options

• Rename most references, in the analyzer and analysis server, of 'plugins' to be 'legacy plugins' (384964).
• Simplify fields in AnalysisOptionsImpl (389040).
• Simplify parseConfig function to take a non-nullable parameter (389022).
• In CodeStyleOptions, privatize options field (389023) and improve doc comments (390266).
• Remove dead code in AnalysisOptionsImpl (390420).
• Remove unused LinterOptions.analysisOptions (390563).
• Remove unused DriverOptions.strongMode (390666).
• Convert the applyOptions extension method into a builder class (390660).
• Make the 'strict' modes part of the public API (391020).

I'll update this comment as I make more preparatory changes.

[rrousselGit]

:wave: Is this likely to land within 6months or so?

This seems to be actively worked on ; which is cool! But it also makes me re-evaluate how much effort I should spend working on custom_lint. If this takes a while to land >= 6month, I'll likely keep working on custom_lint in the meantime (such as supporting newer analyzer versions whenever they land). But if it's <=6 months, I'd consider stopping support for custom_lint, to use that time on things more valuable in the long term.

[srawlins]

Yes, actively working on it, and hope to send out something in the next month or so. Definitely < 6 months 😁 stay tuned.