Support extracting strings from Dart code into .arb files for Flutter internationalisation

[DanTup]

Moving this from https://github.com/Dart-Code/Dart-Code/issues/3112 since it would need to be implemented in the server.

Localisation strings in Flutter live in .arb files (see https://flutter.dev/docs/development/accessibility-and-localization/internationalization), but often apps are built with hard-coded strings initially and then localisation added later.

There's a pub package that used to help extract things to arb files (https://pub.dev/packages/intl_translation#extracting-and-using-translated-messages) but that appears to not be compatible with the new way for localising, so a request came up to be able to extract strings directly from the code using a code action/assist:

This would likely involve the server having some knowledge about how Flutter's localisation works, and the ability to find/edit these files.

[asashour]

I guess this should be a refactoring, or can we have an assist if we decide the property name on behalf of the user?

The steps needed to implement this could be:

1. Verify that l10n.yaml exists in the root folder: I don't think we should go into initial steps of modifying pubspec.yaml for the user
2. Read template-arb-file and arb-dir properties from l10n.yaml
3. Update (or create) template-arb-file in arb-dir and write the entry specified by the user, with the value of the string in code, ensuring no duplicate properties.
4. Replace the string in the code with the property name

Another feature is to replace the code string with existing properties from the template file (show suggestions of all properties).

[DanTup]

I guess this should be a refactoring, or can we have an assist if we decide the property name on behalf of the user?

Interesting point.. For LSP we can't actually prompt the user for a name right now, so other refactors we just have to generate our own name and rely on the user renaming if it's bad. If we can't easily support renaming on this and we don't think we can generate perfect names (seems like it would be hard), we might need to have add some custom message in the LSP server/VS Code extension until there's a standard way (https://github.com/microsoft/language-server-protocol/issues/1164).

[bwilkerson]

Is there a standard naming scheme for properties?

[stevemessick]

Related: https://github.com/flutter/flutter-intellij/issues/4941

[DanTup]

Is there a standard naming scheme for properties?

Sorry, missed this notification. I don't know if there are any conventions but in the examples in these docs the names are things like:

• title
• backButtonTooltip

So perhaps the fields/named params they're used in would make good suffixes. I'm not sure how well the prefix could be generated/inferred though.

[hpoul]

Kind of related, I've made a analyser plugin to search for untranslated strings to show warnings. (And allow to annotate strings as non-translatable). https://pub.dev/packages/string_literal_finder Right now the only "quick fix" is to add a // NON-NLS marker comment. Unfortunately I didn't quite figure out the plugin API enough to make a source change to app_en.arb but I guess it would simply be a matter of adding another AnalysisErrorFixes at https://github.com/hpoul/string_literal_finder/blob/5e77a05f06667e6a63021caa6799114fb11e893e/packages/string_literal_finder/lib/src/plugin.dart#L224 Are there already other analyser code where I could see how this could be implemented?

[bwilkerson]

We don't have any examples of editing an .arb file, but there are examples of editing YAML files. See https://github.com/dart-lang/sdk/blob/main/pkg/analysis_server/lib/src/services/correction/dart/update_sdk_constraints.dart#L84. (This example is a bit unfortunate because it hasn't been updated to use addYamlFileEdit, but it still might be useful.)

[hpoul]

well.. I've now implemented a very rudimentary version in the plugin. (The key is simply the camelCased version of the string by default, and it's added as the first translation in the arb file, and no attempt of importing AppLocalizations is made yet).

But there are still many problems with the the IntelliJ/VSCode plugins imho..

e.g.: IntelliJ wouldn't show the edit in the "quick fixes" but shows it in the "Dart Analysis" tool window.. (maybe because quick fixes only show it if all modifications are part of a dart file or some other subset of file types.. 🤔 )

and when applying the change the "linked edits" don't correctly work. VSCode completely ignores them, and IntelliJ would only modify edits in the same file (ie. the two occurrences in the arb file, but not in the dart file).

My main problem is that it's pretty hard to discern bugs in my code vs. bugs in the IDE plugins.. Are there other fix suggestions which modify two files at once?

[bwilkerson]

@alexander-doroshko for IntelliJ

Are there other fix suggestions which modify two files at once?

Taking a quick look, I don't see any fixes that modify two files at once.

[DanTup]

and when applying the change the "linked edits" don't correctly work. VSCode completely ignores them

Are you able to provide some sample code (or the JSON representation of a produced edit) that I can debug this with?

It's worth noting that linked edits are only supported in the main file for LSP. We can only insert a "snippet" (used to have linked edits) in the current document being edited in VS Code. We can make edits to other files (that will be opened in the background if not already open), but no snippet functionality.

[hpoul]

It's worth noting that linked edits are only supported in the main file for LSP. We can only insert a "snippet" (used to have linked edits) in the current document being edited in VS Code. We can make edits to other files (that will be opened in the background if not already open), but no snippet functionality.

oh, I guess that's the reason then probably. I didn't realize this.. since every "linked edit" includes a path, I didn't think it must all be the same path..

then I guess my problem isn't really a problem, just not supported.. in case you are interested here is the json anyway..

(it contains a bit of experimentation.. for example useless suggestions) ```json { "fixes": [ { "error": { "severity": "WARNING", "type": "LINT", "location": { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart", "offset": 4698, "length": 11, "startLine": 118, "startColumn": 18, "endLine": 118, "endColumn": 29 }, "message": "Found string literal: Increment", "correction": "Externalize string or add nonNls() decorator method, or add // NON-NLS to end of line. (/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart) (lib/main.dart)", "code": "found_string_literal", "hasFix": true }, "fixes": [ { "priority": 10, "change": { "message": "B: Extract String (increment)", "edits": [ { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/l10n_example/example_app_en.arb", "fileStamp": 0, "edits": [ { "offset": 2, "length": 0, "replacement": " \"increment\": \"Increment\",\n \"@increment\": {},\n" } ] }, { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart", "fileStamp": 0, "edits": [ { "offset": 4698, "length": 11, "replacement": "loc.increment" } ] } ], "linkedEditGroups": [ { "positions": [ { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/l10n_example/example_app_en.arb", "offset": 5 }, { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/l10n_example/example_app_en.arb", "offset": 34 }, { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart", "offset": 4702 } ], "length": 9, "suggestions": [ { "value": "increment", "kind": "VARIABLE" }, { "value": "secondTestincrement", "kind": "VARIABLE" }, { "value": "increment", "kind": "VARIABLE" }, { "value": "secondTestincrement", "kind": "VARIABLE" }, { "value": "increment", "kind": "VARIABLE" }, { "value": "secondTestincrement", "kind": "VARIABLE" } ] } ], "selection": { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart", "offset": 4702 }, "selectionLength": 9 } }, { "priority": 1, "change": { "message": "Add // NON-NLS", "edits": [ { "file": "/Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example/lib/main.dart", "fileStamp": 0, "edits": [ { "offset": 4099, "length": 0, "replacement": " // NON-NLS" } ] } ], "linkedEditGroups": [] } } ] } ] } ```

I'm basically just using the example in my plugin repo to play around /Users/herbert/dev/string_literal_finder/packages/string_literal_flutter_example

Having linked edit support in multiple files would eliminate the problem of having to generate a semi-useful key since the user could just change it.. Is that a limitation of the analysis protocol/LSP?

[DanTup]

Is that a limitation of the analysis protocol/LSP?

It's a limitation of LSP and VS Code. LSP doesn't actually support snippets (required for linked edits) at all in Code Action edits - we use a non-standard extension spec'd by the Rust team: https://github.com/rust-lang/rust-analyzer/blob/911ff38eae53eeba7e94a6b01d44eef568b8c476/docs/dev/lsp-extensions.md#snippet-textedit (the LSP request for this is at https://github.com/microsoft/language-server-protocol/issues/592).

However this needs to be mapped back to a Snippet in VS Code, which only supports "snippet mode" in an open editor (and although this editor could theoretically be one other than that where the code action is invoked, the linked edits couldn't cross the documents, and switching focus to another file when the user invoked a code action would probably not be a good experience).

So for now, via LSP, we only support them in the document the user was editing. This could definitely be revisited if VS Code/LSP gained support for this though.

[mosuem]

Just FYI, Project Fluent also has such an extension https://github.com/macabeus/vscode-fluent.

[lukehutch]

The Flutter Intl plugin for VS Code does seem to allow the user to manually type in the key name:

https://twitter.com/localizely/status/1255175275454881793