format_tool: optimization pass

alexandercampbell-wk commented 8 months ago

We can save a lot (~90%) of execution time of dart_dev format on some repos with two changes:

Not recursing into directories that are a priori excluded from the formatter's list (e.g., hidden directories like .git). Note that this means we no longer build and log a comprehensive list of excluded files. I think it's worth it to run so much faster.
Removing the collapseDirectories feature, to be replaced with batch parallel invocations. Computing the collapsible directories was some kind of superlinear deduplication. This could maybe be optimized in a way that would allow it to remain, but it seems it was intended as a stopgap solution for argv limitations in the first place.

This is technically a breaking change to the API since we no longer return a full list of excluded files and hidden directories. I have retained the collapseDirectories parameter in the name of backward compatibility for now, but it's ignored and marked @deprecated.

This moves the responsibility for handling argv limitations into FormatTool.

aviary2-wf commented 8 months ago

Security Insights

No security relevant content was detected by automated scans.

Action Items

Review PR for security impact; comment "security review required" if needed or unsure
Verify aviary.yaml coverage of security relevant code

Questions or Comments? Reach out on Slack: #support-infosec.

alexandercampbell-wk commented 8 months ago

I just pushed a commit to restore the deleted fields back onto FormatterInputs, but leave them unused and marked as @deprecated. I believe that was the only Dart API breaking change. I want to be extra careful about compat here given the dependency loops between tools.

alexandercampbell-wk commented 8 months ago

Oops, I broke CI. I'll get that addressed today.

alexandercampbell-wk commented 8 months ago

@Workiva/release-management-pp

tomconnell-wf commented 8 months ago

Rosie doesn't require QA on this repo? :sus:

Workiva / dart_dev