Shopify / ruby-lsp

An opinionated language server for Ruby
https://shopify.github.io/ruby-lsp/
MIT License
1.36k stars 125 forks source link

Treat test name strings as document symbols #519

Closed andyw8 closed 1 year ago

andyw8 commented 1 year ago

Motivation

When tests are defined as normal methods using the def test_something syntax, they are treated as document symbols, as any other method would be. But when using the test "some string" syntax provided by ActiveSupport, they are not detected. By adding support for those, VS Code can show them in the Outline View, and list them you press @ in the command palette.

Screenshot 2023-02-26 at 9 46 30 AM

This uses some of the earlier exploratory work by @DeeChau in https://github.com/Shopify/ruby-lsp/pull/490

Implementation

Currently this does not support other syntax such as RSpec, but I hope to add that in future.

Automated Tests

I've added a new fixture and expectation.

Manual Tests

Choose a large project which uses ActiveSupport tests, e.g. a Rails project. Ensure that each tests is listed in the Outline view.

github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.14501 std_dev: 0.037029
          textDocument/diagnostic average: 0.05212 std_dev: 0.003625
      textDocument/selectionRange average: 0.004933 std_dev: 0.000883
        textDocument/foldingRange average: 0.002045 std_dev: 0.000333
   textDocument/documentHighlight average: 0.001859 std_dev: 0.000327
 textDocument/semanticTokens/full average: 0.001025 std_dev: 0.000308
textDocument/semanticTokens/range average: 0.000765 std_dev: 0.000144
               textDocument/hover average: 0.00072 std_dev: 0.000269
        textDocument/documentLink average: 0.000653 std_dev: 0.000151
           textDocument/inlayHint average: 0.000457 std_dev: 0.000121
      textDocument/documentSymbol average: 0.000318 std_dev: 0.000101
    textDocument/onTypeFormatting average: 0.000135 std_dev: 0.000105
          textDocument/codeAction average: 4.1e-05 std_dev: 8.8e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
andyw8 commented 1 year ago

Thanks @paracycle, I've updated with your suggestion.

My thinking behind using @test_class was that we could set it then skip later checks for node.message.value == "test" in non-test classes, but I didn't measure the difference and I expect it's negligible.

Start hooking into the runtime and collect the names of all the tests that the application has loaded from this file. I think this will be a bigger thing to pull off, but the resulting knowledge is exactly what we are after, including the actual names of the tests as they appear at runtime. This will also include any tests that might have been defined dynamically, which the other method will miss.

This is interesting, and I'll keep in mind for future enhancements.

github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.17545 std_dev: 0.039022
          textDocument/diagnostic average: 0.064914 std_dev: 0.007363
      textDocument/selectionRange average: 0.005325 std_dev: 0.00116
        textDocument/foldingRange average: 0.002136 std_dev: 0.00035
   textDocument/documentHighlight average: 0.002114 std_dev: 0.000959
 textDocument/semanticTokens/full average: 0.001073 std_dev: 0.000396
textDocument/semanticTokens/range average: 0.000813 std_dev: 0.000199
        textDocument/documentLink average: 0.000782 std_dev: 0.000356
               textDocument/hover average: 0.000768 std_dev: 0.000363
           textDocument/inlayHint average: 0.000549 std_dev: 0.000248
      textDocument/documentSymbol average: 0.000337 std_dev: 0.00016
    textDocument/onTypeFormatting average: 0.000161 std_dev: 0.000293
          textDocument/codeAction average: 4.6e-05 std_dev: 7.3e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.111102 std_dev: 0.029698
          textDocument/diagnostic average: 0.040567 std_dev: 0.003925
      textDocument/selectionRange average: 0.003531 std_dev: 0.00078
   textDocument/documentHighlight average: 0.001438 std_dev: 0.000408
        textDocument/foldingRange average: 0.001435 std_dev: 8.4e-05
 textDocument/semanticTokens/full average: 0.000794 std_dev: 0.000164
textDocument/semanticTokens/range average: 0.000593 std_dev: 1.9e-05
               textDocument/hover average: 0.000545 std_dev: 0.000284
        textDocument/documentLink average: 0.000517 std_dev: 0.000203
           textDocument/inlayHint average: 0.000404 std_dev: 9.0e-05
      textDocument/documentSymbol average: 0.000273 std_dev: 0.002186
    textDocument/onTypeFormatting average: 0.000111 std_dev: 0.000221
          textDocument/codeAction average: 2.7e-05 std_dev: 2.9e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.109369 std_dev: 0.026449
          textDocument/diagnostic average: 0.039843 std_dev: 0.003488
      textDocument/selectionRange average: 0.003488 std_dev: 0.000746
   textDocument/documentHighlight average: 0.001428 std_dev: 0.000262
        textDocument/foldingRange average: 0.001425 std_dev: 7.5e-05
 textDocument/semanticTokens/full average: 0.000789 std_dev: 0.000164
textDocument/semanticTokens/range average: 0.000589 std_dev: 1.7e-05
               textDocument/hover average: 0.000542 std_dev: 0.000232
        textDocument/documentLink average: 0.000506 std_dev: 0.000155
           textDocument/inlayHint average: 0.000397 std_dev: 8.4e-05
      textDocument/documentSymbol average: 0.000202 std_dev: 5.3e-05
    textDocument/onTypeFormatting average: 0.000104 std_dev: 0.000168
          textDocument/codeAction average: 2.6e-05 std_dev: 2.7e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
andyw8 commented 1 year ago

I've pushed an update which has some additional rules for what we treat as a test, and expanded the number of variations in the fixture.

I've also tried adding support for cases where the name contains interpolations. We can't catch every possible variation in the ways that people might construct test names, but this seems like a common one that is fairly easy to handle.

github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.113524 std_dev: 0.026906
          textDocument/diagnostic average: 0.041817 std_dev: 0.004341
      textDocument/selectionRange average: 0.00354 std_dev: 0.0009
   textDocument/documentHighlight average: 0.001436 std_dev: 0.000326
        textDocument/foldingRange average: 0.001396 std_dev: 7.7e-05
 textDocument/semanticTokens/full average: 0.000795 std_dev: 0.000152
textDocument/semanticTokens/range average: 0.000589 std_dev: 3.2e-05
               textDocument/hover average: 0.00053 std_dev: 0.000213
        textDocument/documentLink average: 0.000519 std_dev: 0.000508
           textDocument/inlayHint average: 0.000393 std_dev: 9.1e-05
      textDocument/documentSymbol average: 0.000204 std_dev: 1.5e-05
    textDocument/onTypeFormatting average: 0.000108 std_dev: 0.00022
          textDocument/codeAction average: 2.6e-05 std_dev: 2.7e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol faster by 11.595 %
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.123031 std_dev: 0.027473
          textDocument/diagnostic average: 0.044486 std_dev: 0.005874
      textDocument/selectionRange average: 0.00372 std_dev: 0.00105
        textDocument/foldingRange average: 0.00158 std_dev: 7.9e-05
   textDocument/documentHighlight average: 0.00157 std_dev: 0.00045
 textDocument/semanticTokens/full average: 0.000822 std_dev: 0.000161
textDocument/semanticTokens/range average: 0.00063 std_dev: 3.7e-05
               textDocument/hover average: 0.000618 std_dev: 0.000234
        textDocument/documentLink average: 0.000556 std_dev: 0.000203
           textDocument/inlayHint average: 0.000431 std_dev: 0.000106
      textDocument/documentSymbol average: 0.000226 std_dev: 6.0e-05
    textDocument/onTypeFormatting average: 0.00012 std_dev: 0.000237
          textDocument/codeAction average: 3.1e-05 std_dev: 3.2e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.110469 std_dev: 0.027266
          textDocument/diagnostic average: 0.040107 std_dev: 0.003722
      textDocument/selectionRange average: 0.003546 std_dev: 0.000877
        textDocument/foldingRange average: 0.001443 std_dev: 6.8e-05
   textDocument/documentHighlight average: 0.001442 std_dev: 0.000438
 textDocument/semanticTokens/full average: 0.000788 std_dev: 0.000154
textDocument/semanticTokens/range average: 0.000591 std_dev: 3.2e-05
               textDocument/hover average: 0.000539 std_dev: 0.000245
        textDocument/documentLink average: 0.000512 std_dev: 0.000184
           textDocument/inlayHint average: 0.000401 std_dev: 8.8e-05
      textDocument/documentSymbol average: 0.000205 std_dev: 5.6e-05
    textDocument/onTypeFormatting average: 0.000111 std_dev: 0.000226
          textDocument/codeAction average: 2.7e-05 std_dev: 2.8e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
andyw8 commented 1 year ago

Since parens are rarely used for test methods, I'm thinking of removing that part for now so that this can ship as a simpler implementation.

vinistock commented 1 year ago

Is it difficult to support test methods with parenthesis? I'd say it makes sense to do it since we're already looking at it.

github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.123454 std_dev: 0.029219
          textDocument/diagnostic average: 0.045377 std_dev: 0.005917
      textDocument/selectionRange average: 0.003784 std_dev: 0.001114
        textDocument/foldingRange average: 0.001578 std_dev: 8.0e-05
   textDocument/documentHighlight average: 0.00156 std_dev: 0.000355
 textDocument/semanticTokens/full average: 0.000846 std_dev: 0.000172
               textDocument/hover average: 0.000631 std_dev: 0.00033
textDocument/semanticTokens/range average: 0.000616 std_dev: 3.8e-05
        textDocument/documentLink average: 0.000561 std_dev: 0.000217
           textDocument/inlayHint average: 0.000436 std_dev: 0.000106
      textDocument/documentSymbol average: 0.000227 std_dev: 6.7e-05
    textDocument/onTypeFormatting average: 0.000118 std_dev: 0.000214
          textDocument/codeAction average: 3.1e-05 std_dev: 3.3e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.11709 std_dev: 0.029477
          textDocument/diagnostic average: 0.042662 std_dev: 0.004811
      textDocument/selectionRange average: 0.003689 std_dev: 0.000845
        textDocument/foldingRange average: 0.001569 std_dev: 6.8e-05
   textDocument/documentHighlight average: 0.001558 std_dev: 0.000348
 textDocument/semanticTokens/full average: 0.000838 std_dev: 0.000181
textDocument/semanticTokens/range average: 0.000609 std_dev: 4.7e-05
               textDocument/hover average: 0.000599 std_dev: 0.000217
        textDocument/documentLink average: 0.000546 std_dev: 0.000197
           textDocument/inlayHint average: 0.000417 std_dev: 8.4e-05
      textDocument/documentSymbol average: 0.000215 std_dev: 5.6e-05
    textDocument/onTypeFormatting average: 0.000118 std_dev: 0.00021
          textDocument/codeAction average: 3.1e-05 std_dev: 2.9e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
github-actions[bot] commented 1 year ago
Benchmark results in seconds (slowest at top)

          textDocument/formatting average: 0.121366 std_dev: 0.027891
          textDocument/diagnostic average: 0.043864 std_dev: 0.004993
      textDocument/selectionRange average: 0.003748 std_dev: 0.000959
        textDocument/foldingRange average: 0.001608 std_dev: 7.0e-05
   textDocument/documentHighlight average: 0.001538 std_dev: 0.000241
 textDocument/semanticTokens/full average: 0.00085 std_dev: 0.000173
textDocument/semanticTokens/range average: 0.000623 std_dev: 3.9e-05
               textDocument/hover average: 0.000598 std_dev: 0.000193
        textDocument/documentLink average: 0.000548 std_dev: 0.000164
           textDocument/inlayHint average: 0.000423 std_dev: 8.9e-05
      textDocument/documentSymbol average: 0.000226 std_dev: 6.0e-05
    textDocument/onTypeFormatting average: 0.000116 std_dev: 0.000193
          textDocument/codeAction average: 3.0e-05 std_dev: 2.8e-05

================================================================================
Comparison with main branch:

 textDocument/semanticTokens/full unchanged
textDocument/semanticTokens/range unchanged
      textDocument/documentSymbol unchanged
        textDocument/foldingRange unchanged
          textDocument/formatting unchanged
          textDocument/diagnostic unchanged
        textDocument/documentLink unchanged
           textDocument/inlayHint unchanged
      textDocument/selectionRange unchanged
   textDocument/documentHighlight unchanged
               textDocument/hover unchanged
          textDocument/codeAction unchanged
    textDocument/onTypeFormatting unchanged
andyw8 commented 1 year ago

Closing since we now have the Test Explorer behaviour.