cursorless-dev / cursorless

Don't let the cursor slow you down
https://www.cursorless.org/
MIT License
1.13k stars 79 forks source link

Support migrating old recorded tests using bulk facet recorder #2390

Open pokey opened 3 months ago

pokey commented 3 months ago

Once https://github.com/cursorless-dev/cursorless/pull/2383 is merged, we should simplify the process of migrating old recorded tests to scope fixture tests using the new bulk fixture recorder.

We proceed as follows:

  1. Find all recorded tests with the given language id

  2. Canonicalize and inspect the command payload. We may want to filter to tests that use a specific set of actions and particular targets. We may also only want to migrate a specific subset of scopes, eg based on if they're marked as supported for the given language

  3. For each chosen test, get initial document state

  4. Skip if we've already processed a fixture with the same document state (or if there already exists a fixture with same document state?)

  5. Generate an entry in the bulk recorder file with a candidate header for each possible facet that has the given scope. For example:

    [[python]]
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    if aaa:
        pass
    ---
    
    [branch.if]
    [branch.if.iteration]
    [branch.try]
    [branch.try.iteration]
    [branch.loop]
    try:
        pass
    except:
        pass
    ---
  6. The user will then delete the unnecessary lines and clone the block as necessary eg to test both scope and iteration scope

  7. When the user saves, we should probably delete the recorded tests that they actually ended up migrating

To discuss: which tests to migrate?

We need to figure out how to decide which recorded tests to migrate. Some ideas:

We also need to figure out if we want to drop recorded tests if their command payload doesn't match a particular shape. Eg skip if multiple scope types are mentioned, or of the command isn't one of a few expected commands

pokey commented 3 months ago

update from meet-up:

Migrate the above, and then see what's left and maybe make filter more lenient or just do them manually

fidgetingbits commented 3 months ago

Does this mean I should hold off an updating old tests for now to see how this goes? Porting the lua, bash, and nix tests are still on my list todo and was thinking recently about circling back to to it, but if it can mostly be automated that works too :D

pokey commented 3 months ago

Up to you. I've been adding new tests in new format, and some piecemeal migrations if I'm messing with the given scope, but waiting to do wholesale migrations till we have this