Evaluate GitHub Copilot

[samreid]

GitHub Copilot sent me an invite and I got a chance to try it out this morning (June 25, 2022). It’s an AI-powered code completion engine by Microsoft. I’m very stunned at how well it is working. I don’t know how/whether we would leverage it practically, but as a novelty or proof of concept, it seems incredible. I tested it twice by running the following experiments in Mean: Share and balance. There is not currently any code that gets the cup with the most water, or determines how much water it has.

• I typed // Find the cup with the most water and it offered to autocomplete:

  
  private findCupWithMostWater(): WaterCupModel {
  let maxWater = 0;
  let maxWaterCup: WaterCupModel;
  this.iterateCups( ( cup2D, cup3D ) => {
    if ( cup3D.waterLevelProperty.value > maxWater ) {
      maxWater = cup3D.waterLevelProperty.value;
      maxWaterCup = cup3D;
    }
  } );
  return maxWaterCup!;
  }

I had to add the !.  This is not the most efficient or simple way to compute it, but it does work.

* Next I tried typing this comment `// Find the maximum amount of water in any cup` and it suggested this method, also correct:

```ts
  // Find the maximum amount of water in any cup
  private findMaxWater(): number {
    let maxWater = 0;
    this.iterateCups( ( cup2D, cup3D ) => {
      if ( cup3D.waterLevelProperty.value > maxWater ) {
        maxWater = cup3D.waterLevelProperty.value;
      }
    } );
    return maxWater;
  }

Again, not the most efficient or simple way to do things, but very stunning that it could figure out that much intent from natural language, and knows how to operate with our APIs (like getting Property.value and using our pre-existing iterator). In the comment thread, I’ll share a short video of the first example.

[samreid]

https://user-images.githubusercontent.com/679486/181131985-bebe071a-2d3e-4dc0-ae66-9423e5c852e1.mov

[samreid]

I just wrote:

// Find the cup with the most water

And it made this:

  private getMostWaterCup(): WaterCup {
    const mostWaterCup = this.getActive2DCups().reduce( ( mostWaterCup, cup ) => {
      return cup.waterLevelProperty.value > mostWaterCup.waterLevelProperty.value ? cup : mostWaterCup;
    } );
    return mostWaterCup;
  }

Then I asked it to write:

// Determine whether the water slider can be dragged

And it produced this:

  public canDragWaterSlider(): boolean {
    const mostWaterCup = this.getMostWaterCup();
    return mostWaterCup.waterLevelProperty.value < 1;
  }

[samreid]

I cannot explain how shocked I am at these outputs. In Circuit Construction Kit, I asked:

// Return true if one Vertex can be reached from another Vertex by any path

And it correctly returned:

  // Return true if one Vertex can be reached from another Vertex by any path
  canReach( vertex1: Vertex, vertex2: Vertex ): boolean {
    const queue = new Queue<Vertex>();
    queue.enqueue( vertex1 );
    while ( queue.count > 0 ) {
      const vertex = queue.dequeue();
      if ( vertex === vertex2 ) {
        return true;
      }
      const neighbors = this.getNeighboringVertices( vertex );
      for ( let i = 0; i < neighbors.length; i++ ) {
        queue.enqueue( neighbors[ i ] );
      }
    }
    return false;
  }

And when I tried to trick it by saying it could not cross Wires, it did this:

  // Return true if one Vertex can be reached from another Vertex by any path but not including Wires
  canReachWithoutWires( vertex1: Vertex, vertex2: Vertex ): boolean {
    const queue = new Queue<Vertex>();
    queue.enqueue( vertex1 );
    while ( queue.count > 0 ) {
      const vertex = queue.dequeue();
      if ( vertex === vertex2 ) {
        return true;
      }
      const neighbors = this.getNeighboringVertices( vertex );
      for ( let i = 0; i < neighbors.length; i++ ) {
        if ( !( neighbors[ i ] instanceof Wire ) ) {
          queue.enqueue( neighbors[ i ] );
        }
      }
    }
    return false;
  }

Not quite correct, but very close. I requested a short time for Thursday's status meeting to share a demo.

I also want to clarify I have not yet had an opportunity to use this in practice.

[samreid]

I shared a demo with the team today and we looked at several demos. @emily-phet mentioned Clayton Lewis has interest and expertise in this area. I don't plan any more work for this issue, closing.