exercism / javascript-analyzer

This is Exercism's automated analyzer for the JavaScript track.
GNU Affero General Public License v3.0
16 stars 16 forks source link

Add analyzer for resistor-color #13

Closed SleeplessByte closed 5 years ago

SleeplessByte commented 5 years ago

This issue is for discussion and assignment for the resistor-color core exercise in the javascript track.

🔗 implementation | mentor-notes | problem-specification


This exercise introduces the student to Arrays as well as showing that you can export multiple bindings at once. Solutions using an Object will actively be discouraged as speed is not a good reasoning here.

Optimal solution(s)

export const COLORS = [
  'black', 'brown', 'red', 'orange', 'yellow', 'green',
  'blue', 'violet', 'grey', 'white',
];

export const colorCode = color => COLORS.indexOf(color)

Approvable alternatives are:

SHOULD comment and disapprove

Cases we SHOULD comment on with this analyzer:

COULD comment (tips)

Cases we COULD comment on with this analyzer:

zeckdude commented 5 years ago

@SleeplessByte I'd like to tackle this one.

zeckdude commented 5 years ago

I'm digging through the code and found the fixtures for each exercise. Got a few questions about those:

  1. Where are the test fixtures coming from? I highly doubt those were manually added. There's 499 different approaches to solving the two-fer exercise in there. That's wild.
  2. I can see the fixtures being used in the batch-runner to display general stats, but I think they would be useful for unit tests where we can assign an expected result type for each of the fixtures and if someone changes the code at a later time, the tests should catch it automatically if one of the tests no longer passes. Thoughts?
  3. I got the batch-runner to work by running node bin/batch-runner two-fer. It outputs a table with general data about each possible result type. I would really like to use the batch-runner to get a list of each fixture in each category. So then I see exactly in which category a specific fixture is being placed. Perhaps I'm missing something, but it appears that currently the only way to verify that certain fixtures are showing for a certain result type, you need to run it manually on a specific fixture. I'm hoping there's a better way, more automated way, than that. 🙏
SleeplessByte commented 5 years ago

@zeckdude sure! Before you start writing code, please lay down the approach you'll take here. That gives us some time to merge #17 and #18.

Where are the test fixtures coming from? I highly doubt those were manually added. There's 499 different approaches to solving the two-fer exercise in there. That's wild.

These are the first 500 submission provided by Jeremy to me. I will ask Jeremy for resistor-color fixtures. If you run `node bin/batch-statistics you'll find the following metrics. Valid here means it was parseble and unique is unique solutions based on AST (no whitespace and no comments). (original source message on slack)

edit: #24 done and done.

total unique valid invalid
500 358 485 15

The statistics for the runner back then was this -- should be slightly different now:

Statistics

Status Count Comments Unique Avg Median Total
Approve (optimal) 7 0 0 411ms 414ms 0s
Approve (comment) 0 0 0 0ms 0ms 0s
Disapprove (comment) 355 16 7 413ms 414ms 14s
Refer to mentor 121 2 2 413ms 415ms 5s
Total 483 17 8 413ms 415ms 19s

I can see the fixtures being used in the batch-runner to display general stats, but I think they would be useful for unit tests where we can assign an expected result type for each of the fixtures and if someone changes the code at a later time, the tests should catch it automatically if one of the tests no longer passes. Thoughts?

https://github.com/exercism/javascript-analyzer/issues/9 and specifically https://github.com/exercism/javascript-analyzer/issues/9#issuecomment-496215168

I got the batch-runner to work by running node bin/batch-runner two-fer. It outputs a table with general data about each possible result type. I would really like to use the batch-runner to get a list of each fixture in each category. So then I see exactly in which category a specific fixture is being placed. Perhaps I'm missing something, but it appears that currently the only way to verify that certain fixtures are showing for a certain result type, you need to run it manually on a specific fixture. I'm hoping there's a better way, more automated way, than that. 🙏

I don't quite understand what you're asking for. The batch runner outputs the analysis.json files into the fixture directories which you can use. Would you like a table of "fixture number" -> "output status" -> "output comments"?

SleeplessByte commented 5 years ago

@zeckdude let me know if you have further comments. If not, I might go ahead and add a minimal solution for resistor-color analyzer which you can use as a basis.

SleeplessByte commented 5 years ago

https://github.com/exercism/javascript-analyzer/pull/26

Raw output

{
  "invalid": 0,
  "valid": 498,
  "total": 498,
  "unique": 309
}

Parsing statistics

This is the number of unique Abstract Syntax Trees after stripping commentary, location data (whitespace) and other tokens.

total unique valid invalid
498 309 498 0
SleeplessByte commented 5 years ago

Since we have an initial analyzer, I'm closing this issue.