search

exercism / go

Exercism exercises in Go.
https://exercism.org/tracks/go
MIT License
967 stars 649 forks source link

Write generators for all exercises #605

Open petertseng opened 7 years ago

petertseng commented 7 years ago

In x-common, we have various exercises with canonical-data.json representing the recommended set of tests for that exercise. We created generators to more easily keep up with changes in x-common. Whenever x-common changes, the task is as simple as re-running the generator.

The current documentation is at https://github.com/exercism/xgo#generating-test-cases - if at any point we find that we should explain something better, we should add any necessary documentation.

Preliminaries:

This issue concerns exercises that don't yet have a generator.

For each exercise, first we should decide whether we even want a generator. An example where we don't want one is diamond, since property-based tests aren't represented in canonical-data.json. So don't be afraid to suggest that we pass on having a generator if it creates more work than it saves. After all, the point is to make maintenance of this track easy.

Or, we may decide to lazy-evaluate the generator: If our current tests already match x-common, there is no current need to have a generator since it will make no difference currently. So we may decide that we'll make a generator the next time the x-common JSON changes, and until then we'll stick with what we have.

If we decide we do want a generator for a given exercise: First, we should check whether there are any cases that our track has that the common JSON doesn't have. For each of those, we should consider the options:

petertseng commented 7 years ago

The Ruby track takes on a similar project at https://github.com/exercism/xruby/issues/396.

If, in the course of writing a generator, we find that there is significant duplicate code between generators that could be extracted into https://github.com/exercism/xgo/blob/master/gen/gen.go, we should do so, so that writing each generator is as little work as possible.

robphoenix commented 7 years ago

Please note the new exercise file structure for generators #610

robphoenix commented 7 years ago

The checklist in this issue is due to be replaced by separate issues.

Relevant comment by @petertseng from #611:

splitting #604 & #605 into separate issues

Y'all are right about the difficulty of keeping checklists up to date. We've seen it done both ways before in this repo too!

For #275 we split up into many different issues. I think this was when Blazon was new and I was very much still in that mindset.

In #414 and in #470 we had a checklist. Of these two, I was managing the checklist in #414 and not #470, but #414 is much smaller so I don't have much experience with that. I can say that when #414 was filed there were only eight exercises affected and I didn't want to set up filing the per-exercise issue for just eight of them. The recommendation is to use https://github.com/github/hub , I simply didn't have that set up (I used a modified version of Blazon for #275 ).

Which makes contributors' lives easier (Don't think only of ourselves)? It can be that the individual issues seem more tractable (harder for someone to know they need only take one item in a checklist). The disadvantage is if there's a chance of contributors seeing "Oh this project has 60 open issues they're obviously not very active" but perhaps that is not serious.

If agree on splitting them up, feel free to repurpose the #604 and #605 as the progress issue that the per-exercise issues link back to. I can't get to splitting them up until at least this weekend if not longer, so feel free to beat me to it. I'll say something if I start so that we don't duplicate work.

robphoenix commented 7 years ago

I'd like to suggest this issue is put on hold until #604 is closed. I'm worried that if this is also split into separate issues that it will create too many long running issues than is good for the repo. It also seems like a logical step to ensure our current generators are up to date before we start adding more.

tleen commented 7 years ago

@leenipper @robphoenix : May be worth throwing the experience into some documentation?

stevejb71 commented 7 years ago

I'm using a template based approach for the OCaml generator. This has proved quite successful so far (30 out of 40 OCaml exercises have auto-generated tests, and also 1 Purescript exercise).

Templates are in here: https://github.com/exercism/xocaml/tree/master/tools/test-generator/templates. The language is a bit ad-hoc, I'm sure it could be improved.

tleen commented 7 years ago

Thanks @stevejb71 looks interesting, will check it out!

tleen commented 7 years ago

803 is a temp fix to pangram till a generator is made for it.

shaleh commented 6 years ago

I played a bit this evening with writing a Python script to generate test code based on the canonical json. It works pretty well for many of the inputs. Some JSON files are just not as conforming. With a little work there this would be a reasonably solvable problem and then no humans have to mess with the cases_test.go (or in theory any of the other languages).

For example, bowling. If instead of mixing the roll and score properties in one long list the "cases" was made of two lists it would be pretty straight forward to generate the right output. It can still be done by keying off of the "property" tag even with the current JSON.