[Recurring] Keep exercises up to date with problem specifications

[angelikatyborska]

This is a recurring task that has to be done from time to time. Do not reference this issue by writing resolves/fixes #XXX in your PR because that will close it.

We want to keep the track up to date with problem specifications, which means regularly checking if anything changes and needs updating.

How to do this task:

Step 1: fetch configlet

Run:

./bin/fetch_configlet

Step 2: check which exercises need updating and in which way

Run:

./bin/configlet sync 

The output will list all Elixir exercises that are out of sync in some way. Example output:

[warn] docs: instructions unsynced: affine-cipher
[warn] docs: instructions unsynced: all-your-base

[warn] metadata: unsynced: acronym
[warn] metadata: unsynced: armstrong-numbers

[warn] collatz-conjecture: missing 2 test cases
       - zero is an error (2187673d-77d6-4543-975e-66df6c50e2da)
       - negative value is an error (ec11f479-56bc-47fd-a434-bcd7a31a7a2e)

Choose one or more problems and attempt to fix it. Metadata and doc updates can be all done together for many exercises in a single PR, but please create separate PRs per exercise when changing tests.

Step 3.1: Sync all docs

Run this command to update all exercise docs (introductions):

 ./bin/configlet sync --docs --update

Press y when asked sync the above docs ([y]es/[n]o)?.

Step 3.2: Sync all metadata

Run this command to update all exercise metadata:

 ./bin/configlet sync --metadata --update

Press y when asked sync the above metadata ([y]es/[n]o)?.

Step 3.3: Update tests for an exercise

The general goal is that all Elixir practice exercises follow problem specifications, except for when it doesn't make any sense in Elixir.

Important:

• Get familiar with the problem specifications repository and the canonical-data.json format it uses for describing tests cases.
• Read CONTRIBUTING.md for the general rules of this repository.
• Read exercises/practice/<exercise>/.meta/design.md if it exists to learn about design decision that we took in the past for this exercise (e.g. to drop a required function from resistor-color). Create this file if you're making unexpected decisions yourself!
• Please create separate a PR per exercise when changing tests.

Step 3.3.1 Update tests.toml

Update tests.toml by running this command for a chosen exercise:

$ ./bin/configlet sync --exercise <exercise-slug> --tests include --update

This command assumes that we want to implement all the new tests, and that is our default approach, but it still needs verifying on a test-by-test base.

Step 3.3.2 Analyze the new test cases and update the Elixir tests files accordingly

Find all the test cases affected by the previous step in the exercise's canonical-data.json file in the problem specifications repository. Analyze what the changes actually are.

You might need to delete a test. You might need to add a new test. You might also need to replace an existing test with a new one (you'll see in tests.toml that it "reimplements" another).

If you believe a new test case shouldn't be implemented, add include = false to it in tests.toml and document why in exercises/practice/<exercise>/.meta/design.md.

When editing the Elixir tests, make sure to:

• Follow the coding style used in the existing test file.
• Use the test description from the tests.toml file as texts of describe and test blocks (unless that would invalidate the first rule).
• Keep the order of the tests in the Elixir test file the same as they are in the tests.toml file (unless that would invalidate the first rule).
• "Translate" problem specification requirements into Elixir idioms. E.g. when problem specifications say "returns an error", you probably want to use a {:ok, result} | {:error, error} return type (not raise errors!).

Step 3.3.3 Check that the example solution works with new test cases

After modifying the Elixir tests, run this command:

./bin/test_exercises.sh <exercise-slug>

If it fails, double-check if the new tests are correct. If they are, that means our example solution for this exercise is no longer valid for the new tests and needs to be updated!

When adding a completely new function that students need to implement, or when changing the return types of an existing one, make sure to also update the stub file.

Need help?

• Configlet docs
• General track docs
• Don't hesitate to ask here in case you can't find answers to your questions!

[angelikatyborska]

@neduard Now that your proposed change to rational-numbers was added to problem specifications (https://github.com/exercism/problem-specifications/pull/1914), it would be great if you followed instructions from this ticket to add it to the Elixir track 🙂

[neduard]

Thanks @angelikatyborska ! Created #1036, feel free to have a look :slightly_smiling_face:

[antoine-duchenet]

The instructions of Step 1 seem (slightly) outdated, the command to run is ./bin/fetch-configlet :bowtie: .