Beyond expert level are the unfixed tasks. These tasks cannot be solved due to problems with the tasks. There are bug reports on them in the original ARC dataset repo, but they have never been fixed.
Attempting to solve a task that is broken is confusing.
Solution
Keep the original bad files as they are.
For each bad file, make a fixed file with a _v2 suffix. Should there be a need to revise the fixed file, then it's possible to increment the counter to _v3.
When encountering a file that is broken, then check if there exist a _v2 suffix or higher.
Problem
Beyond
expert
level are theunfixed
tasks. These tasks cannot be solved due to problems with the tasks. There are bug reports on them in the original ARC dataset repo, but they have never been fixed.Attempting to solve a task that is broken is confusing.
Solution
Keep the original bad files as they are.
For each bad file, make a fixed file with a
_v2
suffix. Should there be a need to revise the fixed file, then it's possible to increment the counter to_v3
.When encountering a file that is broken, then check if there exist a
_v2
suffix or higher.