Question: Does black have a special role in generated tasks?

[drawlinson]

Looking at the original tasks and those generated by re-arc, it seems that in ARC the background colour is usually black and that there's an implicit quality of black that it represents empty space. This doesn't seem to be the case in re-arc. Is there an easy way to create that constraint?

[michaelhodel]

The idea was to have no bias against or for certain symbols, unless of course a given task logic demands it, so: no, at least not in a straight-forward manner at generation time.

Oftentimes, the background symbol is sampled from a set of eligible colors. I typically used the variable name bgc for storing the symbol that corresponds to the background color. bgc = choice( occurs 205 times in generators.py. Hence, modifying all those to bgc = 0, where 0 is mapped to black, would probably work for a majority of those tasks, but does not seem like a robust / exhaustive solution.

What I would suggest instead is some post-generation trickery. What may work and would be efficient in terms of generating not too many throw-away examples would be to do a color permutation s.t. the most common (or whichever other notion defining "background color" suitable for your use case) color maps to black and then re-running the verifier on that example and only keeping it if the verifier still works. But that is probably also not super robust in and of itself.

I think the easiest way to achieve having only examples with a black background would be to simply filter out all generated examples which do not have a black background color. Note however that this may not work for all tasks, since a task may have a fixed other usage for black which happens to therefore never be the background color (or a task may have a fixed other color for the background, for whatever reason), in which case you'd end up with an empty set of examples. Thus you'd have to reject some tasks (ideally as soon as you - based on the generated examples - find the probability of black possibly being a background color low enough, if you care about being efficient).

If you care about exhaustiveness of tasks and don't care too much about retaining the exact logic of a given task, you could add color permutation to the rejection sampling idea: Whenever you see an empty set of examples for a given task after filtering for black background color, pick another background color for which the set is non-empty and simply "redefine" the task s.t. you apply the same color permutation with the picked color mapping to black on all examples in your set of filtered examples. Note that doing the same color permutation on all examples of a given task by definition (as the symbols don't bear meaning beyond being distinct from oneanother) gives you a (possibly new) valid task. But also note that even with that extension to the simple filtering approach you'd probably be left with some nonsensical results (e.g. there are some tasks where there just is no notion of background color in the first place).

Hope that helps.

[drawlinson]

Thanks Michael that does help. In your opinion does black have a special status as default background colour in ARC, even though that isn't mentioned?

[michaelhodel]

For the original ARC dataset 0 is definitely predominantly the most common symbol. However I view this as more an artifact similar to the fact that grids of size 10 by 10 are also overrepresented. But ARC itself does not define further meaning of those 10 unique symbols, and surely does not require mapping 0 to black when visualizing grids. In my view, whether or not there are 8 or 10 unique symbols or whether or not symbols 1 and 2 are overrepresented relative to symbols 6 and 7 etc. are not and should not be defining characteristics of the ARC - both as a corpus and as a format.