fchollet / ARC-AGI

The Abstraction and Reasoning Corpus
Apache License 2.0
3.31k stars 548 forks source link

79fb03f4 test is unsolvable, water flow #100

Closed neoneye closed 3 months ago

neoneye commented 6 months ago

Problem

ARC-Interactive

My first prediction

My reasoning.

Screenshot 2024-02-25 at 02 04 43 copy

My second prediction

My reasoning.

Screenshot 2024-02-25 at 02 09 55 copy

At this point I looked at what was the expected result.

Expected

expected

I'm quite surprised to see that the water flows into the hole. But not around the entire structure. It seems like a mistake.

There is no example among the train pairs, that shows this behavior.

Solution

Tweak one of the train pairs, so it demonstrates what to do in similar scenarios.

CerebralAdvantage commented 3 months ago

It's true, the train pairs do not train for this specific situation, and it seems very difficult. I solved it the first try. The key to solving it is not overthinking what is happening. There are very few things that the "water" actually does. Because it splits around a "stone", it can then do two things... until it passes said stone. After which, it needs to continue, immediately in line with the obstacle, as a single stream. The area in question is very concerning, but only if you get carried away with your narrative. I know that last statement doesn't help. It is 100% solvable. Try thinking more "simply."

AGI enthusiasts bonus for being gihub-savvy: note the use of "water" and "stone," something most adults have experience with (I grew up in a forest). How can one possibly help a machine relate to these "naturally human" ideas? Answer that question, and your code might do quite well! Hint: Grab a billion photos of hillside creeks from the world wide web... not!

CerebralAdvantage commented 3 months ago

That said, example 5 output is quite incorrect. The middle stone has its flow blocked, and it should only continue after the upper and lower stone. It definitely should not be flowing back uphill. I would love to be corrected. If there are these tiny errors in the examples, someone (or some thing...) attempting to use every detail of every example will fail. As a human, I simply ignored the complexity of 5 output. But after further thought ("Agua...") I want back to the crime scene and sure enough, the wrongness is apparent. --

Since it is solvable, then the 5 output must be incorrect. Otherwise, @neoneye's "if it can flow uphill" comment would totally hold. It's one or the other, and I'd put my money on "bad example." Anyone else solve this and feel that 5 output is OK?

CerebralAdvantage commented 3 months ago

(sigh) on much more careful examination, example 4 output shows precisely what should happen. The flow is just blocked on one side, and it does a "half stream" process on the other side. example 4 proves example 5 is incorrect.

CerebralAdvantage commented 3 months ago

image suggested edit of example 5 output...

...or leave it! Humans make mistakes all the time, and other humans (granted, not all other humans) can usually work around those mistakes. If you want truly G AGI, it would have to be able to navigate an imperfect world!!! "Thanks for the workout..." -Bruce Lee

fchollet commented 3 months ago

Thanks for the reports. I issued a fix for this task (live at HEAD).