(serial) zippering over ranges/domains/arrays of different shape/rank works, but shouldn't

[bradcray]

[Capturing what I consider to be a longstanding issue that I'm not finding an existing issue for]

Today, Chapel does not complain about serial zippered iteration over ranges, domains, and arrays that have the same number of elements but a different shape or rank. For example, a loop like:

for ijk in zip(1..6, {1..2, 1..3}, {1..3, 1..2}) do
  writeln(ijk);

works, where it should cause an error. I believe that this is a bug that we've been cognizant of for some time, but have not yet fixed.

The reason for this behavior today is due to the way we implement serial zippered loops, which is to iterate over each iterand independently. In my mind, the way to fix it would be to introduce a notion of serial leader/follower iterators that can do these sorts of checks for types that want to enforce them (e.g., leader says "I'm going to yield these things" and the followers can say "OK" or "I can't do that"). This would also permit users to create unbounded/shape-conforming types or iterators, similar to how unbounded ranges get special treatment in the compiler and language today (where that special behavior could be implemented via their serial follower iterators.

While it could be argued that perhaps Chapel is intentionally supporting such patterns, I don't believe this should be the case in order to be consistent with whole-array operations and parallel zippered loop, which are also supposed to match in shape/size/rank for these types (though some cases slip through today because of https://github.com/chapel-lang/chapel/issues/11428).

For users who do want to express such patterns, I think we'd want to do it through various utility iterators, such as linearize() or reshape() iterators that could change the conceptual shape of an iterands yielded values (e.g., see the discussion on https://github.com/chapel-lang/chapel/issues/9693, https://github.com/chapel-lang/chapel/issues/11504).

Associated Future Test(s): test/statements/loops/zippered/shapeMismatch.chpl #20979

Configuration Information

• Output of chpl --version: chpl version 1.29.0 pre-release (fb644c9f84)

[vasslitvinov]

What should we do in the following cases:

for ij in zip({1..2, 1..3}, myIter()) do ...
for ij in zip(myIter(), {1..2, 1..3}) do ...

where myIter() happens to yield six times without making any claims about its shape?

[bradcray]

for ij in zip(myIter(), {1..2, 1..3}) do ...

Ultimately, I think that in order to be used in a zippered context, myIter() would need to declare its shape when it was a leader (e.g., "I yield 6—or an unknown number of—singleton elements") and then it would be up to the 2D domain's follower iterator to say "Sorry, I don't conform to that without help."

for ij in zip({1..2, 1..3}, myIter()) do ...

In this case, I'd expect the domain's leader iterator to say "I'm yielding a 2D 2x3 thing" and for myIter()'s follower iterator to either say "I can't conform to a 2D leader" or to conform to it as it wishes.