Specification of `each`

[qlambert-pro]

Is the following model valid?

model DimensionTest
  constant Real ref[2] = {1.9, 3.0};

  model A
    Real x[2](start = ref[1:2]);
  equation
    der(x) = -x;
  end A;

  A a[3](each x(start = ref[1:2]));  
end DimensionTest;

My understanding is that it is not. My reasoning is the following. each applies the modifier ref[1:2] to all elements of the enclosing array. In the context of a.x.start the dimension of the enclosing array are [3, 2] therefore the modifier is expected to be a scalar.

The same reasoning would apply to the following model:

model DimensionTest2
  type B = Real[2];

  B b[3](each start = {1, 2});
equation
  der(b) = -b;
end DimensionTest2;

[henrikt-ma]

I thought these would be legal, as the array dimensions of the modifier would be considered when resolving the each; the array to populate has dimensions [3, 2] and the modifier has dimensions [2] matching the tail of the array dimensions, hence the array to populate is considered having three slots of type Real[2] to fill with the given value. Is any tool rejecting these models?

[eshmoylova]

According to the Specification it is legal. In https://specification.modelica.org/master/inheritance-modification-and-redeclaration.html#modifiers-for-array-elements there is an example:

model C
  parameter Real a [3];
  parameter Real d;
end C;
model B
  C c[5](each a ={1,2,3}, d={1,2,3,4,5});
  parameter Real b=0;
end B;

This implies c[i].a[j] = j and c[i].d = i.

The text says:

The each keyword on a modifier requires that it is applied in an array declaration/modification, and the modifier is applied individually to each element of the array (in case of nested modifiers this implies it is applied individually to each element of each element of the enclosing array; see example).

The emphasis is mine.

[qlambert-pro]

What bothers me about the example is that {1, 2, 3} has the correct number of dimensions for a in the first place. Therefore, it is applied to each element of the enclosing array without any ambiguity. But in the example I gave, the start is an attribute that concerns a single variable, it seems to me it is a scalar value. A modified version of the example that would be more analogous to my original models would be:

model C
  parameter Real a [3];
end C;
model B
  C c[5](each a = 1);
end B;

,

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](each a ={{1,2,3}, {1,2,3}});
end B;

or

model C
  parameter Real a [2, 3];
end C;
model B
  C c[5](each a ={1,2,3});
end B;

The text your quoting is very difficult for me to understand. It seems to me that it talks about how each relates to slicing rather than the ability to specify exactly as many dimensions as one wants, as long as one leaves one at the end to be filled automatically. I understood that sentence to mean that, given a binding that would have been valid as a declaration equation of the modified component, each applies that binding to all elements of the corresponding sliced array.

English is not my mother tongue so I am not sure whether there is ambiguity but I could imagine "enclosing array" being different than "unfilled dimensions" since an array can have several dimensions.

[henrikt-ma]

A modified version of the example that would be more analogous to my original models would be:

I like those examples, and think they could be a valuable addition to the current ones where each array component is declared with just a single dimension. However, I also think it would be even better if the rules for each were written so clearly that we wouldn't be dependent on half a dozen examples to complete the picture.

[qlambert-pro]

I agree, but first we need to agree on what is allowed.

[HansOlsson]

I thought these would be legal, as the array dimensions of the modifier would be considered when resolving the each; the array to populate has dimensions [3, 2] and the modifier has dimensions [2] matching the tail of the array dimensions, hence the array to populate is considered having three slots of type Real[2] to fill with the given value. Is any tool rejecting these models?

I agree. Obviously we can attempt to make the text clearer, but I don't understand the misunderstanding here - so I don't know how to best improve it.

[qlambert-pro]

Ok, does this also apply to the three other models I gave?

[HansOlsson]

Ok, does this also apply to the three other models I gave.

I see them as illegal, and I don't see the misunderstanding either.

[henrikt-ma]

I see them as illegal, and I don't see the misunderstanding either.

… and I thought they would be legal due to the equivalence between a multi-dimensional array and nested arrays.

[qlambert-pro]

Can you share your reasoning for the three models being illegal. The values given in the modifier can be used as argument of a carefully constructed call to fill which seems to be what @henrikt-ma is suggesting the meaning of each is? And mine is that all of these should be illegal because they are not valid binding to the original modified component.

[HansOlsson]

I see them as illegal, and I don't see the misunderstanding either.

… and I thought they would be legal due to the equivalence between a multi-dimensional array and nested arrays.

Are we talking about the same three examples?

I agree regarding seeing it as fill for, e.g.:

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](each a =...);
end B;

seems equivalent to a similar one with fill:

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](a =fill(...,size(c,1),size(c,2)));
end B;

However, for more complicated modifiers that is not always possible.

[qlambert-pro]

However, for more complicated modifiers that is not always possible.

I don't see that can you provide an example? or an idea of why it wouldn't be possible?

Are you then saying that my Cs are valid?

[HansOlsson]

However, for more complicated modifiers that is not always possible.

I don't see that can you provide an example? or an idea of why it wouldn't be possible?

Are you then saying that my Cs are valid?

No, invalid - same as:

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](a =fill(1, size(c,1),size(c,2)));
end B;

Basically the c.a is 5x2x3 array and the rhs is a 5x2 matrix.

[qlambert-pro]

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](a =fill(1, size(c,1),size(c,2),size(c,3)));
end B;

I meant that. Henrik explained that you would have to count the number of dimensions of the values and adjust to fill all dimensions of the array.

[HansOlsson]

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](a =fill(1, size(c,1),size(c,2),size(c,3)));
end B;

I meant that, Henrik explained that you would have to count the number of dimensions of the values and adjust.

But size(c, 3) does not exist! Note that I don't see "each" as a general fill all, but fill based on the enclosing array(s) (in this case c).

[qlambert-pro]

Sorry,

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](a =fill(1, size(c.a,1),size(c.a,2),size(c.a,3)));
end B;

[henrikt-ma]

But size(c, 3) does not exist!

In my mind, that should have been size(c.a, 3), but maybe this is what enclosing array is meant to rule out…

What I find odd then is that we have no way of filling it all with a single value in the given example, but I suppose one could still do this?

record R
  Real rr;
end R;
model C
  parameter R a [3];
end C;
model B
  C c[5, 2](a(each rr = 1));
end B;

Right – because there is never a need to specify each on levels above any given each?

[qlambert-pro]

Why isn't the size of the enclosing array of a.x.start in DimensionTest, [3, 2]?

[henrikt-ma]

Why isn't the size of the enclosing array of a.x.start in DimensionTest, [3, 2]?

Because the each sits above the x; enclosing refers to where the each is, not where the modifier is?

[qlambert-pro]

Would that be invalid then:

model DimensionTest4
  constant Real ref[2] = {1.9, 3.0};

  model A
    Real x[2](start = ref[1:2]);
  equation
    der(x) = -x;
  end A;

  A a[3](x(each start = ref[1:2]));  
end DimensionTest4;

and why isn't DimensionTest2 invalid?

[henrikt-ma]

Would that be invalid then:

model DimensionTest4
  constant Real ref[2] = {1.9, 3.0};

  model A
    Real x[2](start = ref[1:2]);
  equation
    der(x) = -x;
  end A;

  A a[3](x(each start = ref[1:2]));  
end DimensionTest4;

Good question. It somehow reminded me of model E in the specification, which I'd like to extend as follows:

model E
  B b[2](each c(each a={1,2,3}, d={1,2,3,4,5}), p={1,2});
  …
  B b3[2](each c(a={{1,2,3}, {4,5,6}, d={1,2,3,4,5}), p={1,2}); /* Give full array for 'a'. */
  B b4[2](each c(a={{1,2,3}, {4,5,6})); /* Remove modifiers for 'd' and 'p'. */
end E;

Here, instead of an each appearing "deeper down than expected", we end up with an each that does nothing.

By the way, shouldn't the p in this example actually be b?

[eshmoylova]

What bothers me about the example is that {1, 2, 3} has the correct number of dimensions for a in the first place. Therefore, it is applied to each element of the enclosing array without any ambiguity. But in the example I gave, the start is an attribute that concerns a single variable, it seems to me it is a scalar value.

I do not see how the start in x in the DimensionTest can be viewed as a scalar. You have:

model DimensionTest
... 
  model A
    Real x[2](start = ref[1:2]);
...
  end A;

  A a[3](each x(start = ref[1:2]));  
end DimensionTest;

DimensionTest.A.x is a array of size 2. So would be DimensionTest.A.x.start you can have DimensionTest.A.x[1].start and DimensionTest.A.x[2].start. DimensionTest.a.x is a [3,2]-array. And each element in that array has the start attribute.

A modified version of the example that would be more analogous to my original models would be:

model C
  parameter Real a [3];
end C;
model B
  C c[5](each a = 1);
end B;

,

model C
  parameter Real a [3];
end C;
model B
  C c[5, 2](each a ={{1,2,3}, {1,2,3}});
end B;

or

model C
  parameter Real a [2, 3];
end C;
model B
  C c[5](each a ={1,2,3});
end B;

I think all of those models have the wrong dimensions specified for a.

The text your quoting is very difficult for me to understand. It seems to me that it talks about how each relates to slicing rather than the ability to specify exactly as many dimensions as one wants, as long as one leaves one at the end to be filled automatically.

The way I understand it each is not meant to specify as many dimensions as one wants. Maybe there can be some different keyword for that.

I understood that sentence to mean that, given a binding that would have been valid as a declaration equation of the modified component, each applies that binding to all elements of the corresponding sliced array.

English is not my mother tongue so I am not sure whether there is ambiguity but I could imagine "enclosing array" being different than "unfilled dimensions" since an array can have several dimensions.

I interpret each and "enclosing array" as "each with respect to" whatever the first array that surrounds the variable for which the modification is given. For example, in model B above you have C c[5](each a = {1,2,3}); I read it as each a with respect to c, so c[i].a. So you need to determine what dimensions of c[i].a would be and provide the corresponding dimensions in the modification. This is my interpretation of what the text in the specification says and the examples seem to confirm it as far as I can see. I might be wrong, of course.

[qlambert-pro]

@eshmoylova I agree with a lot of what you said. The way I expressed myself was ambiguous. What I don't understand is, given:

So you need to determine what dimensions of c[i].a would be and provide the corresponding dimensions in the modification.

By the same reasoning, I would do in DimensionTest, DimensionTest.a[i].x[j].start is scalar, therefore the value of the modifier should be too.

[eshmoylova]

@eshmoylova I agree with a lot of what you said. The way I expressed myself was ambiguous. What I don't understand is, given:

So you need to determine what dimensions of c[i].a would be and provide the corresponding dimensions in the modification.

By the same reasoning, I would do in DimensionTest, DimensionTest.a[i].x[j].start is scalar, therefore the value of the modifier should be too.

I think if you want to specify it as a scalar you need to specify it with respect to DimensionTest.a[i].x[j] which you would do by moving each within the modification for x:

  A a[3](x(each start = ref[1]));  

or as:

  A a[3](each x.start = ref[1]));  

I see now that this is what you have in DimensionTest4 and why it would make it invalid.

[qlambert-pro]

@eshmoylova I can see that making sense. If the others agree, I can think of a proposal for a better wording and a helpful example.

[eshmoylova]

I think DimensionTest2 is valid because again I interpret it as each with respect to where each is written. You have:

model DimensionTest2
  type B = Real[2];

  B b[3](each start = {1, 2});
...
end DimensionTest2;

(Sorry for repeating the examples again but I find it hard to scroll back and forth while trying to explain or reading the explanation.) I interpret it as "each start with respect to the b[i]" because this is the point where each is written. b[i].start should be an array of size 2, so the modification is consistent. But... it can also be interpreted as each with respect to b[i,j] because this is the enclsoing array. In the latter case I would need to change the implementation in our tool.... There is not direct example of that in the specification, so I think both interpretations can be considered valid.

I also found #1596 where we discussed the issue of various versions of each the last time and came up with the current formulation. But I am not sure whether reading through the discussion would clarify the issue or would make it more confusing ;-)

[HansOlsson]

I interpret each and "enclosing array" as "each with respect to" whatever the first array that surrounds the variable for which the modification is given. For example, in model B above you have C c[5](each a = {1,2,3}); I read it as each a with respect to c, so c[i].a. So you need to determine what dimensions of c[i].a would be and provide the corresponding dimensions in the modification. This is my interpretation of what the text in the specification says and the examples seem to confirm it as far as I can see. I might be wrong, of course.

I agree with this.

[HansOlsson]

I think DimensionTest2 is valid because again I interpret it as each with respect to where each is written. You have:

model DimensionTest2
  type B = Real[2];

  B b[3](each start = {1, 2});
...
end DimensionTest2;

(Sorry for repeating the examples again but I find it hard to scroll back and forth while trying to explain or reading the explanation.) I interpret it as "each start with respect to the b[i]" because this is the point where each is written. b[i].start should be an array of size 2, so the modification is consistent. But... it can also be interpreted as each with respect to b[i,j] because this is the enclsoing array. In the latter case I would need to change the implementation in our tool.... There is not direct example of that in the specification, so I think both interpretations can be considered valid.

To me the second variant would make more sense (i.e. it is not consistent), but I agree that it is less clear.

[eshmoylova]

@henrikt-ma:

Good question. It somehow reminded me of model E in the specification, which I'd like to extend as follows:

model E
  B b[2](each c(each a={1,2,3}, d={1,2,3,4,5}), p={1,2});
  …
  B b3[2](each c(a={{1,2,3}, {4,5,6}, d={1,2,3,4,5}), p={1,2}); /* Give full array for 'a'. */
  B b4[2](each c(a={{1,2,3}, {4,5,6})); /* Remove modifiers for 'd' and 'p'. */
end E;

Here, instead of an each appearing "deeper down than expected", we end up with an each that does nothing.

I think the modifiers for a in b3 and b4 are invalid. B.c.a is supposed to be [5,3]-array. b3.c.a is [2,5,3]-array. I interpret B b3[2](each c(...)) to mean (...) applies to each b3[i].c, so the modification for a should provide [5,3]-array.

By the way, shouldn't the p in this example actually be b? You are right, it should be b. But I vote in favor of renaming parameter b in model B as p because there are so many other b's already here.

[henrikt-ma]

I think the modifiers for a in b3 and b4 are invalid. B.c.a is supposed to be [5,3]-array. b3.c.a is [2,5,3]-array. I interpret B b3[2](each c(...)) to mean (...) applies to each b3[i].c, so the modification for a should provide [5,3]-array.

Yes, my intention was rather:

  B b3[2](each c(a=fill({1,2,3}, 2, 5), d={1,2,3,4,5}), p={1,2}); /* Give full array for 'a'. */

However, your answer still contains an answer to the central question here, namely whether the each that was added for the sake of d and p could be disregarded for a. It makes sense that it doesn't, but it also appears as an unfortunate limitation that giving the complete array for a implies that the each we wanted for d and p has to be removed.

[qlambert-pro]

To me the second variant would make more sense (i.e. it is not consistent), but I agree that it is less clear.

I would be in favor of that too.

Maybe we could include "(with regard to the position of each)" after "enclosing", this would have helped me a lot in understanding what was meant, and maybe add DimensionTest2 as an example once we have agreed how it should be handled.

[eshmoylova]

@henrikt-ma, in response to https://github.com/modelica/ModelicaSpecification/issues/2630#issuecomment-670363349 (Is there a better way to specify what comment you are quoting?)

Yes, my intention was rather:

  B b3[2](each c(a=fill({1,2,3}, 2, 5), d={1,2,3,4,5}), p={1,2}); /* Give full array for 'a'. */

However, your answer still contains an answer to the central question here, namely whether the each that was added for the sake of d and p could be disregarded for a. It makes sense that it doesn't, but it also appears as an unfortunate limitation that giving the complete array for a implies that the each we wanted for d and p has to be removed.

I think if you want to use each for c.a you should not give the full array.

   B b3[2](each c(a=fill({1,2,3}, 5), d={1,2,3,4,5}), p={1,2});

If you want to give the full array for c.a and still use each for d you could do:

   B b3[2](c.a=fill({1,2,3},2, 5), each c(d={1,2,3,4,5}), p={1,2});

Note that in all of the examples for b3 each did not apply to p.

[henrikt-ma]

If you want to give the full array for c.a and still use each for d you could do:

   B b3[2](c.a=fill({1,2,3},2, 5), each c(d={1,2,3,4,5}), p={1,2});

Right, I can see how splitting the modifier like this solves the problem with only having the each on the modifier that should have it, but doesn't this kind of splitting of modifiers risk eventually violating this rule from 7.2.4?

When using qualified names the different qualified names starting with the same identifier are merged into one modifier.

… or can one simply avoid violating that rule by always using nested modifiers instead of modifiers with qualified names – since there is no corresponding rule for nested modifiers?!

[eshmoylova]

If you want to give the full array for c.a and still use each for d you could do:

   B b3[2](c.a=fill({1,2,3},2, 5), each c(d={1,2,3,4,5}), p={1,2});

Right, I can see how splitting the modifier like this solves the problem with only having the each on the modifier that should have it, but doesn't this kind of splitting of modifiers risk eventually violating this rule from 7.2.4?

When using qualified names the different qualified names starting with the same identifier are merged into one modifier.

… or can one simply avoid violating that rule by always using nested modifiers instead of modifiers with qualified names – since there is no corresponding rule for nested modifiers?!

If you wrote:

   B b3[2](c.a=fill({1,2,3},2, 5), each c(d={1,2,3,4,5}, a = fill({1,2,3},5)), p={1,2});

then, yes, you would violate the single modifier rule (the modifier for c.a is given twice). But there is always a risk somebody would write (c.a = ..., c(a=..)) The specification allows it. I do not know if there is anything you can do to avoid or minimize the risk.

[henrikt-ma]

If you wrote:

   B b3[2](c.a=fill({1,2,3},2, 5), each c(d={1,2,3,4,5}, a = fill({1,2,3},5)), p={1,2});

then, yes, you would violate the single modifier rule (the modifier for c.a is given twice).

Really – I only see one modifier using qualified names? My plan for circumventing the restriction was to never use any qualified name at all:

B b3[2](c(a=fill({1,2,3},2, 5)), each c(d={1,2,3,4,5}), p={1,2});

What I am talking about here is a procedure that gives maximum freedom in the use of each (within the limits of its current semantics): By never having more than one leaf modifier in any nested modifier, every modifier value can have it's own level of eachness. What makes me a bit concerned, though, is that this procedure goes in the direction of splitting modifiers, whereas the specification talks about the need for merging modifiers (but only when using qualified names). I must be missing something, right?

[maltelenz]

The linked pull request #2648 which closed this issue explicitly states that it does not resolve the issue. Reopening.

[HansOlsson]

This ticket has too many minor issues and mistakes back and forth that it is not easy to see what the actual issue is; and it needs to be cleaned up.