"For" Loop support

[daneb]

Hi

I am well aware of https://github.com/manastech/crystal/issues/725#issuecomment-107708015 where "for" is proposed and it's suggested that "iterators" are sufficient.

Ruby does support "for" and it's referenced in benchmark.rb as part of stdlib documentation.

So my question - are we intentionally choosing not to support "for" as a design decision? I do see the keyword in lexer_spec.cr (line 132)

Thank you

Dane

[jhass]

Already answered in IRC but for completeness here too.

for is redundant to internal iterators, Iterator etc. It's since long discouraged in the Ruby community due to its redundancy and unexpected scoping behaviour. In fact it calls .each underneath in Ruby and is slightly slower in recent versions than using .each directly. A doc patch towards Ruby to remove the mention in its stdlib documentation examples will have good chances of being accepted.

Like with other occasions of redundancy in Ruby, Crystal has chosen to leave it out, this is an intentional design decision. It's in the lexer because it's valid and part of the macro language, which in turn doesn't support the .each call, this gives a better visual distinction between the macro language and the code it generates.

[ysbaddaden]

Actually, for is available in macros (hence why the keyword is defined):

{% for key, index in %w(get post put) %}
  ...
{% end %}

[daneb]

Thank you very much @jhass

[ozra]

For is a redundancy, but a very nice one in my eyes. I prefer being able to do things in more than one way - choosing the one that looks best to me / my team in a given situation / construct.

I've used each constructs extensively in different languages but I still prefer for x, i in list by far. It's short, it's just the same thing - as mentioned - but with much cleaner syntax imo. The macro visual distinction argument.. I don't buy that ;-)

I really hope this can be reconsidered in the future. It's an unnecessary superficial limitation on the programmers choice to express code.

[refi64]

To me, the issue is that sugar is addictive.

It starts with for loops. Then you have a keyword as an alternative to reduce (or whatever it's called in Ruby). Next thing, you have every single OO, FP, concatenative, active, and parallel feature on the face of the planet, and a language reference just as big.

[asterite]

Yes, that's the main problem: you introduce a feature because some like it, but then everyone has to learn it because they might encounter it. In fact I didn't know about "for" in Ruby and when I encountered it I didn't know what was its purpose, but had to learn it anyway and never used it.

[ozra]

Yes, this really comes down to two mindsets, or schools if you like, to the acceptance of such things, I'm very pro choice in coding. Some people even frown upon suffix if's with disgust - I like them, and find that, especially in early return patterns, they are preferable to the common variant. (Luckily crystal has them :-)

I believe the community and eco system should promote a style, but that all kinds of styles, constructs and honey should be available (as long as they don't cause tough parsing ambiguity, which inevitably locks down things so future additions get hurt) so that all coders and teams can choose without limitations for their projects. If I'd code a library module I thought could be of great use to others, I'd stick to the norm of the community, at the very least for the API / public interface.

Most people only learns the common parts, expanding per need anyway. Few people know all the karate of C++ and get by perfectly with their every day coding (not a good example, since it's horrible, but.. ;)

In any event - I can live without for :-)

[bcardiff]

I like the idea of having mostly one way to do each thing. Less things to learn when you read others code.

But for each project or for a specific scenario I also like to be able to adapt the language to the needs.

Adding the for/in as a macro might be too much. Since it seems something to core to the language. But it is doable. The following snippet does the work

http://play.crystal-lang.org/#/r/5rc

Yet there are some checks to be done in order to ensure the "in" between the for. Right now any identifier can be placed instead the in

[ozra]

Hey, that's awesome!

Is it possible to get rid of the 'do' at the end? And is it possible to also get the for v, i in a - oh, and something akin to: for _, i in a or for ,i in a?

I'll use this instead of the lisp.notation.inverse (as I've come to view dogmatic OOP-pattern use). Most people do think 'for...' in pseudo - not the Yoda way. I like it when pseudo is a second away from being runnable. That said - perhaps this all comes down to Germanic, Romance, Japonic natural language bindings, stemming from SOV vs SVO structure. If so, it's racist to not have for ;-)

And, once again for the 'less to learn' argument - I still find that doesn't hold water at all:

• Inevitably this is what happens
• People looking at my code will go, either:
  1. For in? Wtf! That's awesome, didn't know crystal had it. And fail when trying in their code, check the man, ask the forum, then look up the macro, then decipher it.
  2. For in? Wtf is that? Better check the man. Fail. Ask the forum, then look up the macro, then decipher it.
  3. Wtf? A for in macro? Looks it up, then decipher it.
• And personally: compile time might suffer

for in as part of the language would be preferable ;-) Possibly then also the postfix for, just like if, while etc.

In any event - thanks again for the macro :-)

[bcardiff]

@ozra I'm glad you like it

Playing with macros to somehow extend the language reduces to find a valid crystal statement and transform it to give the semantic. But the catch is that is has to be a valid crystal statement.

I defined an empty single argument macro and start checking if it parses different alternatives. Then navigate the ast to extract the information.

So

Is it possible to get rid of the 'do' at the end?

No. We need a block for the body. Only some first class statement do not force the do in order to have a body: class, def, if, etc.

And is it possible to also get the for v, i i in a

Not like that. The for macro in my previous comment work by with the expression v in a with is actually v(in(a)) with a block in the inner in method call. (Sorry, nasty things going inside the macro)

The possible alternatives I have are for v i in a do ... without the comma. Or foreach a[i] = e do ....

something akin to: for _, i in a

No, because _ is not a valid identifier

or for ,i in a?

And not again. It is not a valid syntax. But for i in a.indices could do the work if Array#indices is added. But it is more code to type.

btw, i updated the macro to perform a syntax check regarding the in. Check https://gist.github.com/bcardiff/05482bb9dddeacf41856

[ozra]

Aha, this is how I added type checking and macros to LS - by writing legal code, and mutating the AST, then rendering and eval'ing code dynamically - equally ugly ;-) So same thing here, got it.

Thanks for the effort! Much appreciated. I'll try and hack in the for v[i] in a variant - that was an innovative one! Much clearer regarding intent then the commonly used ones! :-) Both should be possible to support through conditional macro branches, right?

[vendethiel]

Aha, this is how I added type checking and macros to LS - by writing legal code, and mutating the AST, then rendering and eval'ing code dynamically - equally ugly ;-) So same thing here, got it.

( I'd love to see that, though :). )

[bcardiff]

Yes @ozra, it should be doable with conditional macro branches :-)

for arr[i] = e do
  puts e
end

Generates the following ast

Crystal::Call
  @name:
    for :: String
  @args:
    [0]
    Crystal::Call
      @obj:
        Crystal::Call
          @name:
            arr :: String
      @name:
        []= :: String
      @args:
        [0]
        Crystal::Call
          @name:
            i :: String
        [1]
        Crystal::Call
          @name:
            e :: String
  @block:
    Crystal::Block
        @body:
            ... puts e ast...

While

for e in arr do
  puts e
end

Generates

Crystal::Call
  @name:
    for :: String
  @args:
    [0]
    Crystal::Call
      @name:
        e :: String
      @args:
        [0]
        Crystal::Call
          @name:
            in :: String
          @args:
            [0]
            Crystal::Call
              @name:
                arr :: String
          @block:
            ... puts e ast...

With that and the gist you should be able to get the desired macro.

[ozra]

For the record, my wish for a for... construct is that it scopes iteration vars within the loop.

It really should just be pure sugar (as already mentioned) for using .each, .each_index, .each_with_index or whatever they're all called, behind the scenes - just selecting the appropriate method depending on which of val, index or both are asked for via the much terser notations:

for val in list
    print val
end

for val[ix] in list  # or `for val, ix in list`
    print val, ", ", other_list[ix]
end

for [ix] in list  # or `for _, ix in list` or `for ,ix in list`
    print other_list[ix]  if list[ix] == 47
end

The syntaxes using subscript to denote index var as shown above in @bcardiff examples, slightly varied for macros sake, is a really nice idea! Makes all three choices equally clean in syntax, and makes the programmers intent very clear. I'm more used to the comma variants. I would love to see this in Crystal :-D

Will I get shot if I open a new issue specifically with this as a wish, seeing that it's been up already?

[ozra]

@bcardiff - OT here, but how do you debug print the AST? I'd like to know both for macros and for the real AST. Only answer I've gotten so far is "probably not possible", did you write above AST manually??

[bcardiff]

I've just extracted a shareable version :-) https://gist.github.com/bcardiff/a663b1ea1e8fd6a308cc

Happy Macroing!

[ozra]

Thanks a f-in million! Was afraid I was gonna have to code it, haha. :-D

[bcardiff]

@ozra there is also a snippet in https://groups.google.com/forum/?fromgroups#!searchin/crystal-lang/json/crystal-lang/VBcUdJNUM-s/WCX4ke53Z10J that produces a json

[vendethiel]

Ohh, patching crystal for dev purposes... I like it

[ozra]

@bcardiff - cool - the YAMLish output was great for investigation though :-) Aaah yeah.

[schovi]

Hi, just reach this and my humble opinion is that for loop is just nice very nice sugar to

i = 0
while i < 100
  do something

  i += 1
end

for loop looks just better in this. And bonus is simple. Iteratig over large count of numbers without need to create some huge array, range etc.

[kostya]

range is nothing, 2 ints on stack:

0.upto(100) { |i| do something }
(0..100).each { |i| do something }

[schovi]

@kostya Yes, but syntax is not natural as in for ...

for(i=0; !isDone, i +=5) { ... }

iterating each 5 and until some special condition is done

[jhass]

Not to forget

100.times { do_something }

If you can show some real usecases for for, we can show you idiomatic alternatives.

[ysbaddaden]

@schovi There is nothing natural in for loops, but a habit of other languages that have it. They can be rewritten to more idiomatic Crystal constructs (valid for Ruby too), usually using #times, #to, #upto, #downto, #step, or iterators on a Range. All these methods only have 2 or 3 integers on the stack and should be optimized out by LLVM.

Your example can be rewritten as the following for example:

0.step(by: 5) do |i|
  break if done?
end

Please look at the documentation for Number, Int and Range for more details.

[RX14]

It's very likely that in release mode that using upto times even Range etc. will compile down to the same assembly as a for loop would.

[schovi]

@ysbaddaden You have "bug" in example It should be

0.step(by: 5) do |i| break if done? end

I have to admit it is pretty slick :) I am just old fashion and live longer time than I want in javascript world for now.

[ysbaddaden]

@schovi you're right. I fixed it. It's kinda weird that the first argument is the optional limit. @asterite what do you think? Shouldn't the args be inverted, or not?

[asterite]

@ysbaddaden Yes, it should probably be inverted. We also have Iterator#step(n) where n is the step. But the original order came from Ruby.

Or maybe we can make those two always be passed by name, so the confusion is never there. (and also rename limit to to so it's shorter to write, and similar to Ruby)

[ysbaddaden]

always pass them by name + rename limit as to

:+1:

[schovi]

Can I try to implement that when there will be final conclusion? I tried to dig into crystal source code some time ago, but nothing fit to me.

[asterite]

@schovi Sure! You can start whenever you want and send a PR. Make sure to change all def step in the standard library to use the by name, and make it only be available as a named argument (there are some step methods in Range and Iterator). Be sure to read this first.

[schovi]

@asterite Thanks. I read all these docs so many times, but cant make it usefull, till now!

[RX14]

Forcing step(by: 5, to: x) sounds good.

[asterite]

Yes, one can also do step(by: 5) (assumes no limit) or step(to: 10) (assumes by = 1), but never step(1, 2).

I wonder what should we do with Range#step and Iterator#step. They only receive one argument, the by. Maybe we shouldn't force a named argument there, but still use by as the argument name. That way we can still do [1, 2, 3].each.step(2) and [1, 2, 3].each.step(by: 2) without necessary forcing the second usage.

What do you think?

[schovi]

@asterite I like [1, 2, 3].each.step(2) and [1, 2, 3].each.step(by: 2) and for Number .step(by: 2, to: 10). It looks natural.

[Iainmon]

After looking around for a while I put together this: https://gist.github.com/Iainmon/2beb2a75c20f78408e39529db6a6dc74

[RX14]

@Iainmon please don't invent your own control structures, it will be confusing for other devs. There are plenty of ways to do the equivalent more declaritively.

[stefandd]

@Iainmon please don't invent your own control structures, it will be confusing for other devs. There are plenty of ways to do the equivalent more declaritively.

I find a request to not be creative in this context bizarre, especially since what is done here is perfectly in line with the purpose of macros. In addition, I don't like to assume other devs have to be intellectually protected or shielded. One should not infer from one's own being confused that anyone else would be too.