stdin

[saleyn]

Is there a simple way to have gsl read from stdin instead of a given script file? If so, could you please document it?

I understand that I can use the workaround below to open stdin device and read from it:

define in = file.open("/dev/stdin", "r")
while 1
    string = file.read(in)
    gsl string
endwhile

This doesn't seem to work either:

$ gsl <(echo 'echo "test"')
2015/05/08 19:09:03: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2015/05/08 19:09:03: gsl/4 I: Processing /dev/fd/63...
2015/05/08 19:09:03: gsl/4 E: Error processing /dev/fd/63...
2015/05/08 19:09:03: File not found

[dmeehan1968]

Looking at the sources, the filename argument is mandatory, and there is no syntax for accepting stdin.

[saleyn]

I also checked the sources, and it looks like the gsl is always trying to find the filename of the script using a set of rules. Is this a shortcoming that the rules are not relaxed for accepting stdin, or this is required by design? I can prepare a pull request at some point to fix this if there are no objections.

[hintjens]

Having a console/REPL way of working would be fun, indeed.

On Sat, May 9, 2015 at 1:19 PM, Serge Aleynikov notifications@github.com wrote:

I also checked the sources, and it looks like the gsl is always trying to find the filename of the script using a set of rules. Is this a shortcoming that the rules are not relaxed for accepting stdin, or this is required by design? I can prepare a pull request at some point to fix this if there are no objections.

— Reply to this email directly or view it on GitHub https://github.com/imatix/gsl/issues/95#issuecomment-100466795.

[saleyn]

Since the use of the gsl_*() family of functions is not well documented, could someone point out what's wrong with the following inserted at https://github.com/imatix/gsl/blob/master/src/gsl.c#L200 ?

            if (is_stdin)
              {
                char
                    *line = NULL;
                size_t
                    len   = 0;
                THREAD
                    *gthr = gsl_start(thread-> queue, 0, switches, 1, & root);
                while (getline(&line, &len, stdin) >= 0) {
                    RESULT_NODE *new_node = new_result_node();
                    gsl_evaluate(gthr, line, 0, &new_node, thread-> queue);
                    destroy_result(new_node);
                }
                gsl_finish(gthr);
                break;
              }
            else
                gsl_execute (thread-> queue, 0, switches,
                             1, & root);

Insertion of this code along with allowing the use of "/dev/stdin" results in the input being evaluated but doesn't print evaluation result:

$ cat /tmp/input.txt 
echo class(shuffle)
echo shuffle
$ gsl /dev/stdin < /tmp/input.txt 
2015/05/09 12:52:55: GSL/4.1a/github Copyright (c) 1996-2011 iMatix Corporation
2015/05/09 12:52:55: gsl/4 I: Processing /dev/stdin...

I checked the content pointed to by new_node in that loop, but it seems to be empty. What's the proper way of printing result one line at a time? The gsl's state machine code is far from trivial for someone not familiar with its design. :(

[hintjens]

@saleyn you're the first person to modify this code, except for @jschultz who wrote it... so I can't help much. Jonathan is usually here somewhere. If not I can try to work through what's happening...

[saleyn]

Thanks! The more I play with gsl the more useful it seems to be. :-) Having a functional top-level console evaluation loop for gsl would be quite helpful. Though I have a wish for a few more syntactic features:

1. optionally allow to use ';' for separating commands on the same line for brevity
2. implement a builtin fold function: fold(Callback, InitState, Scope) -> State,
    where Callback is a function: (Item, State) -> NewState.
3. support in-place lambda syntax:
        (Args...) -> Expr
        (Args...) -> { Expr; Expr; return Value; }
    This would allow fun things like get the max length of names of all nodes in the given scope:
        max_length = (inscope, name) -> fold((it, n) -> string.length(it.$(name)), 0, inscope)
        len = max_length(root->nodes, "id")

[hintjens]

These are neat ideas.

One warning on using gsl: if you're good with abstractions, it rapidly takes you places where other people can't easily follow. This can be counter-productive, if you want to make software that others can contribute to.

[saleyn]

I agree that current gsl imperative syntax is straight-forward to learn. Though given the fact that most of the modern languages nowadays include some functional features, I should say that people are not as scared about lambdas as they were a few years back.

[hintjens]

For sure! We merge all patches so long as they don't break existing code. If someone is particularly offended by a change they can patch it out again. :)

On Sat, May 9, 2015 at 7:54 PM, Serge Aleynikov notifications@github.com wrote:

I agree that current gsl imperative syntax is straight-forward to learn. Though given the fact that most of the modern languages nowadays include some functional features, I should say that people are not as scared about lambdas as they were a few years back.

— Reply to this email directly or view it on GitHub https://github.com/imatix/gsl/issues/95#issuecomment-100523485.

[saleyn]

Just to get back on the original "stdin" issue, I pushed an experimental branch that you or @jschultz can take a look at: https://github.com/saleyn/gsl/tree/stdin (compare: https://github.com/imatix/gsl/compare/master...saleyn:stdin) as the starting point where I left it off with the comments above.

[jschultz]

I'll try to find take a look at the stdin issue in the next few weeks. It does seem sensible to me to have predefined stdin/out/err file identifiers, in the interests of portability. And then a way for them to be used as the source of the gsl script or the output.

One difficulty might that gsl parses scripts by reading a file until the end, rather than until a coherent block of gsl code has been read, so any naive use of stdin would have to do the same. A smarter system would take considerably more work as it would require some internal restructuring. Could be nice though.

|1. optionally allow to use ';' for separating commands on the same line for brevity

This idea has been discussed in the past and rejected on the basis that as a template language, gsl is (or should be) line-based. That said, it is probably simple enough, though my guess is there will be unforeseen complications owing to the template mode.

1. implement a builtin fold function: fold(Callback, InitState, Scope) -> State, where Callback is a function: (Item, State) -> NewState.
2. support in-place lambda syntax: (Args...) -> Expr (Args...) -> { Expr; Expr; return Value; }

I presume these two go together, as I can't see any purpose for an anonymous function except as the callback for the fold function. In any case, these are non-trivial additions to the gsl syntax and seem to me pretty unlikely to be realistically achievable. Maybe gsl 5.0?

— Reply to this email directly or view it on GitHub https://github.com/imatix/gsl/issues/95#issuecomment-100520161.

[saleyn]

@jschultz thank you for taking the time to look into this.

This idea has been discussed in the past and rejected on the basis that as a template language, gsl is (or should be) line-based. That said, it is probably simple enough, though my guess is there will be unforeseen complications owing to the template mode.

In the defense of the ';' proposal, in the template mode it is quite frequent to have a one-liner assignment or function call surrounded by the 'if/ifend' pair. in those cases it would be more succinct to be able to put that on one line.

these are non-trivial additions to the gsl syntax and seem to me pretty unlikely to be realistically achievable. Maybe gsl 5.0?

After looking through the source of gsl I can see that it is not trivial to make gsl grammar changes. I wonder why in its design some common parser/lexer wasn't used (e.g. yacc or its analog) with a BNF-like grammar? It seems to me that gsl 5.0 could very well be implemented on top of some common runtime that addresses many things that gsl does (such as multi-threaded concurrent job control, etc). Erlang seems to be a good fit for it as it handles concurrency out of the box, is cross-platform, has yacc-like parser, XML support, etc. All it would take would be to write a BNF grammar for gsl syntax, and an Erlang expression evaluator...

[hintjens]

I think we started GSL in 1991 or so, and the focus was always on fitting the job of code generation. We experimented with it as a general purpose scripting language (e.g. for web page generation and server scripting) but that was pointless given other languages much better at that.

We're still making small improvements to it, and this is how we like to work: solve clear and obvious problems with minimal patches.

In terms of future code generators, I think one should revisit the question. For simple procedural code generation, GSL is unbeatable. However for serious use, grammar-based code generation is far more effective. We did that in 2005 with XNF, a generator generator driven by XML grammars. This is where new languages would be profitable.

[ghost]

I think I asked this before, but the situation might have changed: Is XNF publicly available somewhere? I remember, that you told me it is to complicated for normal people. ;-) Any plans to change that?

[hintjens]

Yes, there's a git repo with the entire stack we used to build OpenAMQ, which includes XNF:

https://github.com/imatix/openamq/tree/master/tooling/base2

On Tue, May 12, 2015 at 9:15 AM, Achim notifications@github.com wrote:

I think I asked this before, but the situation might have changed: Is XNF publicly available somewhere? I remember, that you told me it is to complicated for normal people. ;-) Any plans to change that?

— Reply to this email directly or view it on GitHub https://github.com/imatix/gsl/issues/95#issuecomment-101163124.

[jschultz]

After looking through the source of gsl I can see that it is not trivial to make gsl grammar changes. I wonder why in its design some common parser/lexer wasn't used (e.g. yacc or its analog) with a BNF-like grammar? It seems to me that gsl 5.0 could very well be implemented on top of some common runtime that addresses many things that gsl does (such as multi-threaded concurrency job control, etc). Erlang seems to be a good fit for it as it handles concurrency out of the box, is cross-platform, has yacc-like parser, XML support, etc. All it would take would be to write a BNF grammar for gsl syntax, and an Erlang expression evaluator...

Certainly using tools like yacc and Erlang have long seemed to me like an interesting way to go. However if I remember correctly, gsl is not describable as a BNF grammar, because its template/non-template lines mean that it isn't context-free. But maybe you know more about this than I do?

Cheers, Jonathan

[saleyn]

It seems to me that the complication of having template/script context can be reduced to having a context-free grammar if at the stage of the lexer it treats a whole template block as a blob of text, while working in the script mode, and at the parsing stage would apply a secondary-level lexer to the template text in order to being able to detect $(...) expressions.

A better approach would be to define a grammar for a stateful lexer, so that it's able to switch between the template/script states.

There are some Erlang projects available for implementing advanced stateful lexers, e.g.: https://github.com/erlydtl/slex (here is what a sample lexer's grammar looks like for DJANGO syntax: https://github.com/erlydtl/erlydtl/blob/master/src/erlydtl_scanner.slex)

Though my plate is quite full for the next few months, but thereafter I could offer some help if there is interest in trying this approach.

[jschultz]

This is all no doubt possible, but the fact of having to think about work-arounds before even starting makes me even more doubtful that it would be worth the substantial effort.

Cheers, Jonathan

On 13/05/15 23:23, Serge Aleynikov wrote:

It seems to me that the complication of having template/script context can be reduced to having a context-free grammar if at the stage of the lexer it treats a whole template block as a blob of text, while working in the script mode, and at the parsing stage would apply a secondary-level lexer to the template text in order to being able to detect $(...) expressions.

A better approach would be to define a grammar for a stateful lexer, so that it's able to switch between the template/script states.

There are some Erlang projects available for implementing advanced stateful lexers, e.g.: https://github.com/erlydtl/slex (here is what a sample lexer's grammer looks like for DJANGO syntax: https://github.com/erlydtl/erlydtl/blob/master/src/erlydtl_scanner.slex)

— Reply to this email directly or view it on GitHub https://github.com/imatix/gsl/issues/95#issuecomment-101661904.

[saleyn]

One thing that's hard not to agree with is that gsl does what it's designed to do quite well. :-) The ease of its extensibility is a real question.