Cannot pass function as input via call block using implicit arguments

[brollb]

I have been encountering an error when trying to pass a function to a higher order function block as below. I suspect that perhaps the ring in the map block is causing the first input not to be detected as an empty input.

I ran into this in NetsBlox and was going to dig into it but wanted to first make sure this wasn't by design (and avoiding some bigger issue if the behavior is changed!). Is this the expected behavior?

[cycomachead]

Doesn't the ring need to go around the 2 x ( ) block? Otherwise you're passing in 0. :) Though that doesn't quite make sense with the error message still...

[brollb]

Haha, good catch! Here is the updated figure :)

[cycomachead]

orr OK, yeah, this is a bit unfortunate...but here's the correct way. You need to use formal parameters to make this work.

[cycomachead]

The reason the 2nd one doesn't work is that you can't "remove" the ring input to map, so you need to use the #1 trick to get around that. @brianharvey I forget if we've discussed before if this is something to try to address or if the solutions are just too messy.

I ran into this in NetsBlox and was going to dig into it but wanted to first make sure this wasn't by design (and avoiding some bigger issue if the behavior is changed!). Is this the expected behavior?

My own thoughts are that:

• call or map are not broken, but this is definitely a very confusing case.
• the error message that's reported is not super helpful.
• passing a lambda using something like call probably shouldn't create an additional lambda, but I'm not sure of a resolution that' not kind of a hacky special case? Or maybe just all lambdas should be directly passed as lambdas and we ignore the extra input type...

[brollb]

Hmm... I was hoping there was a way to avoid that... :) I posted a simple example so I didn't get caught up in a bunch of irrelevant details but basically I am actually using this in the autograder where the assignment creator is defining a function which accepts the custom block as an input so they are unable to add formal parameters (ie, the inputs need to be automatically detected). As an example:

I am sure I can detect them automatically but it seemed like this falls under the umbrella of improving the automatic argument input detection (assuming this isn't the expected behavior!). However, my specific use case is a little bit of an exception since I know that all the inputs are empty so they don't really need to be detected in any clever way...

[brollb]

I guess my question could be more specifically asked as "should ring inputs be considered empty (ie, isEmptySlot) if they have no children?" It seems to me that the answer should be yes but I could be overlooking some edge case...

[cycomachead]

I see. Yeah, the PR makes sense. I think we've gone back and forth about what's "right", if I am remembering previous discussions. However, I do think empty rings should be considered empty slots. IMO, that makes sense, but I may be forgetting something.

[brianharvey]

A while ago we decided that an empty reporter ring should mean the identity function. That decision might have broken this, although we could consider them empty for purposes of substitution, and then if they're /still/ empty, treat them as the identity function.

[cycomachead]

Ah, I vaguely remember this. What was the reason for that? identity functions are useful, but it seems a little odd.

[brianharvey]

Well, it had to mean /something/, even if only an error message. And, if you think of the inside of the ring as a slot into which something can be substituted (rather than including the ring as part of the emptiness) then it's kind of like the block-with-no-name sort of identity function.

[jmoenig]

I'm not completely opposed to this suggestion, in fact, we've been discussing it on and off for a long time. Three points are important for me:

1) Contrary to the title of this issue we can, in fact, pass a function as input to another block with a "ring type" input via the call block using a formal parameter. We're not lacking the expressivity, just the modality of implicit parameters. 2) Not all input slots support implicit parameters. 3) I think it's important to agree on the meaning of "empty slot" for rings.

ad 1) @cycomachead has pointed out the formal parameter syntax. It's a little bit funny how I imagined formal parameters to be the normal use case, and how folks keep being surprised they exist at all, because they're only aware of implicit ones. This is both a design success story (yay for @brianharvey !) and a little worrying, because Snap! might appear more different to other programming languages that only have named parameters than is necessary. I'm afraid that this wrong impression of not supporting "real" named inputs might even be a motivation for educators to "move to a more adult" programming language sooner.

ad 2) "empty" slots are pretty much universal in Snap! except for cases where the input type isn't (yet) first-class. The common example is that of the color-type input, which currently can't be blank:

Even though @brianharvey is working on changing that as we speak, and we agreed - in principle - to support dropping reporters that return costumes and lists in those slots. But that still won't make them support implicit parameters, and I'm not even sure whether and how those could / should be supported at all.

Other examples of inputs that can't (as of now) be blank are upvars and "static" lists.

ad 3) what does an "empty ring" even mean? I'm not unhappy with @brianharvey's insistence on letting it behave as identity function, at least for some input types, especially lists. This lets us easily create shallow copies of lists just using an empty function slot in map. But it's clearly a hack, a fallback behavior for an as of yet missing normal case. To me - and I hope to Snap's users, a ring is a function literal, an expression that takes another expression and an optional set of input names and returns a function. A "plain" ring really has two "empty" slots, one for an expression (a reporter or command script) and another one for inputs. Snap is currently not supporting either "expressions" or "inputs" as first-class citizens.

For the future I really want to extend Snap! with first-class expressions ("blocks") and also with first-class variables (which can then also be formal parameters for functions). This will let us programmatically create scripts as well as functions, even custom blocks. There are, of course, lots of design issues around this, starting with how to visually represent and "quote" an expression. We've been discussing this on and off, I'm currently leaning towards @bromagosa's suggestion to simply visualize expressions as plain blocks, but @brianharvey's proposal to overload rings with the semantics of quoting expressions (as we've also done with continuations) is also possible, interesting and attractive. We'll also need the reverse operations that will make functions truly first-class: Querying a function for its body (an expression), its name (we're currently abusing block-specs for this, and this might be right, except in the case of translations, which is, of course, the normal case we tend to forget and ignore outside the U.S.), and its inputs (which might just be text, a list of names, or something more block-ish).

While this is exciting to think about it also uncovers some more or less serious design flaws of present Snap, especially those concerning implicit inputs. For example, just looking at a Snap function that uses implicit inputs makes it impossible to tell how many implicit inputs it has or expects. In Snap's current practice this doesn't matter much, and is even kinda "beautiful", but when we're going to make functions fist-class, it's a cardinal sin. We might even end up treating implicit-input functions differently than formal-input ones. The latter can be reasoned about, the former only "used".

Also, this kind of going meta on functions and expressions might not be the lowest hanging fruit pedagogically. I'm personally excited about it, but I'm also excited about spreading messages that aren't so super meta in the first place, like recursion, hofs and linear algebra. And in the nearer term making the IDE more customizable to support actual curriculum ideas might be more worth the effort than adding yet more esoteric features which only a handful of savants are going to appreciate and criticize.

Anyway, great discussion!

[brollb]

Thanks for all the info! I suspected it might be a part of a much bigger discussion and it is nice to be brought up to speed! :)

@jmoenig - I updated the issue title to better reflect the actual issue!

[cycomachead]

I love these discussions! :)

My own sort of related thoughts:

I personally am not wedded to the idea that an empty ring is an identity -- I've always read it an an empty input, but I can see why it's more useful than an error message.

I think that part of the problem with the explicit system is that the parameters come after you use them. It's always felt a little backwards to me. I wonder if something where the inputs are first might help a little? This is inspired by JS:

I definitely agree that knowing how many & what inputs a function should have is important. It's made somewhat less important by the fact that Snap! tries very hard to be nice and permissive (e.g. filling multiple empty slots in map). However, it makes discovering the "intended" behavior a little more challenging when necessary. I'm not sure we should look to the textual languages here, because they're all over the map. (LOL!)

# Ruby
2.3.8 :001 > def mult(x, y)
2.3.8 :002?>   x * y
2.3.8 :003?>   end
 => :mult 
2.3.8 :004 > [1,2,3].map(&:mult)
ArgumentError: wrong number of arguments (given 0, expected 2)
# JS
> mult = (x, y) => x * y 
[Function: mult]
> [1,2,3].map(mult)
[ 0, 2, 6 ]
# Python
>>> def mult(x, y):
...     return x * y
... 
>>> map(mult, [1,2,3])
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: mult() takes exactly 2 arguments (1 given)

I definitely agree that there are more pressing larger issues, though I think Brian's fix is logical and is an improvement wherever there is an empty ring, overall. There's definitely bigger discussions but I don't think we need to solve every issue right now.

[brianharvey]

Wow, so many topics at once!

Michael, I had to laugh at your selection of text languages to look at for inspiration. (Hint: Six letters, starts with "S," ends with "e." :~P)

Jens, I'm not sure what you mean by "'static' lists" (as things that aren't substitutable into).

Also, Jens, being able to look inside the pieces of a data type is not part of the definition of first classness. (Example: functions in Scheme, the paradigmatic example of first classness.)

About the confusion between procedures and expressions, this is my fault. It's the result of one of the few cases in which Jens and I had a disagreement in which I was the one who wanted to optimize the behavior for naive users! Namely, way back in the BYOB 2.99 days, Jens correctly pointed out that if the ring means lambda, then the reported value of the lambda expression isn't a lambda expression, and should be displayed without a ring. But I said, and still think, that seeing the ring in the speech balloon makes it instantly clear to a lambda neophyte what they're seeing, and only much later will they start worrying about whether it's right or wrong to show a ring. I think one of the virtues of empty-slot substitution is precisely that it blurs the distinction between expressions and functions.

(Side comment: In APL, in which all functions are monadic or dyadic, all user-defined functions use α and ω as implicit left and right formal parameters.)

About making the arity of anonymous functions visible, we had a long discussion about ways to do this visually, with suggestions such as these: but Jens was, at the time, strongly opposed to any such thing. (Note that the arity of an anonymous function is specified only when it's used in a context that supplies a specific number of inputs. In the case of a user-defined function providing the context, the user would explicitly provide the arity of that function slot in the long form input dialog. Yes, it's possible to write weird custom blocks that call the same function with different arities in different circumstances, in which case the user just won't specify the arity of that slot and Snap! will behave as it does now.)

About macros (which is the name for being able to manipulate expressions as data), yes yes yes!!! Our biggest missing feature, not in terms of what users need right now, but in terms of us being a real grownup programming language. On my list of things to do (which, sadly, is growing over time rather than shrinking) is to use pumpkinhead's script library as the basis to implement macros as a library, not because I think that's how we should do it for users, but as a platform for us to work out the rough spots of a design.

Minor detail about passing a ringed input into an empty ringed slot: It's already the case that when you insert a ringed expression into a ring, one of the rings is absorbed by the other. (It should be that the one that isn't absorbed is the one with explicit formal parameters, if any...) So I don't think we have to worry too much about rings appearing inside rings; that happens only if you explicitly choose "ringify" from the context menu of the expression.

Umm, I wouldn't characterize my position on an empty reporter slot meaning the identity function as "insistence"; that idea came from users on the forum, and I supported it. (It's an interesting theoretical question what an empty predicate ring should report! An empty command ring should clearly, I think, just be a no-op.) But in any case, as is already the case for empty numeric slots, the empty slot should first be taken as a candidate for substitution, and if it's still empty after substitution, then it has its non-substituted meaning (0 for numeric slots, etc.).

[jmoenig]

@cycomachead I absolutely agree about the ordering of inputs and function body in rings, it's a conceptual problem that the formal parameters come after the body expression. Of course that's also something that makes it easier for users to get started with implicit parameters, which is why I said it's kind of "beautiful". Another similarly "beautiful" hack is automatically unringifying nested lambdas, e.g. when dropping a variable into a ring. It lowers the entry bar for using functions but makes understanding them much, much harder in the longer run. In many ways I feel we've made Snap! the perfect entry language for functions, kinda like a guitar lets you quickly advance to playing 2- and 3-chord songs, and we've successfully shifted the learning curve further away in time, but it's still there to come back with a vengeance, and it gets even harder in some cases, because of all those design kludges and little inconsistencies we put in (e.g. placing formal parameters after the function body in rings, allowing totally different semantics for handling multiple empty input slots inside a lambda at call time). I'm not saying these decisions are bad, in fact they are beautiful in some ways. But they also lower the expressive ceiling.

@brianharvey your color slot designs are a great example of that lower ceiling. They don't address the issue I've raised about determining the number of inputs for a function with implicit parameters, because by design we cannot tell before call-time. And even then those colors aren't helpful because now we're talking about doing to programmatically inside what you call macros, i.e. programmatically generated expressions. Sure, the "official" definition of first-class doesn't mention attributes. But those attributes sure come in handy when you want to reason about and use these things. Not being able to ask for the width and height of a picture - or getting an ambiguous answer - would be similarly awkward as not being able to tell the number of inputs to a function.

We don't need a kiddie library to play with blocks as data! This is what our code is doing precisely, this is what we do as Snap developers, it's what blocks.js is all about.

It really boils down to what we think a ring is. I've stated my opinion: A ring is function literal, an expression that takes another expression and a set of variable names and returns a function.

Another opinion could be: A ring is a shape for an input slot indicating that its expected data type is a function.

I'm sure we all agree on the first definition, but some of us also agree on the second one :-D

[brianharvey]

I knew you were going to say that about pumpkinhead's library! But, see, this way I can play with the ideas and the design entirely in Snap! without worrying about you yelling at me for uglifying blocks.js. :~) We can even let users propose competing designs, all entirely without messing with JS code. This should make you happy. (Of course it means pumpkinhead is doing the messing-with-JS instead, but he's already done it!)

It's true that putting the formal parameters after the expression in a ring helps users who don't use formal parameters at all, but it wouldn't be so terrible if it looked like instead. The real issue is that the way we have it emphasizes the expression, whether the user provides formal parameters or not. I would argue for the pedagogic superiority of the way we have it, with no loss of expressive power.

I don't understand what you're saying about the Colors blocks. Could you give a specific example? Thanks.

Automatic unringification is a whole different kettle of fish. That's a case in which we are violating a uniform syntax rule in response to a very clear need of users to whom function-as-data is a new idea. It's exactly correct that what we're doing is pushing the learning curve to the right, rather than eliminating the learning curve, which clearly can't be done. I don't agree that we're lowering the expressive ceiling; we have the Ringify option to handle the super-advanced uses. This case, I would argue, is pretty analogous to you inventing the COLUMNS primitive rather than a more general TRANSPOSE one. If you had somehow made it impossible to define a multi-dimensional transpose, then we'd be talking about lowering the ceiling. But no, what you did is lower the floor, and the same with automatic unringification.

I think the problem is entirely about documentation, namely, that I've been using the Reference Manual as also being the Tutorial Manual. Ideally, a Reference Manual would describe exactly how rings work, including all the edge cases, and why; the Tutorial Manual would explain the easy cases, give a lot of examples, and refer the reader to the Reference Manual for the full story.

The phrasing "an expression that takes another expression..." makes me nervous, because if a ring were an expression it would be evaluated in applicative order, i.e., the inner expression would be evaluated before invoking the ring itself. Rings are special forms, not expressions. And, yeah, a ring is also a shape for an input slot, not so very different from using a (stylized) picture of a list as the shape for a List-type input slot.

"Not being able to ask for the width and height of a picture - or getting an ambiguous answer - would be similarly awkward as not being able to tell the number of inputs to a function." Yes! But that (not being able to ask for the width and height) is exactly what is the case in Snap! at the moment: We're in this situation because you insist on the implementor's-eye view of the Turtle costume being something other than a costume, but I promise you that no user thinks of it that way. (This is also why NEXT COSTUME doesn't work as users expect.)

I don't think it's problematic that, e.g., the + block is monadic when used with MAP but dyadic when used with COMBINE. What's problematic is that we haven't taken advantage of the ability to use graphics to give the users the information they need, as in my pictures a few messages ago. Users don't know the arity of variadic functions either! I don't think that bothers anyone.

The reason the Scheme standard doesn't have selectors for body and parameters (although some implementations do provide them) is a deliberate choice, because given lexical scope, it's misleading to show users the body of a lambda expression and encourage them to think they know from that what the function does. They also need to see the defining environment. Environments aren't first class in Scheme entirely as a compromise for efficiency, but also, if they're implemented in a straightforward way, they are extremely circular lists (because environments point to functions, and functions point to environments; see SICP 3.2) and therefore hard to display.

[brianharvey]

P.S. If you think of arity not as a number but rather as three numbers, min/default/max, then you can tell users that the arity of + is 0/2/2.

[jmoenig]

[brianharvey]

Umm, okay... That's an interesting tool but I'm not sure what it's the answer to.

[jmoenig]

oh yeah, haha, pressed okay too quickly ;-)

that's an expression. It goes into an empty ring, eventually.

[jmoenig]

btw, I don't understand your remark about sprites without a costume, a.k.a. the "Turtle costume", or how that compares to a function with ambiguous parameters. I think this issue might be veering off topic.

[brianharvey]

Oh, you said that not knowing the arity of a procedure is as bad as it would be not to know the dimensions of a costume, so I pointed out that you don't know the dimensions of a costume, if it's the Turtle costume. We don't have to resolve that longstanding disagreement (you should've called it "None" if you didn't want users to think of it as a costume!); I just wanted to point out that you're not consistent in thinking that it's terrible if you can't see inside some data structure.

[DarDoro]

The reason the 2nd one doesn't work is that you can't "remove" the ring input to map, so you need to use the #1 trick to get around that>

Sligthly related to OP question