Variable declaration vs assignment

[akva]

I figured that general discussion of CoffeeScript belongs to issue tracker. So here it goes.

It seems like coffee-script does not distinguish variable declaration and assignment. This means that, unless you are very careful with naming local variables, it's easy to unintentionally assign to a global variable. Consider this js code:

sys = require('sys');
var foo = 42;
(function() {
    var foo = 43
}());
sys.puts(foo);

Here variable foo inside a function is local to that function. It is not possible to reproduce this code in CoffeeScript. CoffeeScript 'equivalent' will override global variable foo.

sys = require 'sys'
foo = 42
(() -> foo = 43)()
sys.puts foo

This may lead to hard to find bugs. What's your thoughts on this?

Regards, -- Vitali

[StanAngeloff]

It has been well documented and discussed on the tracker before. I am actually quite happy with how Coffee handles scoping at the moment.

[akva]

I just read the documentation again more carefully. Indeed, I was too quick to post an issue. My bad. However, let me elaborate on this issue a little bit. This is a part of the documentation that explains my point in fewer words.

Because you don't have direct access to the var keyword, it's impossible to shadow an outer variable on purpose, you may only refer to it. So be careful that you're not reusing the name of an external variable accidentally, if you're writing a deeply nested function.

Consider this code

...
... many lines of code (likely not written by me)
...
func = () -> # a new function that I add
    foo = 42 # not clear if I declare a local var or assign a global one

JS has different syntax for variable declaration var foo = 42 and assignment foo = 42. The problem with it imho is that declaration syntax is more verbose than assignment syntax (and var is easy to forget). It seems that variable declarations, especially if one tends to write in functional style, appear in source code more often than assignments. Therefore declaration syntax should be more concise and assignment could be more verbose.

My preferred syntax would look something like this:

foo = 'foo'
bar = 'bar'
...
... many lines of code
...
func = () ->
    foo = 42  # declares local var. shadows global
    bar := 43 # assigns global var
    fooo := 44 # compile error

This approach is only slightly more verbose but makes programs more explicit and robust, imho. And again, in my own code I tend not to use destructive assignment that much.

What do you think?

P.S. Python uses nonlocal keyword but it's a little overkill for my taste.

[StanAngeloff]

How about referencing a variable from the global scope? If you simply do alert bar from a function, should it throw an exception if bar is not in scope? It seems like it should since assigning does so and we need things to be consistent.

When you are writing in Coffee, you will not always have full control over what files you interface with. While it may be possible to throw compile time errors when referencing a non-existing global variable, we cannot reliably do so when you are working on a file that just happens to be a small part of an application which in turn might have been written in pure JavaScript.

Coffee is also used on the server-side. In node.js it's common to have many callbacks, like so:

fs.readFile 'file.js', 'utf8', (err, contents) ->
  # do some other stuff
  process.nextTick ->
    throw err if err

Here we are referencing err in a function that is nested inside another function. Since err is not in scope, but it's also not part of the global one, how do you assign or reference it?

Keep in mind Coffee is a dynamic language just as JavaScript is. There is no type checking, no NullReference exceptions and no actual compilation.

If I have to point out one benefit of the existing scoping, it would be that it's consistent. Define once and use everywhere.

[akva]

Maybe I didn't express myself clear enough. I guess I should have used the word outer scope instead of global.

I am not proposing to change the scoping rules. Quite the opposite, I propose to keep them exactly the same as in JavaScript, and therefore making Coffee more consistent with JavaScript (I am as big fan of consistency). One way to do it is to introduce the following syntax

1. foo = 42 to mean JavaScript's var foo = 42;
2. foo := 42 to mean JavaScript's foo = 42;

Right now foo = 42 can mean two different things in Coffee depending on the context. It can be a variable declaration, or an assignment to a lexically scoped variable. And this is not consistent.

So your example with fs.readFile will just work unchanged as expected. err is in lexical scope of the nested function so everything is all right.

[goj]

How about keeping current semantics and re-introducing JavaScript's var keyword to force declaration?

v = 1
fun = () ->
    var v = 2
fun()
alert v # shows 1

[akva]

If I understand correctly what you mean by forced declaration, it doesn't really solve the issue. v = 2 is still ambiguous (can be local declaration or assignment to outer scope) depending on the context, and the context is as big as the whole source file. What "forced" declaration says is - I know there is this variable v in the outer scope and I want to shadow it. What I am trying to achieve is when I see v = 2, I could conclude - This is a local var and I don't need to check whether it's already declared in outer scope or not

To give anouther example. Say you have a function somewhere at the bottom of the source file.

fun = ->
    v = 2

Here v is a local var. Fine. Then a few months later a new developer comes in and adds v = 1 at the top of the file. Now, he altered the behavior of fun without even touching it. And this will be hard to debug.

JavaScript copied scoping rules from Scheme but imho it got the defaults wrong. var should have been implicit and assignment more explicit.

I realize that proposed foo := 42 syntax might look old fashioned. I don't insist on this particular one. It's just the most concise form I could think of.

[goj]

Yes, you understood me correctly.

AFAIK there is no way to shadow variables in CoffeeScript, which is, as you explained is a big problem.

Using var was a proposal to allow that in backward-compiatible way. This is much better than only solution that we have today ("be careful with naming your variables") but it results in almost copying the problem JavaScript have today ("always remember to put vars in your code or you'll be in trouble").

The more I think about your proposal the more I like it. It really solves the problem and gives us more consistency with JavaScript.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version and this really seems worth it.

Speaking of syntax, := stands out a little, but it should stand out when we change something from outer scope, so that's more than OK. The other way to go would be to use some sigil, like &, which would play nice with destructuring assignment:

[&im_from_outer_scope, declare_me] = stuff

Other way to word akva's proposal that would probably make Stan happier is: you assign to local variable using = (as you do now) but if you want to change something from the outer scope, then you have be explicit (use :=). This makes both code easier to read and bugs harder to introduce.

[satyr]

Note that [we do have a way](http://satyr.github.com/cup/#foo%20=%2042%0A((foo%29%20-%3E%20foo%20=%2043%29(%29%0Aputs%20foo) (+1 by the way) to shadow outer variables.

See also: #238

[akva]

Speaking of syntax, := stands out a little, but it should stand out when we change something from outer scope, so that's more than OK.

Exactly. Destructive assignment is a dangerous operation and should stand out as a warning. And, to repeat myself, programs written in functional style won't have too many :='s. If one finds oneself using too many :='s it's probably a good sign that code needs refactoring.

you assign to local variable using =

Don't want to sound pedantic but just to make sure we're on the same page. By assignment I mean destructive assignment, or rebinding the earlier declared variable. In this sence you use := to assing/rebind local var too. That is:

local = initialize()
if something()
    local := new_val()

@satyr

Note that we do have a way to shadow outer variables.

Yes, but shadowing is not the problem. I've been thinking about the best way to summarize my issue and I think it can be summarized into the following two problems:

1. Delocalization (I am just making up the term) Every time you indroduce a new variable in scope A you have to make sure the variable name is not already used in any of the nested functions. Otherwise you'll break it.
2. Ambiguity var = val is ambigues. To understand what it means you have to inspect all outer scopes.

It is a huge backward incompatibility, but hey, CoffeeScript is before 1.0 version and this really seems worth it.

Totally agree. It took Python 3.0 to get the scoping right - http://www.python.org/dev/peps/pep-3104/ (though they had a different problem to start with). CoffeeScript has a chance to get it right from the very beginning. Especially considering that it's not hard to implement, since JavaScript already gets the scoping (almost) right.

[jashkenas]

Sorry, folks, but I'm afraid I disagree completely with this line of reasoning -- let me explain why:

Making assignment and declaration two different "things" is a huge mistake. It leads to the unexpected global problem in JavaScript, makes your code more verbose, is a huge source of confusion for beginners who don't understand well what the difference is, and is completely unnecessary in a language. As an existence proof, Ruby gets along just fine without it.

However, if you're not used to having a language without declarations, it seems scary, for the reasons outlined above: "what if someone uses my variable at the top of the file?". In reality, it's not a problem. Only the local variables in the current file can possibly be in scope, and well-factored code has very few variables in the top-level scope -- and they're all things like namespaces and class names, nothing that risks a clash.

And if they do clash, shadowing the variable is the wrong answer. It completely prevents you from making use of the original value for the remainder of the current scope. Shadowing doesn't fit well in languages with closures-by-default ... if you've closed over that variable, then you should always be able to refer to it.

The real solution to this is to keep your top-level scopes clean, and be aware of what's in your lexical scope. If you're creating a variable that's actually a different thing, you should give it a different name.

Closing as a wontfix, but this conversation is good to have on the record.

[akva]

I was going to write a long response, but instead decided to replace it with a short question. Here

foo = 42
(-> for foo in [1..2] then)()
alert foo # 42

loop variable foo (which is just a local variable?) shadows the top-level foo, which is inconsistent with the rest of the language. How do you explain this inconsistency?

[jashkenas]

I'm not saying that shadowing isn't technically possible -- just that it's bad style, and CoffeeScript shouldn't make it easy for you to accomplish. It would be far better to use a different name for that index.

[akva]

I totally agree with you that shadowing is a bad style. And instead of shadowing it's indeed better to just choose a different name.

But I am not talking about deliberate shadowing. I am talking about situations when you are not aware that top-level variable exists and assign it accidentally, or when top-level variable is added later. Check this example from my comment. The body of fibonacci function is supposed to be a black-box, but it's not. And what's worse, even unit test won't spot it - when run in isolation fibonacci function will work just fine, but will break in the context of the whole module. In other words the current scoping rules break encapsulation. This example is particularly illustrative since cache is such a common name and can easily clash.

[satyr]

 foo  = 'foo'
 bar  = 'bar'
 func = ->
   foo   = 42  # declares local var. shadows global
   bar  := 43  # assigns global var
   fooo := 44  # compile error

Adopted this strategy on my coco branch and am quite happy with the decision.

[akva]

@satyr Looks great. Would you like to share some more details about your experience of using it?

Honestly, I reckon Ruby scoping rules are just wrong. It seems that they may change in Ruby 2.0

[satyr]

Would you like to share some more details about your experience of using it?

During the change among CS sources, I've found:

• only 10 instances of = that needed conversion to :=.
• a place where a variable on an upper scope was accidently modified.

[akva]

@satyr Thanks

only 10 instances of = that needed conversion to :=

That's about what I expected.

a place where a variable on an upper scope was accidently modified.

It's true that these bugs are quite rare BUT they are extremely hard to find. Even unit tests won't help. I rather deal with the bugs that happen all the time but easy to find than with those that are quietly sitting around as a ticking bomb waiting to explode.

[ggRalf]

I discovered the scoping problem on my own. After a small chat i was pointed to this thread. And finally I looked at the coco branch. Looks really nice! Since this language is really young I will switch to the coco branch and hope it will be maintained. Thanks for the really nice language and syntax!

[DmitrySoshnikov]

So I've reached this thread also. In fact, it's very arguable whether the shadowing is a bad style. Behind and before this term "shadowing" stands the basic term of programming -- an abstraction.

A helper procedure should be really a black box. It's the main principle of substitution the arguments for a formal parameters even in math functions. The same stands for the local variables.

The abstraction barrier which separates the level of implementation of a procedure from the level of usage of the procedure should be the real barrier in a well-designed system. Which means -- it should not bother the level of usage with exact variable names used inside.

A designer of the procedure also shouldn't worry about which variables to use as just helper local variables. In general, it can be a casual case -- to reuse a some 3rd-party procedure in own project.

Exactly for this concept of a scope and in particular nested scopes (namespaces, modules, classes, etc) is invented.

That the Ruby have chosen to declare local vars without any keyword and refer these variables as outer from closures (lambdas, blocks, procs) is just the exact and the particular case -- and actually very arguable implementation.

The mentioned reason ("it's a completely lexical scope") in fact just breaks the concept of nested scopes. And moreover, it smells like a substitution of concepts. The lexical scoping is one when it's possible at parsing stage to determine in which scope a variable will be resolved in runtime. And this determination is made by the place of the variable's definition. I.e. we may have several nested definitions with the same name and it still will be the lexical scope.

If you don't like a special keyword for the definition, you may consider instead Python's way (which was mentioned also above in this thread). Though, less ugly keyword than nonlocal can be used. E.g. outer:

a = 10
b = 20

foo = ->
  outer a = 30
  b = 40

alert a, b # 30, 20

It requires to capture manually needed closured vars thought.

Or indeed, maybe nevertheless to return the definition keyword but not in the definition semantics but in semantics of localizing the scope. E.g. let. It's really like in math. First you say, that x is 10, but later, you decide that for this function, "let x be a string.

x = 10

foo = ->
  let x = "test"
  y = 30 # Syntax Error (undefined global/local var)

foo()

alert x # still 10

Which semantically, repeat, sounds even not as "define a variable", but as "assume for this scope name x to be with this value". Thus, in global scope a var can be created without keyword.

Or e.g. Lua's local keyword can be used also. That you can in general determine whether the var was already declared, right? However, it's hard with eval.

Anyway, current breaking of abstraction -- that is, breaking the black-box with all it's "offal" which belong to this black-box (and belong by the right) is very arguable.

Repeat, a 3rd-party programmer should be able to use any names of local (to the black-box) variables. And at the same time another 3rd-party programmer should be able to reuse this function written by the first programmer -- and without reviewing the code of the function. It's the main principle of the abstraction which seems just broken in Coffee.

Notice, that in Ruby (in contrast with Coffee) the picture is a bit different. There most frequently a user works with methods (defs) which doesn't capture (by default) local variables of the surrounding context. That is, if in the same example that 3rd-party programmer writes his method, he calm about writing a = 10 without thinking whether this name is already (or will be in unknown in advance environment -- that is worth) borrowed. And it's (probably completely) another case (from which you borrowed the design) are closures (lambdas, procs, blocks, etc) -- with they already a local user usually works and he usually knows which local vars he defined and that they will be captured. Though, in general, this also is not guaranteed that he remembers that -- in the same respect he can make a mistaking thinking that he uses a local var, but indeed he just forgot that has already defined it above).

Once again, Ruby's semantics in this respect is not the same as in JS/Coffee -- in JS all functions are closures (so the complete port of the semantics cannot be used as a best argument in explanations). And moreover, Ruby's design is not perfect in this question; consider it.

So Lua's way with local or proposed ES6 let or even Python's with proposed mine outer (i.e. even if we should "mark" needed vars "to be closured") seems more bugs- and fool-proof and without breaking the abstraction principle.

Dmitry.

[odf]

It may help (or, as it were, end) this discussion to note that coffeescript now has the do construct. a = 2 do (a) -> a = 1 console.log a console.log a prints 1 2

[DmitrySoshnikov]

odf how will it help to restore breaking principle of an abstraction? Should all programmers starts their functions with do?

So I still propose to consider Python's way but with outer keyword. Or Lua's / ES6 way -- with let keyword.

Dmitry.

[odf]

Well, I'm hoping that eventually the behaviour of do will be fixed in such a way that we can write a = 2 do (a = 1) -> console.log a console.log a in the example above. Have a look at issue #960 for some discussion on do. I would have preferred let as the keyword, but it was decided to use one that's already a reserved word.

[DmitrySoshnikov]

Yes, I'm ware about what do instruction does in Coffee. Actually it's analog of the let, yes (starting the semantics even from Scheme and desugars into immediately invoked function).

However, it's not the generic case. I mean, if you suggest to start every function with that do, I am sure it will be very annoying.

One more time. Currently Coffee isn't even consistent in chosen strategy. On one hand it says -- "no name shadowing" (i.e. there are no two frames in the environment with the same name binding), which by itself, as I wrote above, already breaks the main principle of the abstraction and nested scopes. On the other hand, it contradicts to the first chosen way, and nevertheless allows two frames with the same name bindings -- it's achieved via formal parameter names or with the same do.

And one more time. Arguing that "this is like in Ruby" isn't completely correct. Since repeat again -- in Ruby methods doesn't capture local vars of the surrounding context and create local vars via assignment (that's for the example with two 3rd-party programmers which share the code -- in Ruby and in contrast with Coffee they can do this safely). And Ruby closures (blocks, lambdas, procs, etc) do captures vars, but with closures usually already the user himself works and know his vars (though, again repeat, even this is not safe in Ruby and can be considered like a design flow).

In JavaScript as you know, all functions are closures. So the case with two 3rd-party programmers fails breaking the main principle of an abstraction.

Dmitry.

P.S.:

Exact proposals:

a = 10 
b = 20
c = 30
d = 40

foo = (a) ->

  outer b, c

  a = 100
  b = 200
  c = 300
  d = 400

foo()

console.log a, b, c, d # 10, 200, 300, 40

That is, a is local since it's a formal parameter, d is local since it's not marked as outer. OTOH, outer b and c are modified.

Another way:

a = 10 
b = 20
c = 30
d = 40

foo = (a) ->

  local b, c

  a = 100
  b = 200
  c = 300
  d = 400

foo()

console.log a, b, c, d # 10, 20, 30, 400

That is, only d was outer.

[odf]

I don't see the point. Why would you have all those variables on the file level when the functions you're exporting are not supposed to use them? If you don't want those bindings to be visible everywhere within your file, don't put them there.

In general, if you don't pollute your scopes with unnecessary bindings in the first place, you'll have no problems with broken abstraction. I think the way to look at this is to consider the context in which a function is defined as a genuine part of it, not just an environment it was thrown into by accident.

[DmitrySoshnikov]

First of all -- do you agree that Coffee isn't even consistent in its chosen way? That is, "no two bindings with the same name in the environment chain" vs. "allow nevertheless two bindings with the same name via formal parameter names and do"?

If "yes", what the difference do you see from defining a local variable via formal parameter name and just a local variable with the same name?

Why would you have all those variables on the file level when the functions you're exporting are not supposed to use them?

OK, let's take a simple example. We (you and me) work together and should support the same source.

I wrote a helper function somewhere above:

square = (x) -> x * x

You three month ago write your function 1000 lines below:

createWidget = ->
  square = new Square 10
  square.onResize = (e) -> console.log e
  square

You just used a local variable name, probably you didn't even know about my square function since that block of code was in my responsibility. What will be with my code execution then? How will we find the bugs?

We'll of course find the bug sooner or later (and moreover since we in one project, you can argue that you should know all the source and all the used identifiers above. By the way, why "should" you if to consider the principle of separation programmer responsibilities?).

But we can complicate the example, when I e.g. may reuse (in the simplest way just to copy-paste) some peace of code from completely another project to mine. Should I review all the "imported" sources to find out used variable names? If yes -- why "yes" since it's a direct breaking of the abstraction?

Dmitry.

[odf]

I don't find your example convincing, at all. If I were unaware of which functions you defined on the file level, what would stop me from simply re-using the name square in the same scope?

[DmitrySoshnikov]

Seems you just ignored my questions I wanted you to answer before your reasoning about the scope theory. Well, OK. Let's assume that you just agree with them.

For details, look at any general scope theory paper (and especially on environment frames concept) -- it will help us in discussion not to mix definitions and in particular the definition of the "same scope". E.g. http://bit.ly/esnkD6

So if you reuse square in the same scope as mine (and by this I mean the definition of the same frame of the environment), then you make an error. So, substitution and mixing of definitions is irrelevant here. We talk about different frames (different scopes), not the same -- and from this viewpoint you are able to use any identifier (and this action is even available in Coffee -- repeat via formal parameter names or via do).

However, if you use the same name in your own scope, it's completely your right as the author of this encapsulated abstraction.

Dmitry.

[DmitrySoshnikov]

Just a small note. OTOH, e.g. Erlang warnings about shadowed variable:

X = 10,

Foo = fun(X) -> X + 1 end, % warning X is shadowed

Foo(20)

It's for that the shadowing can be dangerous. However, in contrast with Coffee/Ruby, Erlang's variables are immutable and just pattern matched (i.e. you can't assign to X inside the Foo function).

Dmitry.

[cairesvs]

You just used a local variable name, probably you didn't even know about my square function since that block of code was in my responsibility. What will be with my code execution then? How will we find the bugs?

Hi Dmitry, I agree with almost everything you said it. But some problems/bugs you'll find testing your code, doing TDD or something like that. Of course when you do your tests you don't think all the possible cases however you can coverage good part of your code.

[DmitrySoshnikov]

@cairesvs yes, I'm aware about TDD. Though, the question was specially to underline the design flow (it wasn't a real asking how we should find the bugs ;) But, thanks anyway for mentioning.

Dmitry.

[cairesvs]

@DmitrySoshnikov Yes, I understand the problem. Just bring another perspective. To me many of the problems and arguments you bring to the table could be solved with simple TDD and TDD could change the design of your project, help to find other ways to approach the same problem. But, like I said it before, I agree with you with could be more easy to understand the code and make less bugs/problems if there is something like local/outer or let keyword. The solution brought by @akva and @satyr is similar to yours and good too.

[StanAngeloff]

This proposal talks about assignments, but how about accessing a variable:

x  = 10
fn = -> x
console.log fn()  # RefErr or 10?

Doesn't make sense to use outer to shadow global vars on assignment, yet still being able to access them without the modifier.

<?php
$x = 10;
function fn() {
  print $x;
}
print fn();  # Undefined $x

...and on that note, given how JavaScript scoping works, it would be insane to even try and implement non-descending scope.

[DmitrySoshnikov]

@cairesvs

Yep, right, TDD may help to catch some bugs. However, the way "to use our language, you should program in TDD style or ... you've been warned - catch your bugs yourself, since our language may easily provide them " cannot be considered as the best way I guess ;)

@StanAngeloff

Nope, in this case x should be normally resolved in the global frame. Keyword global makes sense only when exactly an assignment is presented in the code. In this case it just won't add var in the generated code.

Regarding PHP example (you've added it later) -- in PHP casual functions aren't closures (the same as in Ruby methods aren't closures). So there we should use global keyword. In JS as said, all functions are closures with chained frames in the environment. So a free identifier should be resolved normally in outer frames.

Dmitry.

[StanAngeloff]

2¢ Yeah, I don't see how it makes sense to require outer for shadowing, but keep everything else working as-is.

[DmitrySoshnikov]

@StanAngeloff

Approach with outer is not for shadowing, but vice-versa to open the door for outer frames, since the assignment always creates a local variable. If the variable instead were marked as outer, then in the generated code no var statement is added for it.

In contrast, the approach with local or let is already for shadowing. It vice-versa for outer does generate var keyword for a variable. And assignment just works as assignment -- if the binding exists in the own frame -- it assigns to it. In other case -- it continues the lookup of the identifier in parent frames (if found, then assign, if not -- ReferenceError).

Dmitry.

[StanAngeloff]

That's what I meant by shadowing, i.e., it's explicit.

Not to repeat myself, but I'd be against such a change.

[DmitrySoshnikov]

@StanAngeloff

Please explain then your meaning about issues I described above. What can you suggest to solve these issues? You said you're against the proposal, but you don't mention any word about solving the existing issue.

Dmitry.

[StanAngeloff]

Dmitry, I don't find the current implementation to have any issues. I don't look at it from the point of view you have developed. Therefore I can't explain or suggest anything as to me it all makes sense.

Repeat, a 3rd-party programmer should be able to use any names of local (to the black-box) variables. And at the same time another 3rd-party programmer should be able to reuse this function written by the first programmer -- and without reviewing the code of the function. It's the main principle of the abstraction which seems just broken in Coffee.

Keeping all your modules separated and compiling them individually, you'll never run in the above. Coffee's scoping rules are file-based (as you probably know, no doubt). Stuffing a lot of code in one file and accidentally breaking the black-box by overwriting a global is most likely intentional.

Anyway, it's how I see things. While you, satyr and akva, etc. have the same view, I am entitled to have my own as well ☻

[cairesvs]

@DmitrySoshnikov

I think I did not express myself well. Ins't about bugs, is about find the best design, so you can define the correct scope for each part of your project. When you have problems like this on some project is about bad design or not? I think it is and it seems to me the real problem is on the way the language was designed.

[cairesvs]

@StanAngeloff

I also don't think it is an implementation issue however it seems to me that it would be a natural improvement of the language.

[DmitrySoshnikov]

@StanAngeloff

Stuffing a lot of code in one file

It doesn't matter what is "a lot of code" means in your/my view. This use case can be completely real and in the small code. Then direct question -- should I before using any local variable check the all code in the file you wrote? Don't take a huge file, let it be 100-300 lines. Should I?

@cairesvs yes I see your point and also think that the language should be designed so that even no unit tests are needed to program without bugs. Though, repeat, TDD is a good addition to avoid bugs regardless the exact language.

Dmitry.

[TrevorBurnham]

I'm with Stan on this one; the status quo feels more intuitive to me than the addition of an operator or keyword to make explicit whether you're using a local-scoped or outer-scoped variable. The ethos of CoffeeScript, as I understand it, is:

1. Shadowing is bad—a variable name should only mean one thing within a file. True globals (those that are shared among files) should be attached to window or global.
2. Projects should be broken up into modular files, each wrapped in a closure to prevent scope leaks. A file should be perhaps 200 lines, tops.

Now, could the language be more consistent with these principles? Sure. Shadowing within a file could be eliminated entirely by having the compiler emit a warning, at the least, when a function argument (or, worse, issue 1121-type order issue) causes an outer variable (other than true globals) to be shadowed.

But for the most part, CoffeeScript the language seems well-aligned with its ethos. As long as you use a variable name to mean just one thing within a file, automatic scoping feels to me like the best possible system.

Also, I'm curious how the outer system would work in this situation:

do ->
  outer x = 1
  x = 2

Would that be a compiler error? Would it be inferred that both x's refer to the outer x? Or would the compiler change the name of the local x so that the var declaration wouldn't interfere with the outer x assignment?

[satyr]

Projects should be broken up into modular files

Why then does it provide --join option?

[DmitrySoshnikov]

@TrevorBurnham

Yes, I see the goal and the initial wishes of the chosen principles. And initially they really can be considered as improvements.

However, if you listen to written by you above principles, you'll see that they instead of convenience can sound just like limitations. In other words you say:

1. There are only global (per module) variables. Your code will be issued with a warning in case of shadowing.
2. CoffeeScript isn't good enough for writing more-less complex structures in one file. Even if your file will contain 3-5 procedures, they better should be small (and it's really better by the way) or else you risk to mess with variables. But in reality there can be really complex procedures which needs severl/many local variables.
3. CoffeeScript is basically for programming for only one programmer in the project. In case you have several other programmers, they all should first find out which variable names are already borrowed.

Again -- yes, I agree that the basic wish to make the variable definition syntactically elegant (i.e. without any keyword) is a good wish. However, since JS/Coffee has model of chained environment frames there should be the way to distinguish free variables (that is variables which are not in the own frame) from the local variables.

Currently it's not possible. I still don't understand why in arguing you always mention only one programmer which works with the code. Why do you avoid another programmers which may support the code working in the same project.

And by looking on the following code:

foo = ->
  x = 10
  y = 20

it's not possible to say anything about x and y identifiers -- i.e. whether they are local or not. The programmer should return back and scan all the source first (okey, to use automatic search).

So I also want to make it more convenient and like the initial idea and principles, but at the same time I see the described above issue which breaks the principle of abstraction (though, the ideology "one name per file" already breaks it and seems expressly -- excluding nested scope concept).

I can assume that I can program in the current Coffee's implementation (regarding var definitions/mutations), but at the same time I want to patch the holes I see.

Dmitry.

[StanAngeloff]

I am sorry, I just find this amusing:

CoffeeScript isn't good enough for writing more-less complex structures in one file

Seriously?

it's not possible to say anything about x and y identifiers -- i.e. whether they are local or not.

I can achieve this using comments, without introducing any new keywords to the language:

# Foo does boo.
#
# @globals x, y
# @see my_app.coffee
foo = ->
  x = 10
  y = 20

I can assume that I can program in the current Coffee's implementation (regarding var definitions/mutations), but at the same time I want to patch the holes I see.

Great! You should definitely fork the project, add your patch and then send through a pull request. We can then actually have an implementation to look at and maybe, just maybe, we have a change or heart.

[DmitrySoshnikov]

@StanAngeloff

Seriously?

I though it should went without saying that I rephrased from the other viewpoint your words. I don't want to turn the discussion into the demagogy. Moreover, I think this discussion becomes already noisy and less technical. But on your "seriously" -- I have a recent project on Coffee and manage it normally. It's not about my own inconvenience, it's about analyzing the design of the language.

I can achieve this using comments

You mean "I created a problem and then search the way how to fix this problem (using comments or something)".

I.e. just right after you start to find justifications of something, you already accept that there is an issue. Meanwhile in a well-design system there should be no such issues at all -- and no need to find justifications for anything.

You should definitely fork the project, add your patch

Yeah, maybe, will see.

and maybe, just maybe

Oh, it's a honor, I appreciated, thanks ;)

P.S.: repeat, I can program in current Coffee's way with variable definitions/assignments, I can find the way how to manage it efficiently (the way with comments also good). But what I do -- is analyze the design and mention the issues I see. It's not about me and my convenience, but the wish to avoid possible problems.

Dmitry.

[jashkenas]

Despite this issue's tendency to break out in flames -- strict lexical scoping is very much a core principle of CoffeeScript, and it's a great issue worth discussing.

Dmitry raises a couple of specific criticisms: First, that the lack of shadowed variables "breaks the principle of abstraction", because 3rd party code can't be blindly copied-and-pasted into the middle of your source. Second, that CoffeeScript is inconsistent, because function parameters do give you a way to shadow variables. He then proposes a change: tagging either local variable with a var keyword, or closed-over variables with an outer keyword, to distinguish between the two.

I'd like to persuade y'all that strict lexical scope is a defining feature of CoffeeScript -- a massive improvement over the manual var-tagging of variables, and similar in spirit to the notion of structured programming. ... Think of it as "structured variable naming".

We all know that dynamic scope is bad, compared to lexical scope, because it makes it difficult to reason about the value of your variables. With dynamic scope, you can't determine the value of a variable by reading the surrounding source code, because the value depends entirely on the environment at the time the function is called. If variable shadowing is allowed and encouraged, you can't determine the value of a variable without tracking backwards in the source to the closest var variable, because the exact same identifier for a local variable can have completely different values in adjacent scopes. In all cases, when you want to shadow a variable, you can accomplish the same thing by simply choosing a more appropriate name. It's much easier to reason about your code if a local variable name has a single value within the entire lexical scope, and shadowing is forbidden.

So it's a very deliberate choice for CoffeeScript to kill two birds with one stone -- simplifying the language by removing the "var" concept, and forbidding shadowed variables as the natural consequence.

This brings us to Dmitry's second point: It's still possible to shadow with parameter names, because of the nature of JS functions. I think that Trevor has the right idea here, we should be more strict about shadowing instead of less. It would be great to entertain tickets that either make parameter shadowing a syntax error, or a compile time warning. If we ever go down the road of having a coffee --warn, it should be one of the first rules.

Finally, the arguments about strict lexical scope making CoffeeScript a one-programmer-per-project language are total baloney, in my opinion. Accidentally clobbering an outer variable in a nested function is certainly possible ... but accidentally shadowing an outer variable is just as likely, and can also break your code. If I try to use a top-level define'd variable, in a nested function, but you've shadowed it, I'm hosed. And having strict lexical scope makes it much easier for me to determine what exactly has gone wrong, as opposed to "hunt for the var".

Strict lexical scope isn't going to change in CoffeeScript proper, but I encourage you add outer to your own dialect, or take a look at Coco, which includes two different kinds of variable assignment.

Re-Closing the ticket.

[odf]

In all cases, when you want to shadow a variable, you can accomplish the same thing by simply choosing a more appropriate name. It's much easier to reason about your code if a local variable name has a single value within the entire lexical scope, and shadowing is forbidden.

But that's just the thing. People make errors all the time, and at present, there's no way for the compiler to tell whether someone accidentally re-used a name from the including scope. I support the idea of issuing compile time warnings on parameter shadowing, but I think it would be even more useful if the programmer could indicate that they would like to use a name locally and assume that it's not taken, so that the compiler could issue a warning if they're wrong.

The do construct could be such a way, and I for one would be perfectly happy if it were the only one. I think outer is a very bad idea, because it would make writing closures extremely clumsy, and the proposed let or := would be just as bad as sprinkling the code with var statements. I like how do confines the new bindings to a well-defined place, but the problem at the moment is that if a name already exists in the including scope, then the compiler silently assigns to that outer variable.

I'll continue this on #960.

[jashkenas]

odf: Sure, in theory. In practice, well structured JavaScript code doesn't litter the top-level scope with lots of global variables, and keeps function scope shallow. This problem literally never comes up. And in the rare cases it does -- it's the same as when you try to reuse a variable name inside of a deep if/else statement -- you think "oh, I need to use a different name for this" ... and do just that.

[satyr]

Join the club--it's much easier to fork it than convince its creator. ;)