Allow underscore in names

[juancarlospaco]

The development world in general is becoming more polarized nowadays..., basically most developers are doing JavaScript or Python, wether we like it or not, Python is at the top of the most popular programming languages for several years, so it means a lot of people using it, and a lot of potential for new Nim users and companies.

For JavaScript the interop is really good and allows modern features via macros, but for Python, Nim has a limitation when it comes to naming, because Python uses names starting with underscores, ...a lot, had some experiences building Python interop, Python has a huge amount of libraries and is embebeded in several places and Nim can re-use all of that with minimal effort.

But sometimes theres no way to re-use all that code in Nim, not even with Stropping (backticks in names), so interop is not possible at all in some cases sadly, but the fix for Nim is actually easy, just allowing names with underscores, with or without backticks in the name but allot Nim symbols starting or ending with underscores.

The change can be opt-in via a flag, if wanted (--experimental:underscoreNames ?, -d:nimPreviewUnderscoreNames ?, etc). It should not break existing Nim code, because is going more relax with the names, instead of more strict. This is for FFI purposes only, and not a vanilla Nim style. Should be easy to test if a given name works.

Besides "style opinions", theres no huge direct concrete technical benefit for Nim in keeping the limitation. In the end the limitation makes Nim lose even more terrain to Python...

Changes

These names should work in Nim:

• __name__
• __name
• _name

See also

• Nimpy and underscores.
• Illegal underscores for names in Nim.
• Nim patch to allow underscores inside backticks.
• JSON that contains "_key" Nim bug.
• Allow underscore at end of identifiers Nim bug.
• More Nim bugs related to underscores.

🙂

[Araq]

You need to explain why {.exportAs: "__PYTHON_STUFF__".} cannot work...

[juancarlospaco]

Because also relates to importing stuff.

Example __functionality__ in Python, imported as __functionality__ in Nim.

See linked issues for more context...

[Araq]

That would be .importpy: "__functionality__" then... And it's nice to keep Python identifiers in Python-land and to offer idiomatic Nim names. Last time I checked Python's ecosystem is vast and yet almost all of its wrappers and libraries are Pythonic. Something we could imitate rather than making Nim an ugly ducktape of semi-broken stuff.

[juancarlospaco]

But I am not talking about if it is doable or not, I think why not do the interoperability with the most popular languages as easy as possible?.

Again I dont see a benefit in keeping it, finish the phrase:

The big direct concrete technical benefit for Nim in keeping the limitation is . . .

[Araq]

... that the resulting Nim code that nevertheless interfaces with Python successfully and conveniently looks like idiomatic Nim.

[dsrw]

I'm a little late, but I wanted to +1 this. I'm wrapping the Ruby C API, and it would be great if existing code/books/tutorials could be used for reference without having to worry about differing identifier names. I'll also be exposing a more idiomatic, higher-level API, but I think there are valid cases where someone might want to import or export an API without renaming anything.

It's not a huge issue, but if there's nothing technical preventing it I think it would be a small but worthwhile improvement.

[juancarlospaco]

The problem is when you need to wrap something like self.__foo__.__bar__(arg, arg), in that case doing the .importpy: for all the chain is hard and ugly for no gain, and sometimes self.foo.bar(arg, arg) also exists too making it more complicated, and it is also more Bug prone.

[derekdai]

It also can ease document look up.

[derekdai]

In the godot engine, the method that can be overridden will use _ as the prefix, but in fact, this rule is not used in the entire project. It is okay to increase the difficulty of binding development, but the users of binding are even more affected. When to use pragma to change the exposing method name is confusing.

[Araq]

I'm a little late, but I wanted to +1 this. I'm wrapping the Ruby C API, and it would be great if existing code/books/tutorials could be used for reference without having to worry about differing identifier names.

That can never be done anyway since the set of reserved identifiers differs. It's better to document how the mapping was done and leave it at that, otherwise the ugly C names always stick around and there's two names for a single entity, which is even more "confusing".

It's not a huge issue, but if there's nothing technical preventing it I think it would be a small but worthwhile improvement.

Well but it's just a "small improvement" that doesn't really work...

[juancarlospaco]

looks like idiomatic Nim.

So... "style opinions" basically. :+1:

[Araq]

Well you also didn't outline how these trailing/leading underscores effect the identifier equality rules, it's more rules to remember and worst of all: It's ugly. https://youtu.be/8v5CuyuS5wg?t=27 ;-)

The underscore is a separator but if used at the beginning or end there is nothing there to separate.

[juancarlospaco]

But we are talking about allowing them between backticks for FFI. 🙂

[Araq]

That's not what your RFC says...

[juancarlospaco]

I mean if it is not possible to have the names "as-is", because the reasons you mention, ...then at least between backticks, should be possible I guess ?.

[Araq]

It's quite easily possible to change the language and to allow for more underscores. That's not why I'm against the feature. I'm against it because it's ugly. However, as a compromise if you change the language so that __a__ is allowed but identical to a (except that it's kept around as __a__ for introspection purposes) then I wouldn't mind it too much. I guess.

But the case must be argued convincingly, "Python interop project X does not use custom pragmas" is not convincing.

[juancarlospaco]

I'm against it because it's ugly. I wouldn't mind it too much.

🤷

Between backticks at least is Ok, and see the linked issues about the reasons by multiple people on multiple projects.

[Araq]

Well this is supposed to be an RFC, "see the issues linked elsewhere" doesn't suffice and the subset of the issues that I looked at didn't convince me.

[juancarlospaco]

I do not understand, now some people want to remove/deprecate the case insensitivity completely, but still underscores in names are not allowed... 🤔