Proposal: Allow `in`, when container is an object

[axgkl]

Hi, quick one, just for copy and paste.

It fixes a crash when you do "foo" in obj and obj is an instance of a TS class:

--- a/transcrypt/modules/org/transcrypt/__javascript__/__builtin__.mod.js
+++ b/transcrypt/modules/org/transcrypt/__javascript__/__builtin__.mod.js
@@ -130,7 +130,9 @@
                        return container.keys () .indexOf (element) > -1;
                }
                else {
-                       return container.indexOf (element) > -1;
+                       return container.indexOf ?
+                                 container.indexOf (element) > -1
+                               : element in container
                }
        }
        __all__.__in__ = __in__;

[axgkl]

btw: don't know about compat but in chrome the plain js in does work on arrays as well, so I think return element in container could be also the default after the is TS dict checking:

sorry the first array example was bull (one of these js fubars Transcrypt resolves, 1 in ['a', 'b', 'c'] is also true). It happens when you program too long in Transcrypt that you begin to think js was sane :-)

[axgkl]

Hi, in https://github.com/JdeH/Transcrypt/pull/88#issuecomment-238024930 you comment (as always correctly) that obj1 in obj2 is not supported in CPython so it should fail in TS as well.

While I understand the rational argument (changed the subject, this is not a bug), intuitively it feels wrong to me and I try to explain.

Two Arguments why __in__ (and equality) should work on objects (not arrays) like in Javascript

For arrays I think we are all in line: Nobody sane ever wants to program with 0 in ["foo"] => true and "foo" in ["foo"] => false.

But why do I think that in for TS objects should be working like in js - i.e. like hasattr ?

1. TS won't ever be able to really mimic CPythons's hash based in operator anyway - and that applies to general equality as well

It would be a huge bloat to create hashes for each object just to be able to check those for __in__ - or do the tricks of CPython based on those to have {'a': 1, 'b': 2} == {'b': 2, 'a': 1} = True.

Imho we can and should pretty much forget 100% CPython compliance of equality and containment as a goal.

and

2. I think this is a good thing

Because imho the biggest asset of TS is the absolute 'borderless' interplay with js frameworks.

I programmed for a week on a 'real' app in Transcrypt, using bigger js based frameworks like kendo, redux, kefir - and I got into a mode where I sort of program, combing the best of both worlds: Thinking the python way about the problems & a subjective 10times faster understanding of what a function does when re-reading it, and all the code organization features. And that combined with full, native access to javascript. 'Access' is actually the wrong word - its more: You do program 'real' javascrypt - but in a Python syntax.

That pythonic use of native javascript libs and objects combined with my own app's modules, classes and objects is imho the biggest asset of Transcrypt - and not the potential isomorphic use of the exact same code on python server and js client (like I thought in the beginning - now I think if you want that use sth else than TS). TS does, e.g., a lot so that its powerful dicts with all the python methods are accepted by those js frameworks as if they would be javascript { } objects, so I thought this interplay is really one of the major design goals.

I got into a mode where the only constraint to use javascript "stuff" in TS is: Must be parseable by the AST parser, then it'll work like I expect. Thats why I understand the __new__ function has to be in - but for the rest I most of the time have sort of "superset", a "best of" (regarding expressive simplicity) of Python and Javascript at my disposal, using all those JS libs...

And in that sense letting 'foo' in bar crash and that just for the purpose that CPython crashes as well does feel unnatural and unexpected to me.

In fact, funnily, I realize that after only one week of Transcrypt, I considered it a bug first, forgot completely about Python's hash based implementation ;-) And think about potential users of Transcrypt we want to win over from the javascript camp, potentially, they would also not like a crash just for the reason that CPython crashes.

I think we really should not try to let code which works in js crash soleley for the reason of a python crash. foo = False; m = dict(foo) does not crash as well (allthough in CPython it does) and I use stuff like this meanwhile a lot, e.g. when I want a TS dict from an argument of a function which is called from plain js as well as from TS.

Risks?

No risks, imho:

The reader

No reader of a working TS module, trying to understand what it does, would see a 'foo' in bar (with all the python use of native javascript libs next to that) and conclude: Ahh, bar is a Transcrypt object, so the author intended to provoke a container.indexOf is not a function exception.

The writer

I think also no risk to cause wtfs here - like: damn, I wanted to cause the code to except by using in, it should fail but it doesn't. He has other ways to cause an exception, better ways, than using a forbidden in on an object.

---

That was my very personal intuitive argumentation why I would love to have in at hand for objects, just like I have the other full world of javascript at hand, especially since I see no risks.

I'm I completely missing the goal of TS here ?

[daboross]

I would also appreciate this feature, for mostly one reason only: there isn't anything like Python's dict in JavaScript besides objects - when working with JavaScript libraries which return objects to be treated as dictionaries, it seems a bit over the top to require wrapping every result in a new dict before being able to use in on it.

[JdeH]

Hi Gunter

I am trying to follow you argumentation as closely as possible.

Need some help there, so I go by it point by point. If you think I misunderstand a point, please clarify.

You state:

1. TS won't ever be able to really mimic CPythons's hash based in operator anyway - and that applies to general equality as well

I don't understand what exactly you're getting at here. Functionally Transcrypt's 'in' operator should work as much as possible as CPythons one. That it isn't based on hashes has no functional consequences, only performance consequences. So I would rather say: Transcrypt's 'in' operator should approach CPythons 'in' operator as closely as possible without incurring bloat, which I think currently is the case.

You state:

2. I think this is a good thing

Because imho the biggest asset of TS is the absolute 'borderless' interplay with js frameworks

That's perfectly true, seamless integration with js frameworks is one of the main design goals of Transcrypt. But not having an 'in' operator for objects doesn't in any way impede that. A JS lib calling 'in' on a Python object will have no problem, since the Python object is also a JS object and the 'in' will just work. And a Python programmer using a JS library will program 'hasattr' by nature.

You state:

And in that sense letting 'foo' in bar crash and that just for the purpose that CPython crashes as well does feel unnatural and unexpected to me

And think about potential users of Transcrypt we want to win over from the javascript camp, potentially, they would also not like a crash just for the reason that CPython crashes

This seems to be the core of your argumentation, am I right?

You also state:

That pythonic use of native javascript libs and objects combined with my own app's modules, classes and objects is imho the biggest asset of Transcrypt - and not the potential isomorphic use of the exact same code on python server and js client

While 'isomorphic' use isn't the main point of Transcrypt it is an important point. I'd like Transcrypt users to learn Python, and be able to user their skills also in CPython. I'd like children learning to program with Transcrypt, to become CPython programmers as much as possible and not unnecessarily teach them JS habits. So indeed I'd like in

[axgkl]

Hi Jacques and Dabo,

whew, so I may decide (missing the sweat smiley... )

Big honor, but I think I can't.

Why? Because of this:

While 'isomorphic' use isn't the main point of Transcrypt it is an important point.
I'd like Transcrypt users to learn Python, and be able to user their skills also in CPython.
I'd like children learning to program with Transcrypt, to become CPython programmers as
much as possible and not unnecessarily teach them JS habits.
So indeed I'd like <attribute> in <object> to fail.

and now I remember from your docs how important that point is for you. Now - YOU created this and it is YOU solely, who should decide about its reason to exist. And with this one we have a conflict then and people who sort of 'misuse' Transcrypt then must simply accept it, depending on how important we are to you, in contrast to your main audience.

The Conflict

The conflict is between those who want a Transcrypt for building professional, serious applications and those who want to learn Python and "become CPython programmers". The somewhat funny thing, imho, is that I THINK that the thing you built, is, at least in the current state, better, far better suited for the first group - while you seem to intend it for the second group :-)

(btw: did you really design it for that purpose? I thought you designed it like you did because you had to build something "serious", professional for a customer yourself and wanted, like we all a "pythonic" way to write code for the browser in a real scenario, i.e. with acceptable download times, maintainability, debuggabilty - where in summary transcrypt leaves the existing options behind)

Now - I'm in the first group - and Dabo pretty much managed to compress into one sentence all I wrote. Its all about intergrated work with javascript libs and own pythonic Transcrypt code.

---

I try to make the case now for that first group:

If you build something really serious then we all know there is no way around using the jquery, angulars, reacts, riot, redux, kendo, rx, whatnot (...) libs of this world, alone for the DOM manipulation but also stream processing and many others.

And here is the point: If you do that sufficiently long, integrate, call Transcrypt functions from js and js from Transcrypt functions, debug in the console... in your head (at least in mine) emerges a different perception of what you think you are doing:

After a while, you don't think that you program 'real' Python - and magically it runs in the Browser. But its more like: You program Javascript, but like you always wanted it: Pythonic. Superefficient. Maintainable. Real Mulitinheritable, overloading of core code in Projects.... And with all the fubars removed (too many fubars to even wanting to give examples).

I guess this perception begins to evolve after staring that many hours on the browser console next to your pythonic code...

I love Transcrypt, because it is the right design to get these... string blobs of not inheritable, hard to understand javascript out of my company's product for example. I do think that after completing a larger project (which I'll put into production at a Tier one Telco in Europe btw), I'll be able to convert my frontend people with it. People who can do a lot with javascript - but who up to now where not really able to deliver modules with nearly that high reusability factor than my backend people with Python. Also Paul Irish' involvement told me that Transcrypt IS interesting to JS people. Check this, you got a pretty decent review: https://github.com/jashkenas/coffeescript/wiki/list-of-languages-that-compile-to-js, the list tells me that no, Javascript people are not opposed to getting more effective ways of expressing themselves.

And finally: Not sure (because thats not my problem) but I think for people who want to learn CPython and their carrot to motivate them is that they can see what they did in the browser, I think for those Transcrypt's design is not even that well suited, than e.g. pyjs, brython, rapyd. Because your "bloat free" design will always mean that there are fundamental differences, stuff which works here but not there. And I very much hope that bloat freeness remains more important to you in the future than compliance.

Getting me to that hash argument.

In CPython they manage to get things like

[1,2] in [[1,2], 3]  == True; (1,'a') in {(1,'a'): 3} == True; (1,(1,('a',))) in [(1,(1,('a',))), 3] == True

working, by going back to the hashes of the imutables (ints, strings, bools, floats) instead of simple ref checks, like in js, where [[1], 2].indexOf([1]) is false, sorry, -1, but m = {'a': 1}; [m,1].indexOf(m) == 0

Transcrypt's in will (hopefully) remain to "be" javascript's in (with the indexOf constructions needed there all the time "efficiency resolved"). And when you work long enough in Transcript, thinking you program actually Javascript but pythonic, then I would say in should also be true for objects.

And that is what Dabo meant as well I think, coming from a technically different angle - but in the end it boils down to the same argument.

---

Summary

• I want to give an opinion but not decide it, since I don't feel I'm matching your target primary audience.
• You convinced me that from your positioning of Transcrypt there is a Risk and the risk is: A student could conclude that in would also work in CPython, since it would work in TS.
• I gave arguments why there will be (hopefully, because of the bloat which would be inevitable) always a lot of things, especially related to in but also equality checks, that won't work the same, in ref based world compared to a hash based world.
• I gave arguments why my group of Transcrypt users would rather like in work js style (and regard in->indexOf for arrays just as one example of the many Transcrypts' efficiency benefits when programming actually javascript, but pythonic, and not programming Python).

And now its on you to judge how important those two groups are for you, nobody else can :-)

PS: I meanwhile changed all my code to hasattr anyway, so I'll have no problem and respect your decision against it in any case, Transcrypt will remain the best tool for my group.

PS2: Since it fits here: This mental shift happening after a while, i.e. the 'I program javascript but pythonic', does also mean, that I have no problem anymore with the Python version (you remember when I was a bit sad about no Py2 version? It does not matter anymore to me, the exact same code will never run on server and on client, in general).

[JdeH]

Hi Gunter & Dabo,

First clear up an apparent misunderstanding:

While I certainly have affinity with educating young folks to become programmers, they are not the primary target audience of Transcypt. It was designed because I myself needed something pythonic to program browser applications.

So the primary target audience are professional web application developers. I only had a blind spot for JS converts, and mainly wanted to serve the developers who already use Python on the back-end, but not anymore now.

We'll have it in. Since many JavaScript things can be done with Transcrypt objects and variables (owed to the fact they're actually JS things), adding 'in' to the mix is only small sin from an 'isomorphism' point of view.

On the other hand, making a very artificial distinction between dicts and native JS objects would indeed be quite confusing.

I am convinced, it'll be in the next commit.

Thanks for the effort in drawing me over the line. I just want to be very concise about the direction of TS, since many small course changes amount to a totally different direction. But this one clearly is on the right side of the line.

By the way, the review on the jashkenas site was written by myself, anyone can edit that page. But still I think it's accurate.

KR Jacques

[JdeH]

It's committed including a manual test (that's part of the shipment test).

[axgkl]

Dankewell...

I just want to be very concise about the direction of TS

• I absolutely know the feeling (thinking years about a problem and other people after only weeks say this and that feels wrong - they have usually no idea...) and I know that this one is not easy for you, trying to stay as compliant as possible, and totally reassuring everytime where you get something compliant out of the box, telling you that the whole design is right. And you get that not iterable exception in CPython and a similar one (no indexOf) in Transcrypt for free. Tough to change.
• I am absolutely dependent on you NOT changing general direction for TS just because people cry
• Delighted to hear about the positioning of TS (it would have violated anything they told me about software design actually, if you would have designed it for other purposes than what it turned out to be ;-))

---

I think someone of us should write a sentence or two at a prominent place in the docu about general philosophy of handling situations like this. Explain this more, say non strict approach, so that people evaluating the tool have the right mindset. Because I admit that in the beginning I would have been also impressed by getting the exception. But those were the days where I also did put ".defineProperty" into js pragmas, no fully comprehending that Transcrypt is not ONLY a tool with a high compliancy to CPython but it is a tool with a superhigh compliance with Javascript AS WELL, with the ast parser as the only limit. Never programmed in two languages with the same code :-)

I think that should be documented, since this is not about in only, there are more situations like this, where CPython would fail while TS keeps working. Example: def foo(bar): bar=dict(bar) and you always have a dict even if bar is False. Good to know about the possibility to test those not compliant things as well, btw, We need to have those fixed forever, otherwise apps might crash after updates, should you change those. I'll suggest tests for such things in the future and you decide if this and that always can stay like this or you prefer a "fix" in the compiler - so that I early adapt my code to a future proof version.

Thanks again.

[JdeH]

Philosophy:

I plan to have a brief chapter about that in the docs. Probably near the end, since to many it won't be the most sexy part. But it sure serves a purpose.

Futureproof-ness and code falling over:

Some expectation-management is necessary here. Even adding 'in' as just done, can of course make code fall over if compiled with the newest version of TS, code that relies on the exceptions when it wasn't in. This is an unrealistic example, but in general, completely disallowing any code falling over would effectively freeze Transcrypt forever.

However careful I tried to be in picking the right overall direction, there will be design flaws which have to be corrected at some point. To at the same time not impose the burden of extra work on the user, it is very important that existing Transcrypt releases remain available. For important projects I even used to store the right tool versions together with the source code.

I must say that due to the availability of any version of any package via the internet (Turbo C++) I have become a bit sloppy in this respect. But still if large interests are at stake, I try to package everything needed to recreate the project.

However unsatisfactory, I've more or less become accustomed to the fact that none of my older code can be compiled with current tools. A striking example of this is C++. Compilers used to be lenient and many useful but somewhat informal constructs were tolerated, const's weren't nagged about etc. Nowadays C++ compilers are downright paranoid and none of my older code translates without modifications, often making it grow quite substantially. This is something I find highly frustrating. I fact to my judgement all the complexity added to C++ to make it safe created a monstrum that no-one completely oversees anymore.

With websites and internet applications the situation is even far worse. It is quite normal to me that a website that a customer paid me for, stops working because some external resource supplier has changed a policy. The most recent example of this was twitterfeeds stopping working because certain 3rd party components didn't match the newest twitter query protocol anymore. Another example is embedded YouTube videos suddenly requiring a different protocol (which made the customer switch to Vimeo after some frustration). I always have a hard time explaining why I cannot completely prevent that kind of thing from happening.

Having said that, I think that, although it will never be perfect, utmost care should be taken to preserve backward compatibility whereever reasonable. As far as I am concerned that is the plan for Transcrypt.

If however the ECMA suddenly decides to drop objects from JS altogether in favor of the most terse and inhuman variant of stateless functional programming, I wouldn't know what to do. O well, I would... I would propose Python as its replacement to the browser manufacturers. And maybe they would seriously consider it in that case... Or, more likely, they would all come up with there own, incompatible alternative. I wish this were only a joke, but the Java / C# rift, the OS2 / WindowNT debacle, attempts to take OpenGL out of Windows and the most recent Windows updates forcefeeding policies have shown us what this industry cares most about. It's not the user.

[axgkl]

Did not say: TS must behave exactly like today next year. Its open source and no user has any guarantee.

But IF something is in the autotest OR manual test, than it should be reasonably safe to rely upon that behaviour. The _in_ exception was never in the tests, I checked and did not see it.

If it was already in the autotest (was it?) then I would not have proposed the change.

The testsuite should be the always up to date spec of TS. I think you said that somewhere and I like that very much (I'm a behave addict in my projects ;-))

[JdeH]

Quite right. If something is explicitly in any test, great care should be taken to preserve it. In fact the tests informally define the language, as far as I am concerned.

[JdeH]

A note about the autotests defining the language needs to be in the docs, indeed. By the way I completely agree about 'behave'. The IT industry is gradually developing some common sense, it seems.

[axgkl]

naaa, forget it. Tried to many times to convince customers of specifying their requirements in an executable way. And still their architects prefer to compile tons of word documents as specs :-)

[JdeH]

Same experience here. But after some striking experiences, I now always try to come up with early prototypes and simulations in an early stage. As soon as they see something working, trust is established and most of the paper pile is forgotten.

The fun is (as no no doubt know) that the true stakeholders (so not the IT department in many cases) tend not to read their own requirements. They only start thinking once the product takes shape. When I first found that out, some 20 years ago, I was shocked. They are much more impatient and realistic that the IT departments that drew up that garbage.

But there's indeed an increased 'formalisation' of the communication between customers and IT companies. Customers seem to think they can avoid risks that way. But even the most detailed contract cannot make success out of failure.

It can help the customer to sue the IT company. But thats a meager consolation. After all what the customer originally wanted was a working application...

[JdeH]

I've added a first version of a global introduction about Transcrypt's philosophy at the end of the docs. It is currently only available on-line.

http://sterlicht.alwaysdata.net/transcrypt.org/docs/html/philosophy_directions.html

[axgkl]

Wow, I love it.

I hope I find the time soon (currently have a few timesuckers on my back) to write one full page with examples about this sentence:

"It means that Transcrypt's and JavaScript's type system are unified, compact, fast and interoperable"

Thanks a lot for taking the time to write this, will help for the evangelizing effort ahead ;-)

[JdeH]

Will be a pleasure to work in this in cooperation! I've added some more details, but it's still very limited. Examples are indeed needed.

[axgkl]

I think most convincing would be something interactive, where people see the immediate compile and download turnaround times. Maybe... an IPython Notebook, where you write Transcrypt into the cells and it executes in (I)Python, but also transpiles, shows the resulting js and also executes that in the browser.

[axgkl]

funny how this thread starts: "Hi, quick one" ;-)

[JdeH]

While interactivity would be nice, it's also quite some work. To show the relation between the philosophy behind Transcrypt and certain design choices, adding some examples to the Sphinx docs would already be very helpful. I'd like to keep the Transcrypt website itself low tech, to minimize time investment in its maintenance. However a link to interactive examples hosted somewhere else would be a quicky.

[axgkl]

got the message (and also have some servers avail). Won't happen the next weeks but I'll find some time... Will happen this year if nobody else does it, its too much fun to not do it.

[JdeH]

I try to keep concentrating upon TS itself, since I sense that things could easily get bigger than I can handle if I not restrict myself to the essence. But docs and examples are very important for acceptation. As far as the docs are concerned I try to stay on top of that myself. As for the examples I have to let go a little bit.

BTW. Could you also please make small nonsense edit to the README.rst and commit it. It seems that this act will get you listed as a contributor, which I would welcome. At least this trick worked in the past.

BTW2. I've submitted a Wikipedia article for review:

https://en.wikipedia.org/wiki/Draft:Transcrypt