Maximum number of arguments in Python

[hut8]

Hey! Quoting your documentation,

When I say it supports everything I really mean everything:

I hate to be that guy, but I'm using your library (which is really amazing, by the way) and I actually encountered this issue in a poorly designed JS file. Reproduction:

test_str = "arr = new Array({})".format(','.join([str(x) for x in range(257)]))
import js2py
ctx = js2py.EvalJs()
ctx.execute(test_str)

💥

Traceback (most recent call last):
  File "/home/liam/.virtualenvs/scraper/lib/python3.5/site-packages/js2py/evaljs.py", line 171, in execute
    compiled = cache[hashkey]
KeyError: b'bT(\xc0iho\x10\xc1q\xe6\x8c\xd1 \xe8U'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/liam/.virtualenvs/scraper/lib/python3.5/site-packages/js2py/evaljs.py", line 174, in execute
    compiled = cache[hashkey] = compile(code, '<EvalJS snippet>', 'exec')
  File "<EvalJS snippet>", line 3
SyntaxError: more than 255 arguments

I guess not much you can do, right? Maybe document it?

[PiotrDabkowski]

Hey, when I said it supports everything I meant that every feature in ECMA 5.1 specification (except ones mentioned in limitations) is implemented :) But there are some rare corner cases that are extremely hard to predict - mostly due to unexpected behaviour of resulting Python code or some very rarely used JavaScript patterns.

To fix your problem I would have to change the design and fixing all these edge cases would take too much time with translation approach. Maybe if I find some time I will add pure Python JavaScript interpreter which should be much more predictable and stable. But current Js2Py works very well even on huge JS libraries like babel - 50,000 lines of code and is able to call them without errors (today I added translation of babel to support ECMA 6, unfortunately its very slow).

I will clarify the readme :) + I believe this interpreter approach would make Js2Py as reliable as node.

[Alexey-T]

Maybe you can support Emmet (js code) later too? (wish)

[PiotrDabkowski]

I will start implementing interpreter today, I am curious how it works out! When its done you will be able to run whatever you want :) Do you want to take part in interpreter development?

[Lexcon]

It sounds like you have quite a worked out plan in mind. Could you give an overview about how you would go about this? How would this be different from the current py2js?

It almost sounds like you are going to write bytecode directly, or am I misunderstanding that?

I'm not sure if I could be a valuable contributer to this but I'd be happy to discuss the possibilities.

Robert

On 16-11-2016 17:11, Piotr Dabkowski wrote:

I will start implementing interpreter today, I am curious how it works out! When its done you will be able to run whatever you want :) Do you want to take part in interpreter development?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/PiotrDabkowski/Js2Py/issues/65#issuecomment-260988614, or mute the thread https://github.com/notifications/unsubscribe-auth/AD5u-744tkt9LO7qo0cSe-tEYYvJfkSjks5q-ysogaJpZM4KzJ4_.

[PiotrDabkowski]

Hey Robert, thank you for your interest :) I think about interpreting the AST directly without bytecode step, this would be a simple approach.

Current functionality of Js2Py - translation from JS to Python - will be retained and should work independently. However interpreting AST directly will give us much more flexibility and we will not be constrained by limitations of python design anymore. I will think of exact design and let you know today or tomorrow. I have to learn about bytecodes and interpreters first :)

[Lexcon]

I'm not sure what means 'interpreting the AST directly'. I'd say each ast node needs to translate to a python statement that is then executed by pyhon. The small difference is that translation goes as-you-go instead of emit-full-python-then-run. In case of unused branches of the code this eliminates some python generation. But how are you going to deal with loops, are you going to interpret each iteration of the loop separately?

And how will that give more flexibility? I think I just don't have it clear what you have in mind.

Robert

On 16-11-2016 17:49, Piotr Dabkowski wrote:

Hey Robert, thank you for your interest :) I think about interpreting the AST directly without bytecode step, this would be a simple approach.

Current functionality of Js2Py - translation from JS to Python - will be retained and should work independently. However interpreting AST directly will give us much more flexibility and we will not be constrained by limitations of python design anymore. I will think of exact design and let you know today or tomorrow. I have to learn about bytecodes and interpreters first :)

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PiotrDabkowski/Js2Py/issues/65#issuecomment-261001519, or mute the thread https://github.com/notifications/unsubscribe-auth/AD5u-5gmMpD1NAgGWWZjm1jvtUu0OgFoks5q-zQAgaJpZM4KzJ4_.

[PiotrDabkowski]

Take a look at ECMA 5.1 specification. Most of it is already implemented, but the execution mechanism is slightly different. I completely ignored their remarks about execution and took a different approach trying to be as compatible as possible. In my approach, I am generating a new Python code that should be equivalent to javascript. In interpreter approach, I will just write one Python script that is able to follow instructions in AST and execute them directly. So no python code will be generated. This gives more flexibility as I will, for example, be able to call a function with more than 255 arguments allowed by python, prevent recursion limit error and have behaviour that is exactly as defined in specification without using ugly python tricks.

As a nice example x++ translates to:

`(var.put(u'x',Js(var.get(u'x').to_number())+Js(1))-Js(1))`

Interpreter will have postIncrement function that simply increments the value as described here.

[ghost]

@PiotrDabkowski btw, if you REALLY interested in performance, have a look at RPython and http://bitbucket.org/pypy/lang-js RPython - both framework and toolchain for creating language interpreters, also you can implement JIT easily. RPython - subset of Python, translatable to C. PyPy use it as it's main framework

[PiotrDabkowski]

Thanks, this looks really interesting, I will check how well it works!

[ghost]

@PiotrDabkowski but it's harder than Python because you can't use dynamic features. As RPython suggest, firstly write your interpreter in Python, and then add support to RPython

[PiotrDabkowski]

As I said most of the code is already written, parser, built-in objects all the internal methods etc. Only remaining thing is this execution framework, it should be done relatively quickly.

[ghost]

@PiotrDabkowski but really, RPython doesn't allow many, many python things

[PiotrDabkowski]

There is no much point in developing a RPython compatible version - first, its already done and second, more importantly, it kind of defeats the whole purpose of Js2py - to be able to use JS from PURE python. You cant enjoy benefits of RPython code (JIT, fast) with standard version of python anyway. I will try to use parts of his code in my Js2Py, it will save me some time :)

[PiotrDabkowski]

I will add an option to build resulting python interpreter with Cython to improve the performance.

[Lexcon]

I finally understand your current purpose, it's a javascript virtual machine implemented in the python virtual machine.

That's a nice intellectual challenge and I love the idea if even just because of that.

Performance will not be great I think, but who knows... maybe you can even compile it to Cython to have a somewhat optimized JS virtual machine available within Python.

It's a brilliant idea.

On 16-11-2016 18:55, Piotr Dabkowski wrote:

As I said most of the code is already written, parser, built-in objects all the internal methods etc. Only remaining thing is this execution framework, it should be done relatively quickly.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PiotrDabkowski/Js2Py/issues/65#issuecomment-261020400, or mute the thread https://github.com/notifications/unsubscribe-auth/AD5u-4uMYQjCVE1erTGZ5CyFzPWOggPsks5q-0N7gaJpZM4KzJ4_.

[Lexcon]

Our mails crossed :)

On 16-11-2016 19:33, Piotr Dabkowski wrote:

I will add an option to build resulting python interpreter with Cython to improve the performance.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/PiotrDabkowski/Js2Py/issues/65#issuecomment-261031147, or mute the thread https://github.com/notifications/unsubscribe-auth/AD5u-9opT9z7-eSU8Sw2vopRoYHFXkQqks5q-0x2gaJpZM4KzJ4_.

[Alexey-T]

I am curious how it works out! When its done you will be able to run whatever you want

it is very good news. Thankx in advance!

[hut8]

Hey @PiotrDabkowski looks like this thread got pretty off-topic. My use case is actually for Arrays only. I think that one AST transformation here would suffice. Specifically, if you could change:

Array( ... ) ➡️ [ ... ]

that would solve my real-life problem 😄 I'm not a JS guru like yourself, but I believe array literals and the Array constructor are exactly equivalent. I've done this in the interim for the project I'm working on but I used a horrible regex hack that I'm pretty ashamed of.

For the actual code that I was testing to crash this, it was at: http://www.startimes.com/f.aspx?t=36588040 (check out var topics = Array(...). The code is so poorly architected that this simple web scraping project I'm working on turned into what felt like cryptanalysis 😉

[PiotrDabkowski]

There is no point solving this specific issue, there are much more edge cases which are almost impossible to solve with the current approach - eg recursion limit of python. I will solve all the problems by implementing a proper interpreter.