Missing constant in v2.1.0

[jaredcwhite]

I started using the new release as a dependency of Ruby2JS and got this error:

Error: …/regexp_parser-2.1.0/lib/regexp_parser/scanner.rb:13:in `<class:Scanner>': uninitialized constant Regexp::Parser (NameError)

Downgrading to 2.0.3 worked again. Since the error is caused by a missing constant within the gem code itself, I don't think it's directly caused by anything in Ruby2JS. (But if so, let me know if I can help with troubleshooting.)

[jaredcwhite]

Actually, it might be the usage of require 'regexp_parser/scanner' directly. I think that was done for purposes of supporting Opal in addition to MRI.

[rubys]

Background information: Opal doesn't have a YAML parser, and Ruby2JS only needs the scanner, so we are looking for a set of requires that does not pull in YAML.

[rubys]

It looks like the scanner depends only on the error object, so if it had a require statement in place, the scanner could be used independently.

[rubys]

sigh. __dir__ is also problematic for Opal.

[jaynetics]

@jaredcwhite @rubys

thank you for the report and investigations!

it might be the usage of require 'regexp_parser/scanner' directly.

i see, i'll release a fix for that shortly.

Background information: Opal doesn't have a YAML parser, and Ruby2JS only needs the scanner, so we are looking for a set of requires that does not pull in YAML.

are you sure this is the case? i have zero knowledge about Opal, but the scanner also contains require 'yaml', since v1.0.0 released in 2018.

YAML is used for unicode property lists, so either Opal does have a YAML parser now, or scanning a Regexp like /\p{digit}/ should cause problems?

sigh. __dir__ is also problematic for Opal.

can you elaborate? problematic as in "can't be used"? is there any other standard Ruby that needs to be avoided?

[rubys]

Looks like I was wrong on YAML.

Perhaps a more full explanation of the motivation here would be in order. Take a look at our demo page.

What you will see is a Ruby statement converted into JavaScript. Ruby's indexing methods are very powerful, but many common patterns can be implemented with a more primitive replace function. To do that conversion, Ruby2JS needs to understand which parens in the regular expression are data, and which are grouping operators. That's done by using regexp_parser. So that's your code, running in the browser helping Ruby2JS know how to interpret the parens.

That's a live demo. Go ahead and change the Ruby code and see what JavaScript code is produced.

Ruby2JS has used the regexp_parser gem for some time now, but recently when I got this demo up and running I had a problem with the full parser, determined that for our modest needs I could make do with just the scanner, so I went with that. At the time, I thought the problem was the require for YAML, but I guess I was wrong about that.

So that's our use case / motivation.

If I upgrade the demo to regexp_parser version 2.1.0, and add the necessary require "regexp_parser/error", I now get:

"__dir__: undefined method `__dir__' for Regexp::Scanner
    at Function.$method_missing (http://localhost:4001/demo/ruby2js.js:1:76991)
    at Function.method_missing_stub (http://localhost:4001/demo/ruby2js.js:1:19812)
    at http://localhost:4001/demo/ruby2js.js:1:4943163
    at Opal.modules.regexp_parser/scanner (http://localhost:4001/demo/ruby2js.js:1:4948332)
    at Object.Opal.load (http://localhost:4001/demo/ruby2js.js:1:34719)
    at constructor.Opal.require (http://localhost:4001/demo/ruby2js.js:1:35126)
    at Opal.modules.ruby2js/filter/functions (http://localhost:4001/demo/ruby2js.js:1:4949731)
    at Object.Opal.load (http://localhost:4001/demo/ruby2js.js:1:34719)
    at constructor.Opal.require (http://localhost:4001/demo/ruby2js.js:1:35126)
    at Opal.modules.filters (http://localhost:4001/demo/ruby2js.js:1:5241701)"

The problem here seems to be https://github.com/ammar/regexp_parser/blob/948441acb09236eab6a150d261207f678443b523/lib/regexp_parser/scanner/scanner.rl#L761

In the browser, there is no file system, and therefore no __dir__.

Opal does have a few limitations, and in many cases they can be avoided by monkey-patching the code being used, but that works best when the code being replaced is isolated and small. Having an assignment statement in the middle of scanner.rl referencing the filesystem is problematic.

Thanks for considering this request.

[jaynetics]

@rubys

cool project! it's fun to play around with ruby and see what it turns into.

i've just released regexp_parser v2.1.1 that restores the ability to require the scanner individually and no longer uses __dir__ during load time.

let me know if this works for you!

b.t.w. have you seen js_regex? it is a gem that translates Ruby regexps to the JavaScript flavor. it might be overkill to use in Ruby2JS, but at least the README gives a good overview of the many differences that one might want to keep in mind when writing a regex that is supposed to work both in Ruby and in JS.

[rubys]

@jaynetics initial testing looks good!

js_regex looks like a good match (definitely not overkill). Ruby2JS has a modest number of tests for functions that would need to be supported. Most notably, interpolation.

It would also take some work to get it buildable with Opal, for example: https://github.com/jaynetics/js_regex/blob/e9845a0836a10f03e635064c024867b0dcc18491/lib/js_regex/converter.rb#L5

[rubys]

Fix has now been deployed on the ruby2js site. This issue can now be closed. Thanks!