search

strongloop / strong-globalize

strong-globalize is built on Unicode CLDR and jquery/globalize and implements automatic extraction of strings from JS source code and HTML templates, lint the string resource, machine-translate them in seconds. In runtime, it loads locale and string resource into memory and provides a hook to persistent logging.
Other
25 stars 16 forks source link

chore: upgrade to espree parser #173

Closed emonddr closed 3 years ago

emonddr commented 3 years ago

Trying the espree parser to replace the esprima parser.

The latter cannot handle the ... spread operator. It only supports 2017 JS.

The espree parser, https://github.com/eslint/espree , looks promising since it was designed to be very similar to esprima

Related to: https://github.com/strongloop/strong-globalize/issues/174

emonddr commented 3 years ago

cli/test/test-extract.js is the only test in the suite that is falling.

Not sure the purpose of the test case.

But instead of reading a file's contents into a string named content and then passing this into the parser, it does a

content = this

which is puzzling. why not read from a file? The text is pointing to the context of the function.

https://github.com/strongloop/strong-globalize/blob/master/packages/cli/test/test-extract.js#L116

If you could take a quick look at this @raymondfeng , it would be appreciated. thx

emonddr commented 3 years ago

The master branch npm test results


  🌈 SUMMARY RESULTS 🌈  

Suites:   33 passed, 33 of 33 completed
Asserts:  1256 passed, of 1256
Time:     2m
raymondfeng commented 3 years ago

which is puzzling. why not read from a file? The text is pointing to the context of the function.

It just use in-memory string for source code to be extracted.

emonddr commented 3 years ago

This PRs test results

Most TAP tests (doesn't use Mocha framework) pass with this change:


  🌈 SUMMARY RESULTS 🌈  

 FAIL  packages/cli/test/test-extract-2.js 2 failed of 57 1s
 ✖ extract005: err msg matches.
 ✖ extract012: out msg matches.

 FAIL  packages/cli/test/test-extract.js 6 failed of 133 3s
 ✖ singleton_head + singleton_body: all locs extracted from JS.
 ✖ multiple_head + singleton_body: all locs extracted from JS.
 ✖ content_multiple: all locs extracted from JS.
 ✖ content_multiple_new: all locs extracted from JS.
 ✖ pseudo msgs: all literals extracted from JS.
 ✖ pseudo locs: all locs extracted from JS.

Suites:   2 failed, 31 passed, 33 of 33 completed
Asserts:  8 failed, 1248 passed, of 1256

I had to remove the --bail from the package.json's npm run test so that TAP doesn't exit after a few failures.

emonddr commented 3 years ago

I need to look into the line number issue...

raymondfeng commented 3 years ago

The error message is slightly different:

https://github.com/strongloop/strong-globalize/blob/dremond_espree_change/packages/cli/test/test-extract-2.js#L84

Espree does not have the description in error object. We need to fix the assertion.

emonddr commented 3 years ago

All tests pass

> tap --bail --coverage --coverage-report=cobertura --timeout=200 packages/*/test/test-*.*

 PASS  packages/cli/test/test-extract-3.js 6 OK 570.051ms
 PASS  packages/cli/test/test-extract-4.js 10 OK 854.592ms
 PASS  packages/cli/test/test-extract-2.js 57 OK 2s
 PASS  packages/cli/test/test-extract-shared-globalize.js 5 OK 817.581ms
 PASS  packages/cli/test/test-extract.js 133 OK 2s
 PASS  packages/cli/test/test-extract-json.js 75 OK 1s
 PASS  packages/cli/test/test-auto-msg-load.js 172 OK 16s
 PASS  packages/cli/test/test-lint.js 143 OK 1s
 PASS  packages/cli/test/test-setregex.js 12 OK 421.433ms
 PASS  packages/cli/test/test-translate-2.js 1 OK 38.557ms
 PASS  packages/cli/test/test-translate-3.js 1 OK 20.691ms
 PASS  packages/cli/test/test-translate-4.js 1 OK 15.068ms
 PASS  packages/cli/test/test-translate-5.js 1 OK 44.807ms
 PASS  packages/cli/test/test-translate-6.js 1 OK 31.884ms
 PASS  packages/cli/test/test-translate-7.js 1 OK 29.733ms
 PASS  packages/cli/test/test-translate-8.js 1 OK 58.831ms
 PASS  packages/cli/test/test-translate-9.js 1 OK 14.779ms
 PASS  packages/cli/test/test-translate.js 42 OK 190.366ms
 PASS  packages/runtime/test/test-format-json.js 24 OK 5s
 PASS  packages/runtime/test/test-globalize-errors.js 12 OK 1s
 PASS  packages/runtime/test/test-globalize-multiple.js 19 OK 1s
 PASS  packages/runtime/test/test-globalize-singleton.js 53 OK 4s
 PASS  packages/runtime/test/test-load-msg.js 14 OK 484.157ms
 PASS  packages/runtime/test/test-load-msg-non-forking.js 61 OK 4s
 PASS  packages/runtime/test/test-logging-multiple.js 128 OK 2s
 PASS  packages/runtime/test/test-load-msg-forking.js 62 OK 5s
 PASS  packages/runtime/test/test-logging-singleton.js 136 OK 3s
 PASS  packages/runtime/test/test-misc-format.js 26 OK 853.068ms
 PASS  packages/runtime/test/test-misc-help.js 5 OK 1s
 PASS  packages/runtime/test/test-package.js 10 OK 288.525ms
 PASS  packages/runtime/test/test-scan-object.js 27 OK 320.261ms
 PASS  packages/runtime/test/test-sort-msges.js 6 OK 95.81ms
 PASS  packages/runtime/test/test-setdir.js 10 OK 114.523ms

  🌈 SUMMARY RESULTS 🌈  

Suites:   33 passed, 33 of 33 completed
Asserts:  1256 passed, of 1256
Time:     2m
emonddr commented 3 years ago

@raymondfeng , it seems like a switch to espree is successful.

However, I still cannot generate intl/en/messages.json in velox/monetization/packages/billing-killbill

using my branch of strong-globalize

Dominiques-MacBook-Pro:billing-killbill dremond$ node /Users/dremond/Documents/PRs_LoopBack/extract_strings/strong-globalize/packages/cli/lib/cli.js --e

--- root: /Users/dremond/Documents/monetization/apic_integration_tests/monetization/packages/billing-killbill
--- max depth: N/A
--- cloned: N/A
--- scanned: 23 js, 0 html 
--- skipped: 0 js, 0 html 
--- extracted: 0 msges, 0 words, 0 characters
raymondfeng commented 3 years ago

Sounds good. Would you please clean up the code for final review?

emonddr commented 3 years ago

Sounds good. Would you please clean up the code for final review?

Hi @raymondfeng , I may have changed the parser, but it still doesn't work on velox/monetization/packages/billing-killbill

> tap --bail --coverage --coverage-report=cobertura --timeout=200 packages/*/test/test-extract-2.js

packages/cli/test/test-extract-2.js 1> <<< BEGIN domBilling1
packages/cli/test/test-extract-2.js 1> out ________________
packages/cli/test/test-extract-2.js 1>  [
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/application.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/component.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/account-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/catalog-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/health-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/index.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/invoice-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/payment-method-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/subscription-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/helpers/unit-helper.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/index.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/interceptors/index.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/interceptors/killbill-security.interceptor.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/keys.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/base.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/billing.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/bulk-subscription.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/index.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/killbill-health-check.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/payment-method.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/product.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/services/subscription.service.js\n',
packages/cli/test/test-extract-2.js 1>   'DOM: processing file /private/dist/utils/globalize.js\n',
packages/cli/test/test-extract-2.js 1>   '\n' +
packages/cli/test/test-extract-2.js 1>     '--- root: \n' +
packages/cli/test/test-extract-2.js 1>     '--- max depth: N/A\n' +
packages/cli/test/test-extract-2.js 1>     '--- cloned: N/A\n' +
packages/cli/test/test-extract-2.js 1>     '--- scanned: 23 js, 0 html \n' +
packages/cli/test/test-extract-2.js 1>     '--- skipped: 0 js, 0 html \n' +
packages/cli/test/test-extract-2.js 1>     '--- extracted: 0 msges, 0 words, 0 characters\n'
packages/cli/test/test-extract-2.js 1> ]
packages/cli/test/test-extract-2.js 1> err ________________
packages/cli/test/test-extract-2.js 1>  []
packages/cli/test/test-extract-2.js 1> ________________
packages/cli/test/test-extract-2.js 1> END domBilling1 >>>
 PASS  packages/cli/test/test-extract-2.js 1 OK 536.458ms

  🌈 SUMMARY RESULTS 🌈  

Suites:   1 passed, 1 of 1 completed
Asserts:  1 passed, of 1
Time:     5s

My latest commit added the domBilling fixture (which contains files from billing-killbill) as the only test case in test-extract-2.js .

Notice

extracted: 0 msges
raymondfeng commented 3 years ago

@emonddr I was able to make it working with the following:

import path from 'path';
import SG = require('strong-globalize');

SG.SetRootDir(path.join(__dirname, '../..'), {autonomousMsgLoading: 'all'});
export = new SG();

Then:

import g from '../utils/globalize';
g.f('abc'); // slt-globalize does not recognize `globalize.g.f('abc')`
raymondfeng commented 3 years ago

@emonddr I added a commit to show the fix.

raymondfeng commented 3 years ago

@emonddr Let's add a few more tests to verify that {...} is supported.

emonddr commented 3 years ago

@emonddr Let's add a few more tests to verify that {...} is supported.

yup :)