Silence ply errors

[warrenspe]

Running slimit on cygwin produces several warnings each time it is run

>>> import slimit
s>>> slimit.minify("")
WARNING: Couldn't write lextab module <module 'slimit.lextab' from '/usr/lib/python3.6/site-packages/slimit/lextab.py'>. Won't overwrite existing lextab module
WARNING: yacc table file version is out of date
WARNING: Token 'LINE_COMMENT' defined, but not used
WARNING: Token 'BLOCK_COMMENT' defined, but not used
WARNING: Token 'LINE_TERMINATOR' defined, but not used
WARNING: Token 'CLASS' defined, but not used
WARNING: Token 'CONST' defined, but not used
WARNING: Token 'ENUM' defined, but not used
WARNING: Token 'EXPORT' defined, but not used
WARNING: Token 'EXTENDS' defined, but not used
WARNING: Token 'IMPORT' defined, but not used
WARNING: Token 'SUPER' defined, but not used
WARNING: There are 10 unused tokens
WARNING: Couldn't create <module 'slimit.yacctab' from '/usr/lib/python3.6/site-packages/slimit/yacctab.py'>. Won't overwrite existing tabmodule
''

Using ply v 3.10. Only fix for me appears to be to revert to ply 3.4

[metatoaster]

This can be worked around by removing those *tab.py files under the slimit directory (along with the related .pyc there or inside the __pycache__ directory); you will likely need to be the root user (or sudo) to remove them and regenerate them (again as root) using the current version of ply that was also installed (by running python -c 'import slimit;slimit.minify("")', and as root for a system level installation).

Really though, this package doesn't look maintained anymore so I ended up forking this, took the good bits as a base for my own.

[rod-app]

I get the same verbose dreck spewed out to stderr. This exceedingly ugly one-liner anywhere after import slimit, but before calling minify() silences the logging:

slimit.lexer.ply.lex.PlyLogger =  \
slimit.parser.ply.yacc.PlyLogger = \
  type('_NullLogger', (slimit.lexer.ply.lex.NullLogger,),
       dict(__init__=lambda s, *_, **__: (None, s.super().__init__())[0]))

This works for me in Python 3.6 with slimit 0.8.1 and ply 3.11 -- maybe others, but YMMV.

All this does is monkey-patches the PlyLogger class with the NullLogger class the ply package defines itself, adapting for the differences in initializer signatures between the two classes (shame, shame). If ply has a configuration setting or optional initializer parameter somewhere that effectively does this -- and it should -- I couldn't find it. (Hint, hint, ply maintainers.) Maybe in lieu of that, slimit should do this internally instead of putting the onus on the user?

[metatoaster]

@rod-app The problem actually goes deeper than that - if those messages are generated every time minify is called, it means that the ply tab modules are being regenerated (which takes as long as a second), and if there are a number of JavaScript files being processed that extra delay and thus performance hit to the program will accumulate rather quickly. Removing the message simply hides the problem but not actually fixing it - the only way to fix it is to manually remove the problematic tab files and regenerate them correct as I had suggested.

The fork that I made (calmjs.parse) does not track the tab files for the source repository, but they are package/generated for every supported (and released) ply versions, such that they are not needlessly being regenerated for the end-users, and in the case a newer version of ply is out there is a specific command (python -m calmjs.parse.parsers.optimize) with the reasoning behind it explained in the README for the package.

Also please note that slimit has not received a new release for nearly 6 years now, so I made that fork as an attempt to rectify the situation (the slimit minify equivalent is available via from calmjs.parse import es5 and then call es5.minify_print).

[rod-app]

Thanks for the response, Tommy. I integrated calmjs into my app, substituting for slimit, and everything works fine. You were right -- slimit' had been shockingly (unacceptably) slow on account of needing to regenerate the tab files each request. I haven't benchmarked anything, butcalmjs` is noticeably faster.

There are a number of out-and-out wrong Python JS minifiers out there; yours is a keeper. Excellent work!

Rod Pullmann Appion, Inc. (303) 589-4405

On Sat, Feb 16, 2019 at 5:27 PM Tommy Yu notifications@github.com wrote:

@rod-app https://github.com/rod-app The problem actually goes deeper than that - if those messages are generated every time minify is called, it means that the ply tab modules are being regenerated (which takes as long as a second), and if there are a number of JavaScript files being processed that extra delay and thus performance hit to the program will accumulate rather quickly. Removing the message simply hides the problem but not actually fixing it - the only way to fix it is to manually remove the problematic tab files and regenerate them correct as I had suggested.

The fork that I made (calmjs.parse https://github.com/calmjs/calmjs.parse) does not track the tab files for the source repository, but they are package/generated for every supported (and released) ply versions, such that they are not needlessly being regenerated for the end-users, and in the case a newer version of ply is out there is a specific command (python -m calmjs.parse.parsers.optimize) with the reasoning behind it explained in the README for the package.

Also please note that slimit has not received a new release for nearly 6 years now, so I made that fork as an attempt to rectify the situation (the slimit minify equivalent is available via from calmjs.parse import es5 and then call es5.minify_print).

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rspivak/slimit/issues/97#issuecomment-464402589, or mute the thread https://github.com/notifications/unsubscribe-auth/AlQiEx4neYv6BZxKj4OPQl0ah7ERsrmVks5vOKHdgaJpZM4RghW4 .

--

---

This message may contain privileged or confidential information for the sole use of the intended  recipient.  Any review or distribution by others is strictly prohibited.  If you are not the intended recipient of this message, you may not make use of, or rely in any way on this information.  You should immediately notify the sender by reply email and destroy this message.  Please advise us immediately if you or your employer do not consent to Internet email for messages of this kind.

---