[patch] Transform utilities

[robertwb]

Reported by dagss on 16 May 2008 16:41 UTC It should now be possible to do stuff like:

class WithTransform(VisitorTransform):
     # from with transform PEP...
     with_fragment = TreeFragment(u"""
_mgr = (EXPR)
_exit = mgr.__exit__
_value = mgr.__enter__()
_exc = True
try:
     try:
         VAR = _value
         BLOCK
...
<snip>
...
""")

     def process_WithStatementNode(self, node):
         return self.with_fragment.substitute({
             "EXPR" : node.expr,
             "VAR" : node.var,
             "BLOCK" : node.body
         })

:-)

(The above is simplified, there's not always a VAR. Also it needs another feature before it can be completely streamlined (automatic "temporaries" that won't clash in the namespace; basically, "with_fragment.substitute(..., temps=("_mgr", ...)). When that is done, supporting the with statement is about as much work as extending the parser, the transform/implementation comes for free.

• I've already discussed the CodeWriter. It only supports a limited subset (with some holes, ~30% perhaps) at this time; but it's what I need for now (for unit tests). I might work further on that too.
• Some changes to Transform.py which I hope goes through... there's a Visitor object there; using the "process_ClassName" pattern (I think that was the conclusion for future performance reasons).
• A clone_node method on Node for proper node copying (shallow object copy except child node lists, which are also copied).
• Here's the controversial bit:

In order to be able to provide proper error messages for string-based code snippets like the above (which are passed to Parsing.py...); I've changed the pointer to the source code (used as the first element in the position tuples found everywhere...) from being a string filename to being a SourceDescriptor object.

A SourceDescriptor can currently be a FileSourceDescriptor, in which case things work like before (it gives the filename on str so much code needed not change), or a StringSourceDestriptor which I use for my new code...

I hope you see the advantages to this from the above code. (There are less intrusive ways to do this, but they would only be hacky and postpone the problem. Better do it properly...? BTW this pattern is rather common, consider for instance Source in the XML Transform APIs/TrAX.)

Migrated-From: http://trac.cython.org/ticket/11

[robertwb]

Comment by robertwb on 3 Aug 2008 06:48 UTC Similar code has now been merged into the core and works great.

[robertwb]

Modified by robertwb on 19 Aug 2008 03:59 UTC

[robertwb]

Modified by robertwb on 19 Aug 2008 04:55 UTC