Zydeco mangles line numbers in source code

[HaraldJoerg]

@smonff's article on PerlMonks and his interest in my work on Emacs' cperl-mode motivated me to give it a try and rewrite some simple modules with Zydeco as a test case for Emacs capabilities with editing Perl syntax extensions. In particular, Zydeco works with "real" keywords instead of imported subroutines, making it comparable to what we might expect with Cor.

Doing this I found a shortcoming in another area of Emacs integration: Zydeco mangles line numbers. This is a bit ugly but not all that critical when stepping through the code in a terminal session, and more annoying when line numbers of error messages come out wrong. But it makes Emacs' capability to display the source code while it is running totally useless. The Emacs window following the source code doesn't follow the execution, and setting breakpoints in the Emacs window often results in Line nnn not breakable.

Is there a workaround / behavior I need to change?

[tobyink]

Zydeco rewrites the source code as it goes along, adding bits here and taking parts out there. It attempts to keep track of line breaks in any parts that it takes out, and insert the same number of line breaks. Sometimes it will notice it's too far ahead and will refrain from inserting line breaks so Perl can "catch up".

The t/80lines.t test is intended to test this, but the way it's written right now, it just outputs some line numbers to compare, and the test always passes. Run it using prove in verbose mode:

prove -lv t/80lines.t

It's tricky to get it exactly right in all cases, and it's a work in progress. I will happily accept any patches to get it working better.

[HaraldJoerg]

Yeah, I understand that this is tricky. When I run the test, I get:

haj@hajtower:~/devel/perl/p5-zydeco$ prove -lv t/80lines.t 
t/80lines.t .. 
# Expected line 19!
# t/80lines.t line 21
# Expected line 29!
# Undef did not pass type constraint "Str" (in $_{"bar"}) at t/80lines.t line 29
#     "Str" is a subtype of "Value"
#     "Value" is a subtype of "Defined"
#     Undef did not pass type constraint "Defined" (in $_{"bar"})
#     "Defined" is defined as: (defined($_))
ok 1
1..1
ok
All tests successful.

I guess the "line 29" thing is as expected, but there's a two-lines offset between 19 and 21. I can get rid of this discrepancy by re-formatting the code so that the opening brace of the class and of the method are not on a line of their own each. But then, the exception reports line 27 instead of line 29. Funny enough, there's a similar bug in CPerl mode, that's why I tried it.

Do you, by chance, miss the newlines happening in (?&PerlOWS) parts of the grammar?

[tobyink]

I think I found the culprit. There was a hard-coded injection of:

#
#
#
#

When it encountered class XXX {}. I think I included that at some point in testing to see how well it coped with having to inject a lot of lines, whether it would throw off line counts for the rest of the file, or if it would recover within a few blocks later. The fact that it recovered by line 29 in the test shows that even when it got thrown off, it would eventually recover.

Anyway, having removed that, it's okay for both line 19 and line 29!

[tobyink]

And, no, it doesn't miss new lines in (?&PerlOWS). The way it does it, is once it knows that code is being replaced with new code, it counts the line breaks in both, and if the new code has fewer line breaks than the original, adds extra line breaks at either the beginning or end of the new code, depending on which keyword is being processed. If the new code has more line breaks than the original (which is rare, because I deliberately try to keep the injected code on as few lines as possible), it will remember that it "owes" that many line breaks, and try to compensate for it next injection.

[HaraldJoerg]

Excellent! If I run directly from your repo, line numbers are fine!

There's the minor quirk that when I step through the program from the beginning, it steps through a few strange points, apparently in the middle of nowhere. As I understand, that's when Zydeco statements are executed at run time which are, of course, invisible to Emacs. But usually I set a breakpoint at "my" code and just run, and then I wouldn't notice.

Suspecting (?&PerlOWS)was a logic flaw on my side, sorry for that.

I'm closing this issue, thanks for the quick fix!