.next-handle seems to change handles but doesn't

[p6rt]

Migrated from rt.perl.org#132885 (status was 'open')

Searchable as RT132885$

[p6rt]

From @briandfoy

#!/Users/brian/bin/perl6s/perl6-latest

I'm playing with .next-handle from IO​::CatHandle. I'm trying to create a situation where I can read just the first five lines from each command line argument​:

  quietly {   my $limit = 5;   for lines() {   state $lines = 1;   FIRST { $*ARGFILES.on-switch = { put "NEW FILE"; $lines = 1 } }   if $lines > $limit {   $*ARGFILES.next-handle;   next;   }   put "{$*ARGFILES.path}​:{$lines++} $_";   }   }

Here's a test file​:

  First   Second   Third   Fourth   Fifth   Six   Seventh

With one or more command-line arguments I get this odd behavior (and lots of warnings that I suppressed)​:

  test.txt​:1 First   test.txt​:2 Second   test.txt​:3 Third   test.txt​:4 Fourth   test.txt​:5 Fifth   NEW FILE   :1 Seventh   read bytes requires an object with REPR MVMOSHandle (got VMNull with REPR Null)   in block \ at lines-method.p6 line 5

It does "switch" after five lines, but then it keeps reading from the same handle while losing a line. Then there's a strange error at the end that kills the whole thing.

I expected that it would close the current handle, open a new one, and continue. If it gets to the end, it would simply not provide any more lines() and things would end normally.

The docs for .next-handle lets you keep changing it as long as you like no matter how many

---

This is Rakudo Star version 2018.01 built on MoarVM version 2018.01 implementing Perl 6.c.

[p6rt]

From @briandfoy

I can fix this by closing the old file handle and checking the new one, but that seems like way to much work at the user level.

  quietly {   my $limit = 5;   for lines() {   state $lines = 1;   FIRST { $*ARGFILES.on-switch = { put "NEW FILE"; $lines = 1 } }   if $lines > $limit {   $*ARGFILES.next-handle.close;   last unless $*ARGFILES.opened;   next;   }   put "{$*ARGFILES.path}​:{$lines++} $_";   }   }

[p6rt]

From @zoffixznet

On Mon, 19 Feb 2018 16​:15​:37 -0800, comdog wrote​:

#!/Users/brian/bin/perl6s/perl6-latest

I'm playing with .next-handle from IO​::CatHandle. I'm trying to create a situation where I can read just the first five lines from each command line argument​:

quietly \{
    my $limit = 5;
    for lines\(\) \{
        state $lines = 1;
        FIRST \{ $\*ARGFILES\.on\-switch = \{ put "NEW FILE"; $lines = 1 \} \}
        if $lines > $limit \{
            $\*ARGFILES\.next\-handle;
            next;
            \}
        put "\{$\*ARGFILES\.path\}​:\{$lines\+\+\} $\_";
        \}
    \}

Here's a test file​:

First
Second
Third
Fourth
Fifth
Six
Seventh

With one or more command-line arguments I get this odd behavior (and lots of warnings that I suppressed)​:

test\.txt​:1 First
test\.txt​:2 Second
test\.txt​:3 Third
test\.txt​:4 Fourth
test\.txt​:5 Fifth
NEW FILE
:1 Seventh
read bytes requires an object with REPR MVMOSHandle \(got VMNull

with REPR Null) in block \ at lines-method.p6 line 5

It does "switch" after five lines, but then it keeps reading from the same handle while losing a line. Then there's a strange error at the end that kills the whole thing.

I expected that it would close the current handle, open a new one, and continue. If it gets to the end, it would simply not provide any more lines() and things would end normally.

The docs for .next-handle lets you keep changing it as long as you like no matter how many

-----

This is Rakudo Star version 2018.01 built on MoarVM version 2018.01 implementing Perl 6.c.

This is because current implementation of IO​::CatHandle.lines simply flattens a gather/taken .lines Seq from each handle, so it never expects the handles to get switched before each .lines Seq is fully-consumed. The .next-handle closes the previous handle, which is where the weird error comes from (it tries to use the now-nulled $!PIO to .read/.eof from). Pretty sure .words is similarly affected.

I think if we give IO​::CatHandle!LINES and IO​::CatHandle!WORDS their own iterators that basically do the same thing as IO​::Handle ones and then make .next-handle also toss the remaining data in the $!decoder, then the OP code would work.

In the meantime, you can use `.get` instead​:

  my $limit := 3;   my $lines = 1;   with $*ARGFILES -> $af {   while ($_ := $af.get) !=​:= Nil {   $lines == 1 and put "NEWFILE​:";   put "$af.path()​:{$lines++} $_";   next unless $lines > $limit;   $af.next-handle and $lines = 1;   }   }

I also noticed another deficiency​: when trying to set .on-switch on $*ARGFILES, you always miss its very first call that's done by IO​::CatHandle.new, since to give you $*ARGFILES, .new has already been called.

Not sure what can be done about it, but I filed it as https://github.com/rakudo/rakudo/issues/1545 IO​::CatHandle is not yet part of any language, so we have the ability to polish the rough edges.

[p6rt]

The RT System itself - Status changed from 'new' to 'open'

[p6rt]

From @zoffixznet

On Mon, 19 Feb 2018 19​:48​:30 -0800, cpan@​zoffix.com wrote​:

On Mon, 19 Feb 2018 16​:15​:37 -0800, comdog wrote​:

#!/Users/brian/bin/perl6s/perl6-latest

I'm playing with .next-handle from IO​::CatHandle. I'm trying to create a situation where I can read just the first five lines from each command line argument​:

quietly { my $limit = 5; for lines() { state $lines = 1; FIRST { $*ARGFILES.on-switch = { put "NEW FILE"; $lines = 1 } } if $lines > $limit { $*ARGFILES.next-handle; next; } put "{$*ARGFILES.path}​:{$lines++} $_"; } }

Here's a test file​:

First Second Third Fourth Fifth Six Seventh

With one or more command-line arguments I get this odd behavior (and lots of warnings that I suppressed)​:

test.txt​:1 First test.txt​:2 Second test.txt​:3 Third test.txt​:4 Fourth test.txt​:5 Fifth NEW FILE :1 Seventh read bytes requires an object with REPR MVMOSHandle (got VMNull with REPR Null) in block \ at lines-method.p6 line 5

It does "switch" after five lines, but then it keeps reading from the same handle while losing a line. Then there's a strange error at the end that kills the whole thing.

I expected that it would close the current handle, open a new one, and continue. If it gets to the end, it would simply not provide any more lines() and things would end normally.

The docs for .next-handle lets you keep changing it as long as you like no matter how many

-----

This is Rakudo Star version 2018.01 built on MoarVM version 2018.01 implementing Perl 6.c.

This is because current implementation of IO​::CatHandle.lines simply flattens a gather/taken .lines Seq from each handle, so it never expects the handles to get switched before each .lines Seq is fully-consumed. The .next- handle closes the previous handle, which is where the weird error comes from (it tries to use the now-nulled $!PIO to .read/.eof from). Pretty sure .words is similarly affected.

I think if we give IO​::CatHandle!LINES and IO​::CatHandle!WORDS their own iterators that basically do the same thing as IO​::Handle ones and then make .next-handle also toss the remaining data in the $!decoder, then the OP code would work.

In the meantime, you can use `.get` instead​:

my $limit := 3; my $lines = 1; with $*ARGFILES -> $af { while ($_ := $af.get) !=​:= Nil { $lines == 1 and put "NEWFILE​:"; put "$af.path()​:{$lines++} $_"; next unless $lines > $limit; $af.next-handle and $lines = 1; } }

I also noticed another deficiency​: when trying to set .on-switch on $*ARGFILES, you always miss its very first call that's done by IO​::CatHandle.new, since to give you $*ARGFILES, .new has already been called.

Not sure what can be done about it, but I filed it as https://github.com/rakudo/rakudo/issues/1545 IO​::CatHandle is not yet part of any language, so we have the ability to polish the rough edges.

Side note​: it could be handy to be able to loop over IO​::CatHandle's handles. The OP code would just be​:

  for $*ARGFILES.handles -> $fh {   put "NEWFILE​:";   put "$fh.path()​:$(++$) $_" for $fh.lines​: 3;   }

Filed that as https://github.com/rakudo/rakudo/issues/1546