Crash when parallel render; not thread safe? Dies with X::HyperRace::Died in test case

[softmoth]

The benchmarking test case in t/11-iterables.t is failing:

An operation first awaited:
  in block  at t/11-iterable.t line 71
  in block <unit> at t/11-iterable.t line 38

Died with the exception:
    A worker in a parallel iteration (hyper or race) initiated here:
      in block  at lib/Template/Mustache.rakumod (Template::Mustache) line 577
      in block  at lib/Template/Mustache.rakumod (Template::Mustache) line 566
      in sub format at lib/Template/Mustache.rakumod (Template::Mustache) line 537
      in sub format at lib/Template/Mustache.rakumod (Template::Mustache) line 397
      in sub format at lib/Template/Mustache.rakumod (Template::Mustache) line 397
      in method render at lib/Template/Mustache.rakumod (Template::Mustache) line 295
      in block  at t/11-iterable.t line 57

    Died at:
        Cannot invoke this object (REPR: Uninstantiable; Callable)

Unfortunately this was not caught because the test case is run only when TEST_BENCHMARK=1 (or TEST_ALL=1). While that is set on the Travis CI runs, there aren't enough CPU cores when running there to run the test. So I am not sure when this started failing.

At line 577 of Template/Mustache.rakudoc it essentially calls .map(&format_section).join. There may be some shared state happening inside the format() routine that could be pulled out to outside of the map, if that is what's needed.

The test case is complicated, and of course the rendering function itself is very complex, so it may take some time to get an isolated test case for this issue. And I don't have a lot of experience with hyper/race handling. Any suggestions or links to relevant docs are very welcome!

[Xliff]

Odd. I get something different:

An operation first awaited:
  in block  at t/11-iterable.t line 71
  in block <unit> at t/11-iterable.t line 38

Died with the exception:
    A worker in a parallel iteration (hyper or race) initiated here:
      in block  at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 577
      in block  at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 566
      in sub format at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 537
      in sub format at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 397
      in sub format at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 397
      in method render at /home/cbwood/.zef/store/Template-Mustache-1.2.3.tar.gz/Template-Mustache-1.2.3/lib/Template/Mustache.rakumod (Template::Mustache) line 295
      in block  at t/11-iterable.t line 57

    Died at:
        Cannot invoke object with invocation handler in this context

This is from Raku v2021.02.1-64-gbf67b1f71

[softmoth]

Yes, Xliff, I get the same error as you. I'd accidentally switched to an older Raku and forgot to switch back before reporting.

[softmoth]

This bug is triggered by calling .Str or .gist on the HyperSeq prior to calling .map. In Template::Mustache, the call to .Str is in sub resolve, and removing line 426 causes the test case to succeed.

This is a golfed down test case:

my $hyp = (gather { take $(:line("line 1")) }).hyper;

# XXX THIS TRIGGERS THE SUBSEQUENT CRASH
$hyp.Str;

my $result = $hyp.map({ .raku }).join;

put "RESULT «{$result}»";

What remains to be resolved is whether this is a bug in Template::Mustache or in Rakudo.

[softmoth]

Workaround pushed as b1fbd76. I don't know what the right solution really is, so leaving this open.

[Xliff]

Nice job, @softmoth!

[2colours]

What's the current state of the issue?

[softmoth]

It's not affecting anyone as far as I know, and I suspect it's a generic gotcha with Raku rather than an actual bug. But I've left this bug open as a warning and reminder to any future maintainer that something brittle and poorly understood may be going on. And as an invitation for anyone who knows better to proffer an improvement.