search

lizmat / App-Rak

21st century grep / find / ack / ag / rg on steroids
Artistic License 2.0
152 stars 7 forks source link

--modify-files crashes or produces wrong results #60

Open greenfork opened 3 weeks ago

greenfork commented 3 weeks ago

I have encountered this strange behavior with rak, here is reproduction https://github.com/greenfork/rak-modify-interl

greenfork commented 3 weeks ago

Also versions

> rak --version
rak - provided by App::Rak 0.3.12, running Raku 6.d with Rakudo 2024.07.
lizmat commented 3 weeks ago

It appears App::Rak is tickling some underlying issue (possibly multiple ones) in MoarVM and/or NQP and/or Rakudo. Am investigating at the moment.

If you specify --degree, it should not happen (albeit at the cost of being slower).

greenfork commented 3 weeks ago

Yeah, --degree helps, thank you! Good luck investigating. Let me know when I can remove the repo with reproduction.

lizmat commented 3 weeks ago

You can remove that repo. Sadly I have plenty of other reproductions :-(

lizmat commented 3 weeks ago

I've just uploaded 0.3.13 which should hopefully fix your query, albeit at the expense of being less asynchronous for big files.

I don't fully understand yet how this happens, but this should be a lot stabler. Please let me know if this didn't fix it.

Also, today I uncovered an issue in Rakudo with the use of / foo /, aka running a regex on the topic in a multi-threaded manner. This could lead to strange disatch errors. Submodules of App::Rakhave been updated to make sure that that cannot happen anymore.

greenfork commented 3 weeks ago

I still see bogus results from the command and it crashes about 20% of the time. Command for reference:

rak --modify-files '{ .subst(/«next»/, "return") if .match(/«next»/)}' jobs/

One of my attempts got this stack trace, maybe it helps:

MoarVM oops: MVM_str_hash_fetch_nocheck called with a stale hashtable pointer
   at src/Perl6/Metamodel/MROBasedMethodDispatch.nqp:13  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/Metamodel.moarvm:find_method)
 from src/vm/moar/dispatchers.nqp:1122  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/BOOTSTRAP/v6c.moarvm:)
 from SETTING::src/core.c/Rakudo/SlippyIterator.rakumod:81  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:slip-all)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/E399776C8F1A3226E3347E20A6BDC43053862F3F (rak):1414  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/E3/E399776C8F1A3226E3347E20A6BDC43053862F3F:)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):193  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:run)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):1636  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):789  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
MoarVM oops: MVM_str_hash_fetch_nocheck called with a stale hashtable pointer
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:905  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:272  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:253  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:250  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-one)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:291  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Thread.rakumod:69  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:THREAD-ENTRY)

   at src/Perl6/Metamodel/MROBasedMethodDispatch.nqp:13  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/Metamodel.moarvm:find_method)
 from src/vm/moar/dispatchers.nqp:1122  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/lib/Perl6/BOOTSTRAP/v6c.moarvm:)
 from SETTING::src/core.c/Rakudo/SlippyIterator.rakumod:81  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:slip-all)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/E399776C8F1A3226E3347E20A6BDC43053862F3F (rak):1414  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/E3/E399776C8F1A3226E3347E20A6BDC43053862F3F:)
 from SETTING::src/core.c/Any-iterable-methods.rakumod:375  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:push-all)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):193  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:run)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):1636  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from site#sources/140A8F9931ADE3D89A19D7E16699DF4A691FB614 (ParaSeq):789  (/home/grfork/.rakubrew/versions/moar-2024.07/share/perl6/site/precomp/F00CDEB998415F4FF2C4EC7C1C647386FBD38CF9/14/140A8F9931ADE3D89A19D7E16699DF4A691FB614:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:905  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:272  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:253  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:250  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:run-one)
 from SETTING::src/core.c/ThreadPoolScheduler.rakumod:291  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:)
 from SETTING::src/core.c/Thread.rakumod:69  (/home/grfork/.rakubrew/versions/moar-2024.07/bin/../share/perl6/runtime/CORE.c.setting.moarvm:THREAD-ENTRY)
lizmat commented 3 weeks ago

Thanks! That is helpful! I will be afk most of today, so it will probably be tomorrow before I can look at it.

One note: '*.subst(/«next»/, "return") is functionally equivalent to '{ .subst(/«next»/, "return") if .match(/«next»/)}' as .subst will return the string unchanged if it couldn't do the substitution.

greenfork commented 3 weeks ago

That is a very nice trick, I haven't thought about it, thank you!

lizmat commented 2 weeks ago

OOC, did this happen right after you started this run, or had it been busy for a while already?

greenfork commented 2 weeks ago

I'd say it's rather when started. But it is hard to say because VM takes time to start, maybe it was in the middle. The replacement itself takes very little time