Panick on comments cache creation

[bryango]

Coming from: https://github.com/NixOS/nixpkgs/pull/280617 Tested nixpkgs revision: https://github.com/NixOS/nixpkgs/commit/7519d9d92b95c1c4face57969956dc78f787fb39

It looks like something in nixpkgs breaks the parser... (or more precisely, the parser cannot handle some expression in nixpkgs).

$ ~/result/bin/manix mergeAttrs
Building Nixpkgs comments cache...
thread '<unnamed>' panicked at src/comments_docsource.rs:194:60:
called `Result::unwrap()` on an `Err` value: UnexpectedWanted(TOKEN_AT, 220..221, [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT])
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Full back-trace is here.

```console $ RUST_BACKTRACE=full ~/result/bin/manix mergeAttrs Building Nixpkgs comments cache... thread '' panicked at src/comments_docsource.rs:194:60: called `Result::unwrap()` on an `Err` value: UnexpectedWanted(TOKEN_AT, 220..221, [TOKEN_L_PAREN, TOKEN_REC, TOKEN_L_BRACE, TOKEN_L_BRACK, TOKEN_STRING_START, TOKEN_IDENT]) stack backtrace: 0: 0x555ef67c638b - ::fmt::h85e32f8f7b08f729 1: 0x555ef67fd890 - core::fmt::write::hae1cf5cde4ee2188 2: 0x555ef67bb2d5 - std::io::Write::write_fmt::h4ae54bff47727b29 3: 0x555ef67c6164 - std::sys_common::backtrace::print::hea248a594c5888cf 4: 0x555ef67c6db7 - std::panicking::default_hook::{{closure}}::h545cae1ad29f25d2 5: 0x555ef67c6b04 - std::panicking::default_hook::hc1d7fee5faf0cc8a 6: 0x555ef67c7318 - std::panicking::rust_panic_with_hook::h0e7975574d129b7c 7: 0x555ef67c71fe - std::panicking::begin_panic_handler::{{closure}}::h420dcc10a77211ab 8: 0x555ef67c65a6 - std::sys_common::backtrace::__rust_end_short_backtrace::he6597460302ecd4a 9: 0x555ef67c6f62 - rust_begin_unwind 10: 0x555ef66ff025 - core::panicking::panic_fmt::h3cf3338e18ada46e 11: 0x555ef66fefe3 - core::result::unwrap_failed::h22597ad5fa95281f 12: 0x555ef6769aaf - ::update::{{closure}}::h88bbede1ba90a798 13: 0x555ef676977a - as rayon::iter::plumbing::Folder>::consume::h8085d629cdb40bbf 14: 0x555ef6766d22 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 15: 0x555ef677c0d2 - rayon_core::job::StackJob::run_inline::he56845543e1811ba 16: 0x555ef6776c56 - rayon_core::join::join_context::{{closure}}::hda1efde214f4edf5 17: 0x555ef6766e19 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 18: 0x555ef6776a86 - rayon_core::join::join_context::{{closure}}::hda1efde214f4edf5 19: 0x555ef6766e19 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 20: 0x555ef6776a86 - rayon_core::join::join_context::{{closure}}::hda1efde214f4edf5 21: 0x555ef6766e19 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 22: 0x555ef677c0d2 - rayon_core::job::StackJob::run_inline::he56845543e1811ba 23: 0x555ef6776c56 - rayon_core::join::join_context::{{closure}}::hda1efde214f4edf5 24: 0x555ef6766e19 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 25: 0x555ef6776a86 - rayon_core::join::join_context::{{closure}}::hda1efde214f4edf5 26: 0x555ef6766e19 - rayon::iter::plumbing::bridge_producer_consumer::helper::h4157951f8fb89844 27: 0x555ef677e0af - as rayon_core::job::Job>::execute::hb6e105b752872627 28: 0x555ef66faa58 - rayon_core::registry::WorkerThread::wait_until_cold::hc583e99eff65099f 29: 0x555ef67a7302 - rayon_core::registry::ThreadBuilder::run::h6c241f9408aa5070 30: 0x555ef67a9e7a - std::sys_common::backtrace::__rust_begin_short_backtrace::h1c1aa5733d7dad65 31: 0x555ef67ab8ef - core::ops::function::FnOnce::call_once{{vtable.shim}}::hd1024c89ad043895 32: 0x555ef67e1805 - std::sys::unix::thread::Thread::new::thread_start::h198fdf616b50df6b 33: 0x7f0d95c07333 - start_thread 34: 0x7f0d95c89efc - __clone3 35: 0x0 - ```

[IogaMaster]

https://github.com/mlvzk/manix/issues/31#issuecomment-1820027197

@akarui-hikarii I have attempted to add this into src/comments_docsource.rs But it causes a lot of issues, do you have any insight?

[IogaMaster]

This seems to have to do with the rnix::Root::parse function. It seems that rnix assumes all code is correct and valid.

[bryango]

This seems to have to do with the rnix::Root::parse function. It seems that rnix assumes all code is correct and valid.

I see... But does that mean that there is a hidden syntax error in nixpkgs? Weird... Probably a bug in rnix?

[bayou-brogrammer]

https://github.com/mlvzk/manix/issues/31#issuecomment-1820027197

@akarui-hikarii I have attempted to add this into src/comments_docsource.rs But it causes a lot of issues, do you have any insight?

Taking a look today :)

[bayou-brogrammer]

Seems to be an issue with one package in nixpkgs.

https://github.com/NixOS/nixpkgs/blob/806f3ff2165447616972483603107e50ae6499dd/pkgs/development/compilers/flutter/update/get-pubspec-lock.nix#L12

This fix is to just allow the parser to continue and log the error

[bayou-brogrammer]

Fixed in #3

[bryango]

Oh my, that's fast! Thank you!

Seems to be an issue with one package in nixpkgs.

NixOS/nixpkgs@806f3ff/pkgs/development/compilers/flutter/update/get-pubspec-lock.nix#L12

I think they should have quoted the string... I will try a nixpkgs PR to fix this.