Broken pipe on next_release

[arcnmx]

I'm trying to use the next-release branch because I wanted to try the new -O option but am encountering the following error:

# echo hi | page
Vim: Error reading input, exiting...

Vim: Finished.
thread 'main' panicked at 'Error sending message: Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }', /build/page-2.3.5-vendor.tar.gz/neovim-lib/src/rpc/client.rs:188:36
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I've tried both fb3b36f7b3a17e0537700bcc9480510440c4d67c and e7bf0eeac38eb9d9ab2d4fbdea91af0afa8ed898 but am unable to get it working. 2.3.5 is working fine however. Any idea what I'm doing wrong or is the branch just currently not usable?

(could it be neovim 0.6 is the problem?)

[I60R]

Any idea what I'm doing wrong

Your error contains neovim-lib but on next-release I've replaced that library with nvim-rs, so you probably use an older version. How did you install it?

is the branch just currently not usable?

It's usable and the problem should be fixed on it. I'm still thinking about changing some flags and migrating from viml to lua before issuing a next release (probably would be 3.0.0)

could it be neovim 0.6 is the problem?

Nope, happened for me on 0.5.2 as well, however, the version I've used previously worked fine.

The error was caused by prefix | in vim.cmd which crashed neovim with e749: empty buffer but only when it was spawned by page, so I've had a fun time when debugging this and that's how the migration to nvim-rs was started.

[arcnmx]

Well I tried both versions (I tried the older commit because I was hoping maybe the switch to nvim-rs wasn't complete yet). With fb3b36f7b3a17e0537700bcc9480510440c4d67c the error is similar but:

Vim: Error reading input, exiting...

Vim: Finished.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: SendError(WriterError(Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }), "nvim_get_current_buf")', src/neovim.rs:66:88
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[I60R]

Ok, this seems to be a different issue. I'm able to reproduce it but only when page's stderr is redirected to something. Trying to fix that

[I60R]

Should be fixed by ab1f6da43ebe60781a5cfbe07948c9155afc3664 let me know if it works for you

[arcnmx]

Nope, but it's inconsistent (some runs it works as expected, but most others it crashes).

thread 'main' panicked at 'Cannot open page PTY: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/page.rs:433:99
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[I60R]

3c9cbdf88a51c584adeeb84a729bc46f995e856b may fix it, could you try?

[arcnmx]

Sorry, completely broken; I get an empty buffer (only containing [Process exited 122]) instead of one containing any contents, and then when I :q it freezes for 2 seconds and then crashes:

thread 'main' panicked at 'Cannot open page PTY: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/page.rs:443:33
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

[I60R]

Maybe some plugin crashes or takes a long time to load and causes problems? Could you test with page -c NONE?

Also, is there something special about your setup? What is the version of /bin/sleep? What terminal emulator you use?

[I60R]

It would be as well helpful if you run RUST_LOG=trace RUST_BACKTRACE=full page 2> page_log then cat page_log and paste the output here

[arcnmx]

Could you test with page -c NONE?

Same issue. About 1 run in 10 does work though.

Also, is there something special about your setup? What is the version of /bin/sleep? What terminal emulator you use?

If you feel up to it you can reproduce a lot of my setup with nix:

arcnix() {
  nix-shell -E \
    -I nixpkgs=https://nixos.org/channels/nixos-unstable-small/nixexprs.tar.xz \
    -I arc=https://github.com/arcnmx/nixexprs/archive/next.tar.gz \
    "$@"
}
arcnix --pure \
  'with (import <arc> {}).pkgs; mkShell { nativeBuildInputs = [ page-develop neovim-unwrapped coreutils ]; }' \
  --run 'echo hi | page -c NONE'

# if you really want a term too...
arcnix --pure \
  'with (import <arc> {}).pkgs; mkShell { nativeBuildInputs = [ rxvt-unicode-arc page-develop neovim-unwrapped coreutils ]; }' \
  --run 'urxvt -e sh -c "echo hi | page -c NONE"'

(remove --pure and page-develop if you want to substitute in your own from $PATH or use ./target/page or whatever)

versions of things:

rxvt-unicode (urxvt) v9.26 - released: 2021-05-14
sleep (GNU coreutils) 9.0
zsh 5.8 (x86_64-pc-linux-gnu)
starship prompt

NVIM v0.6.0
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by nixbld

It would be as well helpful if you run RUST_LOG=trace RUST_BACKTRACE=full page

page.log

[I60R]

Thank you very much! Now I'm see from where the problem comes: for some reason neovim did returned an empty PTY, which is really strange. I will add check for that as well as will try to reproduce and fix the problem

[I60R]

Even with nix-shell I'm unable to reproduce the issue, so it's probably specific to your machine, kernel version etc.

Anyway, since I know from where the error originates from and what most likely causes it I've implemented another workaround which may finally solve the issue. Could you try the last commit on next-release again and post logs here?

[arcnmx]

oh damn... Well here's be0b09de7571162d757743822f4331c0e26e404c

page.log

[I60R]

Maybe lua will help...

[arcnmx]

Maybe lua will help...

Four words no one ever wants to have to hear...

page.log

[I60R]

Page uses /bin/sleep to open PTY in terminal buffer, but on your system sleep probably isn't stored in /bin, so I've pushed another commit where the command was changed to just sleep. Maybe this will solve the problem?

[arcnmx]

Oh yeah that would do it. sleep is in $PATH of course, but /bin doesn't even really exist. 542de30d4e0b6120373ed10fcfaa14f63123b58c seems to be okay at first glance!

[arcnmx]

It does seem to be a lot better now! Though...

if I look at long input, it says -- [PAGE] 99999 lines read; has more --, and :q, it stalls for a while after closing nvim and then says:

thread 'main' panicked at 'Cannot notify end of input: SendError(WriterError(Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }), "nvim_exec_lua")', src/neovim.rs:366:47
stack backtrace:
   0:     0x558fce966aa4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1ef07120adb99796
   1:     0x558fce8d644c - core::fmt::write::h59cc4b3dea155cce
   2:     0x558fce94ed74 - std::io::Write::write_fmt::he7e4fb4b70c5207a
   3:     0x558fce95c3b0 - std::panicking::default_hook::{{closure}}::h67c88fba3a1757d9
   4:     0x558fce95c016 - std::panicking::default_hook::had51536d17f32530
   5:     0x558fce9137a4 - page::_init_panic_hook_::{{closure}}::hdab56c451dcc4904
   6:     0x558fce95cbcf - std::panicking::rust_panic_with_hook::hcdfd560c4f6afa56
   7:     0x558fce966e88 - std::panicking::begin_panic_handler::{{closure}}::h5e7abb77afe4762c
   8:     0x558fce966e06 - std::sys_common::backtrace::__rust_end_short_backtrace::hb15a1d01c910931c
   9:     0x558fce95c6e2 - rust_begin_unwind
  10:     0x558fce85e820 - core::panicking::panic_fmt::h4b7f7457475b4299
  11:     0x558fce85e732 - core::result::unwrap_failed::hcd8e16a338c4f01c
  12:     0x558fce88e327 - page::manage_output_buffer::{{closure}}::hb92138c171a6982a
  13:     0x558fce881028 - page::connect_neovim::{{closure}}::hbb9373bafa6d2ace
  14:     0x558fce86f443 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h19485ebf14ad3d84
  15:     0x558fce9154e9 - page::main::hcd433cd4d8c25dec
  16:     0x558fce925053 - std::sys_common::backtrace::__rust_begin_short_backtrace::he32ba23cc8fda64b
  17:     0x558fce913b03 - main
  18:     0x7fe4aeb3b780 - __libc_start_main
  19:     0x558fce86724a - _start

if I instead :q before it manages to fill the buffer and display the above notification, it instead crashes with:

thread 'main' panicked at 'Cannot write next line: Os { code: 5, kind: Uncategorized, message: "Input/output error" }', src/page.rs:398:83
stack backtrace:
   0:     0x55565918aaa4 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h1ef07120adb99796
   1:     0x5556590fa44c - core::fmt::write::h59cc4b3dea155cce
   2:     0x555659172d74 - std::io::Write::write_fmt::he7e4fb4b70c5207a
   3:     0x5556591803b0 - std::panicking::default_hook::{{closure}}::h67c88fba3a1757d9
   4:     0x555659180016 - std::panicking::default_hook::had51536d17f32530
   5:     0x5556591377a4 - page::_init_panic_hook_::{{closure}}::hdab56c451dcc4904
   6:     0x555659180bcf - std::panicking::rust_panic_with_hook::hcdfd560c4f6afa56
   7:     0x55565918ae88 - std::panicking::begin_panic_handler::{{closure}}::h5e7abb77afe4762c
   8:     0x55565918ae06 - std::sys_common::backtrace::__rust_end_short_backtrace::hb15a1d01c910931c
   9:     0x5556591806e2 - rust_begin_unwind
  10:     0x555659082820 - core::panicking::panic_fmt::h4b7f7457475b4299
  11:     0x555659082732 - core::result::unwrap_failed::hcd8e16a338c4f01c
  12:     0x5556590b22aa - page::manage_output_buffer::{{closure}}::hb92138c171a6982a
  13:     0x5556590a5028 - page::connect_neovim::{{closure}}::hbb9373bafa6d2ace
  14:     0x555659093443 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h19485ebf14ad3d84
  15:     0x5556591394e9 - page::main::hcd433cd4d8c25dec
  16:     0x555659149053 - std::sys_common::backtrace::__rust_begin_short_backtrace::he32ba23cc8fda64b
  17:     0x555659137b03 - main
  18:     0x7f3f16968780 - __libc_start_main
  19:     0x55565908b24a - _start
  20:                0x0 - <unknown>

EDIT: new crash when used as a $MANPAGER:

thread 'main' panicked at 'Cannot switch back to active buffer: NeovimError(Some(1), "Invalid buffer id: 2")', src/page.rs:431:70                                                     
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
/etc/profiles/per-user/arc/bin/man: command exited with status 101: sed -e '/^[[:space:]]*$/{ N; /^[[:space:]]*\n[[:space:]]*$/D; }' | LESS=-ix8RmPm Manual page zstd(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$PM Manual page zstd(1) ?ltline %lt?L/%L.:byte %bB?s/%s..?e (END):?pB %pB\%.. (press h for help or q to quit)$-KRXMfnq MAN_PN=zstd(1) /nix/store/db6wf3ah5q66jk7lmsl8987p30yfm78i-page/bin/page -C -e au User PageDisconnect sleep 100m|%y p|enew! |bd! #|pu p|set ft=man

(for now I'll just use MANPAGER=$PAGER)

[I60R]

if I look at long input, it says -- [PAGE] 99999 lines read; has more --, and :q, it stalls for a while after closing nvim and then says: ... new crash when used as a $MANPAGER: ...

Should be fixed on last commit.

if I look at long input, it says -- [PAGE] 99999 lines read; has more --, and :q, it stalls for a while after closing nvim and then says: ...

This is somewhat expected, since user should know that some input wasn't displayed and why. If I would figure out a proper way to determine whether this error occurs because neovim was closed or crashed unexpectedly then this would be handled more gracefully

[arcnmx]

This is somewhat expected, since user should know that some input wasn't displayed and why

Hm, but the user is fully aware that some input wasn't displayed - that's often the point of a pager, to only display a select portion of a document or log! When I'm viewing the data in nvim sure, I want to be able to tell if I'm at EOF or not - but once I close a manpage or a massive git log after looking up the info I needed, I would not want to be reminded that I didn't scroll all the way to the bottom (the act of quitting indicates that I'm done with the contents and no longer interested in the rest of the output).

If I would figure out a proper way to determine whether this error occurs because neovim was closed or crashed unexpectedly then this would be handled more gracefully

Yeah, it can be tricky in the cases where something actually does go wrong. Though if the pager wasn't closed explicitly by the user, presumably there would already be a log or something output by neovim itself (or whatever else caused it to fail unexpectedly) displayed on the screen. It's also possible that the program emitting the output itself will print its own "broken pipe" error message, even. I'm not really convinced that page would need to add to the output?

431f67947dd598090dc1cb1f0810c5a7cc47c3b4 does seem to fix the first example, thanks! For reference here's an easy reproduction test case for the second example:

{ seq 100; sleep 5; } | page
`:quit` # within 5 seconds
thread 'main' panicked at 'Cannot notify query finished: SendError(WriterError(Os { code: 32, kind: BrokenPipe, message: "Broken pipe" }), "nvim_exec_lua")', src/neovim.rs:357:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

# Or a variant that writes more data after quit:
{ seq 100; sleep 5; echo; } | page

thread 'main' panicked at 'Cannot write next line: Os { code: 5, kind: Uncategorized, message: "Input/output error" }', src/page.rs:402:83
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

EDIT: found one more issue with next_release: running page /some/path/to/file results in a blank (with a bunch of newlines?) term buffer. You can :bd! to see the real contents in another buffer.

[I60R]

Ok, panics during reading should be fixed, at least it works for me. This was relatively easy fix thanks to nvim-rs and async

EDIT: found one more issue with next_release: running page /some/path/to/file results in a blank (with a bunch of newlines?) term buffer. You can :bd! to see the real contents in another buffer.

Cannot reproduce the issue. Does it still happens on latest next_release commit?

[I60R]

Pushed another one which should resolve the issue with blank buffer

[arcnmx]

Unfortunately...

seq 1 50 > /tmp/whee.txt
page -q 999 -c NONE /tmp/whee.txt
[ 0000000117 | WARN  | usage in page ]
Query (-q) is ignored when page doesn't read input from pipe

/run/user/1000/neovim-page/DO-NOT-REDIRECT-OUTSIDE-OF-NVIM-TERM(--help[-W])

Now still results in a blank buffer, but there are no actual contents underneath. I have PAGER set to always pass -q, thus the use of it despite it not being piped input. (note that despite the warning, the presence of -q appears to cause the problem)

[I60R]

Ok, it seems to be without reason to imply output buffer creation with -q option. Fixed