Closed aweis89 closed 1 month ago
can you show the error message?
Do you happen to have tiktoken.lua installed? It's difficult to imagine Lua would segfault.
I also experienced segfaults. They originated in tiktoken. I had tried installing binaries and compiling it on ubuntu 24.04. Both same result (segfault). What worked was uninstalling neovim and luarocks with apt and installing them with brew. Then compiling tiktoken using the luarocks that was installed in brew.
edit: turns out when I compiled it with brew it was installed, but not found. So copilotchat now works without it. Not sure what is even the benefit of counting the tokens?
Yea i experienced same issue when I installed tiktoken binary on new notebook, definitely big issue, @gptlang any idea?
Output from env RUST_BACKTRACE=1 nvim --headless
:
thread '<unnamed>' panicked at src/lib.rs:222:41:
called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::result::unwrap_failed
3: tiktoken_core::new
4: mlua::lua::Lua::create_function::{{closure}}
5: mlua::lua::Lua::create_callback::call_callback
6: <unknown>
7: lua_pcall
8: nlua_pcall
9: <unknown>
10: <unknown>
11: <unknown>
12: <unknown>
13: uv_run
14: <unknown>
15: loop_poll_events
16: process_wait
17: <unknown>
18: os_call_shell
19: call_shell
20: os_expand_wildcards
21: gen_expand_wildcards
22: expand_wildcards
23: expand_wildcards_eval
24: <unknown>
25: <unknown>
26: <unknown>
27: ExpandOne
28: <unknown>
29: call_internal_func
30: call_func
31: nlua_call
32: <unknown>
33: <unknown>
34: <unknown>
35: lua_pcall
36: nlua_pcall
37: nlua_exec_file
38: do_source
39: <unknown>
40: <unknown>
41: main
42: <unknown>
43: __libc_start_main
44: _start
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
[1] 3847543 segmentation fault (core dumped) env RUST_BACKTRACE=1 nvim --headless
This is what I got, for RUST_BACKTRACE=full:
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x7fe7076288bb - std::sys_common::backtrace::_print_fmt::hf9c799a68b2477fc
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:67:5
3: 0x7fe7076288bb - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hf2d48dfeb5948660
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:44:22
4: 0x7fe707660fd0 - core::fmt::rt::Argument::fmt::h507ca91fbb1ef494
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/core/src/fmt/rt.rs:142:9
5: 0x7fe707660fd0 - core::fmt::write::h4d5f6025aa566322
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/core/src/fmt/mod.rs:1120:17
6: 0x7fe70762661d - std::io::Write::write_fmt::hc088b3c8cc6dcb9f
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/io/mod.rs:1762:15
7: 0x7fe7076286a4 - std::sys_common::backtrace::_print::h231201f6cff8f56a
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:47:5
8: 0x7fe7076286a4 - std::sys_common::backtrace::print::h478d6be486f8cd2d
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:34:9
9: 0x7fe7076440ca - std::panicking::default_hook::{{closure}}::hf1c55106b4d45a90
10: 0x7fe707643d6d - std::panicking::default_hook::h7b29000c631cc719
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/panicking.rs:292:9
11: 0x7fe707644408 - std::panicking::rust_panic_with_hook::h2a988919dd121561
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/panicking.rs:779:13
12: 0x7fe707628c9e - std::panicking::begin_panic_handler::{{closure}}::h3a9eaa1d23ecf555
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/panicking.rs:657:13
13: 0x7fe707628ad6 - std::sys_common::backtrace::__rust_end_short_backtrace::h813561fdec15f700
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/sys_common/backtrace.rs:170:18
14: 0x7fe707644132 - rust_begin_unwind
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/std/src/panicking.rs:645:5
15: 0x7fe707512e55 - core::panicking::panic_fmt::hef0307862026e6f9
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/core/src/panicking.rs:72:14
16: 0x7fe707513563 - core::result::unwrap_failed::h8393ef8d489acb45
at /build/rustc-Px5Ywq/rustc-1.75.0+dfsg0ubuntu1/library/core/src/result.rs:1653:5
17: 0x7fe707515fbf - tiktoken_core::new::hbcb3333be3b3fc91
18: 0x7fe70751cee4 - mlua::lua::Lua::create_function::{{closure}}::heed2363ff805704c
19: 0x7fe7075295d8 - mlua::lua::Lua::create_callback::call_callback::h2157948e4b34a838
20: 0x7fe707f46a36 - <unknown>
21: 0x7fe707fa2637 - lua_pcall
22: 0x55ecdd0a365c - nlua_pcall
23: 0x55ecdd0abc29 - nlua_luv_cfpcall
24: 0x7fe70805f415 - luv_async_cb
25: 0x7fe707e0be94 - uv__async_io.part.0
26: 0x7fe707e1fa53 - uv__io_poll
27: 0x7fe707e0cb96 - uv_run
28: 0x55ecdd00bc3b - loop_poll_events
29: 0x55ecdd0188bc - process_wait
30: 0x55ecdd13dd52 - do_os_system.lto_priv.0
31: 0x55ecdd13e489 - os_call_shell
32: 0x55ecdd13e924 - call_shell
33: 0x55ecdd13f385 - os_expand_wildcards
34: 0x55ecdd144c0c - gen_expand_wildcards.part.0
35: 0x55ecdd14543d - expand_wildcards
36: 0x55ecdd145682 - expand_wildcards_eval
37: 0x55ecdcf92266 - ExpandFromContext
38: 0x55ecdcf951bb - ExpandOne
39: 0x55ecdcfe733e - f_expand.lto_priv.0
40: 0x55ecdcfdf756 - call_internal_func
41: 0x55ecdd00a16e - call_func
42: 0x55ecdd0b2357 - nlua_call
43: 0x7fe707f46a36 - <unknown>
44: 0x7fe707fa2637 - lua_pcall
45: 0x55ecdd0a365c - nlua_pcall
46: 0x55ecdd0b1559 - nlua_typval_exec.part.0
47: 0x55ecdd0b3278 - ex_lua
48: 0x55ecdd253c01 - execute_cmd0.constprop.0
49: 0x55ecdd02a3ca - do_cmdline
50: 0x55ecdd18e8b7 - do_source
51: 0x55ecdcf2f937 - main
52: 0x7fe707c131ca - __libc_start_call_main
at ./csu/../sysdeps/nptl/libc_start_call_main.h:58:16
53: 0x7fe707c1328b - __libc_start_main_impl
at ./csu/../csu/libc-start.c:360:3
54: 0x55ecdcf30465 - _start
55: 0x0 - <unknown>
I haven't had that issue. I'll try to reproduce it
Ok I've found the issue. It was newly introduced because of vim.schedule
& concurrency issues such that we falsely expect the tiktoken file to be downloaded despite it not being so
I'm experiencing segfaults when using copilot in Neovim, I'm not sure if it's specific to my setup or what it might be