search

chipsenkbeil / vimwiki-rs

Rust library and tooling to parse, render, and modify vimwiki text and files.
56 stars 2 forks source link

vimwiki convert: EOF while parsing an object #121

Closed brennen closed 3 years ago

brennen commented 3 years ago

At eff0334, tried running vimwiki convert.

16:13:14 brennen@inertia:~ ✮ RUST_BACKTRACE=full vimwiki convert
thread 'main' panicked at 'Failed to load data: Custom { kind: UnexpectedEof, error: Error("EOF while parsing an object", line: 1, column: 120427) }', vimwiki-cli/src/main.rs:25:6
stack backtrace:
   0:     0x5625e06f2548 - std::backtrace_rs::backtrace::libunwind::trace::h63b7a90188ab5fb3
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x5625e06f2548 - std::backtrace_rs::backtrace::trace_unsynchronized::h80aefbf9b851eca7
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x5625e06f2548 - std::sys_common::backtrace::_print_fmt::hbef05ae4237a4d72
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x5625e06f2548 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h28abce2fdb9884c2
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x5625e06796df - core::fmt::write::h3b84512577ca38a8
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:1092:17
   5:     0x5625e06f1bc1 - std::io::Write::write_fmt::h465f8feea02e2aa1
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/io/mod.rs:1572:15
   6:     0x5625e06f107d - std::sys_common::backtrace::_print::h525280ee0d29bdde
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x5625e06f107d - std::sys_common::backtrace::print::h1f0f5b9f3ef8fb78
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x5625e06f107d - std::panicking::default_hook::{{closure}}::ha5838f6faa4a5a8f
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:208:50
   9:     0x5625e06f107d - std::panicking::default_hook::hfb9fe98acb0dcb3b
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:225:9
  10:     0x5625e06f107d - std::panicking::rust_panic_with_hook::hb89f5f19036e6af8
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:591:17
  11:     0x5625e070c128 - std::panicking::begin_panic_handler::{{closure}}::h119e7951427f41da
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:497:13
  12:     0x5625e070c09c - std::sys_common::backtrace::__rust_end_short_backtrace::hce386c44bf47a128
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x5625e070c04d - rust_begin_unwind
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
  14:     0x5625e05c9f70 - core::panicking::panic_fmt::h2242888e8769cd33
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
  15:     0x5625e05ca632 - core::option::expect_none_failed::hb1edf11f73e63728
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:1329:5
  16:     0x5625e06375e2 - vimwiki::main::he52c118b66d3ed61
  17:     0x5625e05f78a7 - std::sys_common::backtrace::__rust_begin_short_backtrace::hb4f41db2d61cb71d
  18:     0x5625e0638b75 - main
  19:     0x7f7c17b6809b - __libc_start_main
  20:     0x5625e05d07aa - _start
  21:                0x0 - <unknown>
zsh: exit 101   RUST_BACKTRACE=full vimwiki convert
chipsenkbeil commented 3 years ago

Huh, this is directly related to the caching, I think. I haven't hit an error like that, though.

Try https://github.com/chipsenkbeil/vimwiki-rs/commit/03395c4f931e6894b8ff0f071e7366a86d7e05e2 to see if we can get more helpful error messages?

Also, what do you see with vimwiki convert -vvvv?

brennen commented 3 years ago

Churns through quite a few pages before hitting:

ERROR - "/home/brennen/notes/vimwiki/_logscratch/05b33f9172e36081806588a2a90905e8c6e04406cb6cf8de28283272eee57a67.wiki" :: cache corrupted: EOF while parsing an object at line 1 column 120427

And sure enough the cache file ends abruptly with {"inner":{"Table":{"cells":{at 120411 chars.

The source there is a ~950 line file with a bunch of code blocks, a hundred line table, etc., so I'm not surprised if something's breaking on it. It's not publically shareable, but I'll see if I can't narrow to some repro case.

brennen commented 3 years ago

Looks to me like this is just blowing up on tables.

For a wiki containing only this single page:

17:18:53 brennen@inertia:~/notes/vimwiki ☼ cat test.wiki 
| foo | bar | baz  |

I get:

17:16:46 brennen@inertia:~/notes/vimwiki ☼ RUST_BACKTRACE=full vimwiki convert -vvvv
TRACE - load_html_config(path = None, merge = false)
TRACE - load_vimwiki_list()                                                                                                                                                                                                                                                    
TRACE - g:vimwiki_list == Some(Array([Object({"path_html": String("~/workspace/notes-html/"), "auto_tags": Number(1), "name": String("markdownwiki"), "valid_": Number(1), "automatic_nested_syntaxes": Number(1), "auto_diary_index": Number(1), "path": String("~/notes/vimwiki/")}), Object({"path_html": String("~/notes/markdownwiki/html/"), "auto_tags": Number(1), "path": String("~/notes/markdownwiki"), "ext": String(".md"), "automatic_nested_syntaxes": Number(1), "auto_diary_index": Number(1), "syntax": String("markdown")})]))             
TRACE - load_ast(_, include = [], cache = "/home/brennen/.cache/vimwiki-cli", no_cache = false, no_prune_cache = false)                                                                                                                                                        
TRACE - Loading wiki @ index = 0 | name = Some("markdownwiki") from "/home/brennen/notes/vimwiki"                                                                                                                                                                              
TRACE - load_wiki_file(path = "/home/brennen/notes/vimwiki/test.wiki", cache = "/home/brennen/.cache/vimwiki-cli", no_cache = false)                                                                                                                                           
TRACE - "/home/brennen/notes/vimwiki/test.wiki" :: text loaded                                                                                                                                                                                                                 
TRACE - "/home/brennen/notes/vimwiki/test.wiki" :: checksum = b75ea5573cad6644d4cac895de5964c14aba591a                                                                                                                                                                         
TRACE - "/home/brennen/notes/vimwiki/test.wiki" :: checking cache at "/home/brennen/.cache/vimwiki-cli/b75ea5573cad6644d4cac895de5964c14aba591a"                                                                                                                               
TRACE - "/home/brennen/notes/vimwiki/test.wiki" :: no cache found                                                                                                                                                                                                              
thread 'main' panicked at 'Failed to load data: Custom { kind: InvalidData, error: Error("key must be a string", line: 0, column: 0) }', vimwiki-cli/src/main.rs:25:6
stack backtrace:
   0:     0x556170e20fd8 - std::backtrace_rs::backtrace::libunwind::trace::h63b7a90188ab5fb3
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/../../backtrace/src/backtrace/libunwind.rs:90:5
   1:     0x556170e20fd8 - std::backtrace_rs::backtrace::trace_unsynchronized::h80aefbf9b851eca7
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
   2:     0x556170e20fd8 - std::sys_common::backtrace::_print_fmt::hbef05ae4237a4d72
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:67:5
   3:     0x556170e20fd8 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h28abce2fdb9884c2
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:46:22
   4:     0x556170da816f - core::fmt::write::h3b84512577ca38a8
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/fmt/mod.rs:1092:17
   5:     0x556170e20651 - std::io::Write::write_fmt::h465f8feea02e2aa1
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/io/mod.rs:1572:15
   6:     0x556170e1fb0d - std::sys_common::backtrace::_print::h525280ee0d29bdde
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:49:5
   7:     0x556170e1fb0d - std::sys_common::backtrace::print::h1f0f5b9f3ef8fb78
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:36:9
   8:     0x556170e1fb0d - std::panicking::default_hook::{{closure}}::ha5838f6faa4a5a8f
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:208:50
   9:     0x556170e1fb0d - std::panicking::default_hook::hfb9fe98acb0dcb3b
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:225:9
  10:     0x556170e1fb0d - std::panicking::rust_panic_with_hook::hb89f5f19036e6af8
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:591:17
  11:     0x556170e3abb8 - std::panicking::begin_panic_handler::{{closure}}::h119e7951427f41da
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:497:13
  12:     0x556170e3ab2c - std::sys_common::backtrace::__rust_end_short_backtrace::hce386c44bf47a128
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/sys_common/backtrace.rs:141:18
  13:     0x556170e3aadd - rust_begin_unwind
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/std/src/panicking.rs:493:5
  14:     0x556170cf7f70 - core::panicking::panic_fmt::h2242888e8769cd33
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/panicking.rs:92:14
  15:     0x556170cf8632 - core::option::expect_none_failed::hb1edf11f73e63728
                               at /rustc/9bc8c42bb2f19e745a63f3445f1ac248fb015e53/library/core/src/option.rs:1329:5
  16:     0x556170d66074 - vimwiki::main::he52c118b66d3ed61
  17:     0x556170d25c00 - std::sys_common::backtrace::__rust_begin_short_backtrace::hb4f41db2d61cb71d
  18:     0x556170d67607 - main
  19:     0x7f8dcfb6209b - __libc_start_main
  20:     0x556170cfe7aa - _start
  21:                0x0 - <unknown>
zsh: exit 101   RUST_BACKTRACE=full vimwiki convert -vvvv
chipsenkbeil commented 3 years ago

@brennen try https://github.com/chipsenkbeil/vimwiki-rs/commit/0d476099e957c67e9768267d237b8d38cb77227f. I forgot to flush output of cache files, which I'm assuming was the culprit.

[EDIT] Not sure about the table part. I'm able to reproduce this one, so I'll have to dive into it as a separate issue.

chipsenkbeil commented 3 years ago

@brennen try https://github.com/chipsenkbeil/vimwiki-rs/commit/7132cd4ed8c5df4df00d20c033cf5a62c8e03a10 to see if that fixes the table example.

brennen commented 3 years ago

try 7132cd4 to see if that fixes the table example.

Yep, sure seems like it. No more errors for pages with tables!

I am getting one remaining thing. Filed that as #122.