thread 'disk' panicked at 'index 207566898 out of range for slice of length 206518322', src/libcore/slice/mod.rs:2555:5

[ArniDagur]

Here is the full activity log:

torrent@arni-pc ~ $ synapse
09/05/19 21:55:19 [synapse::init:24] I: Initializing
09/05/19 21:55:19 [synapse::config:173] I: Loaded config file
09/05/19 21:55:19 [synapse:112] I: Initialized, starting!
09/05/19 21:55:19 [synapse::rpc::processor:92] I: user data could not be read from disk, creating a fresh version: No such file or directory (os error 2)
09/05/19 21:55:19 [synapse::rpc:603] I: RPC SSL parameters not specified, using insecure connections!
09/05/19 21:55:19 [synapse::tracker::dht::rt:102] I: Loaded 23 dht buckets, 183 nodes
09/05/19 21:55:19 [synapse::tracker::dht:57] I: Attempting DHT bootstrap!
thread 'disk' panicked at 'index 207566898 out of range for slice of length 206518322', src/libcore/slice/mod.rs:2555:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
09/05/19 21:57:22 [synapse::control:145] E: crashed thread detected, terminating!
09/05/19 21:57:22 [synapse::init:41] E: Unclean shutdown detected, terminating

The panic occured during the initial hash-verification of my torrents. This was my torrent list one or two minutes before, in case it gives any clues (don't judge me):

[arni][~]% sycli list
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Name                                                                                                    | Done    | DL        | UL        | DL RT | UL RT   | Peers |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| 1-0004317770.flac                                                                                       | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Tejina-senpai - 01 [1080p].mkv                                                           | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Touhou] - Fantasy Kaleidoscope Ep. 6 ~ The Flowers Incident (Later Part) [Eng subbed].mp4              | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [FullSet] Gensou Mangekyou ~The Memories of Phantasm~ Ep 01-11 [HD720p JapDub Rus Chi Eng sub]          | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 9     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [SaltyBucket] Touhou Niji Sousaku Doujin Anime ~ Musou Kakyou                                           | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 6     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [RH] Kiss x Sis [BDRip] [1080p]                                                                         | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 4     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sounan desu ka - 01 [1080p].mkv                                                          | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 6     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [AnimeFreak] Miru Tights - 10 [1080p].mkv                                                               | 100.00% | 31.25 MiB | 0 B       | 0 B/s | 0 B/s   | 20    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [qx] Touhou Gensou Mangekyou OVA 01-11 [DVDRip 852x480 x264] [x3]                                       | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Maid-chan] TRUE - Soundscape (Hibike! Euphonium 2 OP Single) (FLAC)                                    | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [mikudb] ☆VOCALOSSLESS Project -Hatsune Miku &+ Vocaloid in FLAC- (300 albums) [FLAC]                   | 100.00% | 0 B       | 303.5 MiB | 0 B/s | 0 B/s   | 7     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| 【Touhou Anime】東方Fantasy Kaleidoscope Ep.7 ~ The Mysterious Giant Youkai .mp4                        | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sewayaki Kitsune no Senko-san - 05 [1080p].mkv                                           | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| 東方魔法少女 アルティメット☆れいむ【通常版】                                                            | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Audio-4U] (C87) DDBY — Cafe de Touhou 6 (東方) (flac+scans)                                            | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Sheldonsuckz] TRUE - Blast! [Hi-RES]                                                                   | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Tsundere] Non Non Biyori [BDRip h264 1920x1080 10bit FLAC]                                             | 28.14%  | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Vinland Saga - 02 [1080p].mkv                                                            | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 24    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Komorebi] Violet Evergarden [BD 1080p x264-10bit FLAC]                                                 | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| FK_2A_TSDX                                                                                              | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Yuru Music                                                                                              | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Yuru Yuri 2 (2012) [Doki][1920x1080 Hi10P BD FLAC]                                                      | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 2     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Audio-4U] (С88) DDBY — Cafe de Touhou 7 (東方) (flac)                                                  | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| tails-amd64-3.14.img                                                                                    | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 11    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [fong] Lupin III - Touhou Kenbunroku - Another Page [BDrip.1080p.10bit].mkv                             | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Nep_Blanc] KonoSuba II [1080p] [x265] [10Bit] [Subbed]                                                 | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [sono1998] Touhou - Bad Apple!! [PV] (1440x1080 x264 Hi10P AAC).mkv                                     | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Vinland Saga - 01 [1080p].mkv                                                            | 100.00% | 0 B       | 96 kiB    | 0 B/s | 0 B/s   | 43    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Yuru Yuri BD 1080p HEVC AAC                                                                             | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sounan desu ka - 03 [1080p].mkv                                                          | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 34    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| YURUYURI BEST ALBUM3 ゆるゆりずむ♪3                                                                     | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Erai-raws] Machikado Mazoku - 01 [1080p].mkv                                                           | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [qx] Touhou Gensou Mangekyou OVA 1-7 [BDRip 1920x1080 x264] [x3]                                        | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 6     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Tsundere] Non Non Biyori Repeat [BDRip h264 1920x1080 10bit FLAC]                                      | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Kiss x Sis (TV) (2010) [Doki][1920x1080 h264 BD FLAC]                                                   | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 2     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [VCB-Studio] Gensou Mangekyou ~The Memories Of Phantasm~ [Ma10p_1080p]                                  | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 20    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Maid-chan] TRUE - DREAM SOLISTER (Hibike! Euphonium OP Single) (FLAC)                                  | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| phantasm3.iso                                                                                           | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 1     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [VCB-Studio] Gekijouban Hibike! Euphonium Todoketai Melody [Ma10p_1080p]                                | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [FFF-Vivid] Non Non Biyori Vacation [BD 1080p AAC] [3D65299C].mkv                                       | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 26    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| ZUN Touhou Official Music Collection                                                                    | 100.00% | 0 B       | 29.94 MiB | 0 B/s | 178 B/s | 12    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] YuruYuri S3 (01-12) [1080p] (Batch)                                                      | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sewayaki Kitsune no Senko-san - 04 [1080p].mkv                                           | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 21    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Komorebi] Violet Evergarden - 14 (OVA) [BD 1080p x264-10bit FLAC]                                      | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 15    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| FantasyKaleidoscope_04_05_untouched                                                                     | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Vinland Saga - 03 [1080p].mkv                                                            | 100.00% | 0 B       | 128 kiB   | 0 B/s | 209 B/s | 44    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sounan desu ka - 02 [1080p].mkv                                                          | 100.00% | 0 B       | 400 kiB   | 0 B/s | 0 B/s   | 2     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] One Punch Man S2 - 03 [1080p].mkv                                                        | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 18    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [1080p] Hibike! Euphonium S1 [FFF]+ S2 [pls] + OVA [anon] + Liz to Aoi Tori [ReinForce + ChannelOrange] | 100.00% | 0 B       | 5.5 MiB   | 0 B/s | 0 B/s   | 10    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Touhou Project Side Story - Memory of Stars (DVDRip)                                                    | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 3     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Nep_Blanc] KonoSuba II OVA [1080p] [x265] [10Bit] [Subbed] V2                                          | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 8     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| 1-0004245678.flac                                                                                       | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Maid-chan] Hibike! Euphonium 2 Original Soundtrack - Ongaku Endless (FLAC)                             | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [Audio-4U] (C85) DDBY — Cafe de Touhou 5 (東方) (flac)                                                  | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Yuru Yuri (2011) [Doki][1920x1080 Hi10P BD FLAC]                                                        | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [U3-Tsundere] Non Non Biyori OAD 1&2 [BDRip 10bit 1920x1080 FLAC]                                       | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 2     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| Touhou Project                                                                                          | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 4     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [AnimeFreak] Miru Tights - 08 [1080p].mkv                                                               | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] One Punch Man S2 - 04 [1080p].mkv                                                        | 100.00% | 0 B       | 0 B       | 0 B/s | 0 B/s   | 0     |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+
| [HorribleSubs] Sewayaki Kitsune no Senko-san - 06 [1080p].mkv                                           | 0.00%   | 0 B       | 0 B       | 0 B/s | 0 B/s   | 19    |
+---------------------------------------------------------------------------------------------------------+---------+-----------+-----------+-------+---------+-------+

[kpcyrd]

Can you try again with RUST_BACKTRACE=1 set? The backtrace gives additional information where the slice was attempted and makes it easier to figure out where to place additional length checks.

[ArniDagur]

I ran it twice with backtrace enabled. On the second run, the client hasn't crashed yet.

torrent@arni-pc ~ $ RUST_BACKTRACE=1 synapse
09/05/19 22:37:14 [synapse::init:24] I: Initializing
09/05/19 22:37:14 [synapse::config:173] I: Loaded config file
09/05/19 22:37:14 [synapse:112] I: Initialized, starting!
09/05/19 22:37:14 [synapse::rpc::processor:92] I: user data could not be read from disk, creating a fresh version: No such file or directory (os error 2)
09/05/19 22:37:14 [synapse::rpc:603] I: RPC SSL parameters not specified, using insecure connections!
09/05/19 22:37:14 [synapse::tracker::dht::rt:102] I: Loaded 23 dht buckets, 184 nodes
09/05/19 22:37:14 [synapse::tracker::dht:57] I: Attempting DHT bootstrap!
thread 'disk' panicked at 'index 207566898 out of range for slice of length 206518322', src/libcore/slice/mod.rs:2555:5
stack backtrace:
   0: backtrace::backtrace::libunwind::trace
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/libunwind.rs:88
   1: backtrace::backtrace::trace_unsynchronized
             at /cargo/registry/src/github.com-1ecc6299db9ec823/backtrace-0.3.29/src/backtrace/mod.rs:66
   2: std::sys_common::backtrace::_print
             at src/libstd/sys_common/backtrace.rs:47
   3: std::sys_common::backtrace::print
             at src/libstd/sys_common/backtrace.rs:36
   4: std::panicking::default_hook::{{closure}}
             at src/libstd/panicking.rs:200
   5: std::panicking::default_hook
             at src/libstd/panicking.rs:214
   6: std::panicking::rust_panic_with_hook
             at src/libstd/panicking.rs:477
   7: std::panicking::continue_panic_fmt
             at src/libstd/panicking.rs:384
   8: rust_begin_unwind
             at src/libstd/panicking.rs:311
   9: core::panicking::panic_fmt
             at src/libcore/panicking.rs:85
  10: core::slice::slice_index_len_fail
             at src/libcore/slice/mod.rs:2555
  11: synapse::disk::cache::FileCache::read_file_range
             at src/disk/cache.rs:0
  12: synapse::disk::job::Request::execute
             at src/disk/job.rs:481
  13: synapse::disk::Disk::handle_active
             at src/disk/mod.rs:101
  14: synapse::disk::Disk::run
             at src/disk/mod.rs:74
  15: synapse::disk::start::{{closure}}
             at src/disk/mod.rs:201
  16: synapse::handle::Handle<I,O>::run::{{closure}}
             at src/handle.rs:39
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
09/05/19 22:38:09 [synapse::control:145] E: crashed thread detected, terminating!
09/05/19 22:38:09 [synapse::init:41] E: Unclean shutdown detected, terminating

torrent@arni-pc ~ $ RUST_BACKTRACE=full synapse
09/05/19 22:38:22 [synapse::init:24] I: Initializing
09/05/19 22:38:22 [synapse::config:173] I: Loaded config file
09/05/19 22:38:23 [synapse:112] I: Initialized, starting!
09/05/19 22:38:23 [synapse::rpc::processor:92] I: user data could not be read from disk, creating a fresh version: No such file or directory (os error 2)
09/05/19 22:38:23 [synapse::rpc:603] I: RPC SSL parameters not specified, using insecure connections!
09/05/19 22:38:23 [synapse::tracker::dht::rt:102] I: Loaded 23 dht buckets, 184 nodes
09/05/19 22:38:23 [synapse::tracker::dht:57] I: Attempting DHT bootstrap!
09/05/19 22:39:26 [synapse::torrent:878] I: Torrent E057A8A80EE5656AAEC05333CBCFED62FAC8C7B7 completed!
09/05/19 22:39:53 [synapse::torrent:878] I: Torrent 132E4AB0B1E7D4D27B2C0B30F4BEE1868460ADF2 completed!

[kpcyrd]

Thanks, there's a missing check to ensure that FileCache::read_file_range isn't called with offset + buf.len() larger than the file. You can work around this until it is patched by building without the mmap feature.

https://github.com/Luminarys/synapse/blob/1b98a3e74f25313be60a89392e0ccfc207094a2f/src/disk/cache.rs#L142-L153

[Luminarys]

I'll look into this, but generally I'd avoid the mmap feature it possible, my conclusion on it is that the theoretical gains aren't worth the complexity. It will probably be removed in a future release.

[evanrichter]

I wonder how async std's performance of BufReader is in a case like this

[Luminarys]

As of right now async std currently uses a threadpool to handle all its blocking file operations. I've looked into this sort of approach but I'm fairly certain it won't give very good performance and just add more complexity. I think to achieve serious performance something like io_uring would need to be used, but based on my benchmarks this is somewhat unnecessary, synapse is rarely bottlenecked by disk read/write performance as far as I know.

[Luminarys]

@ArniDagur do you know if your torrents were complete when you tried doing hash validation on them?

My best guess as to what happened is that because in validation length isn't passed into the ensure_exists method of the file cache, the existing file was mmap'd but wasn't large enough so the slice access went out of bounds. Regardless I'll probably just remove the mmap code soon which I think should take care of this. In the meanwhile just don't use it.

[ArniDagur]

These were all completed torrents.