Engraver crashes after allocating plotfile on an NTFS partition (linux)

[golyalpha]

Not exactly sure what to make of this, but here's the last few lines of output with backtrace.

Numeric ID:  5373760232911902050
Start Nonce: 0
Nonces:      18922368 (rounded to sector size for fast direct i/o)
Output File: /media/golyalpha/Plot1/5373760232911902050_0_18922368

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 2, kind: NotFound, message: "No such file or directory" }', libcore/result.rs:1009:5
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::result::unwrap_failed
             at libcore/macros.rs:26
   9: <core::result::Result<T, E>>::unwrap
             at libcore/result.rs:808
  10: engraver::utils::preallocate
             at src/utils.rs:119
  11: engraver::plotter::Plotter::run
             at src/plotter.rs:234
  12: engraver::main
             at src/main.rs:246
  13: std::rt::lang_start::{{closure}}
             at libstd/rt.rs:74
  14: std::panicking::try::do_call
  15: __rust_maybe_catch_panic
  16: std::rt::lang_start_internal
  17: std::rt::lang_start
             at libstd/rt.rs:74
  18: main
  19: __libc_start_main
  20: _start
Fast file pre-allocation...%

And here's the file it created.

[golyalpha]

Same happens to me with the latest pre-built binary for Linux.

Also, running cargo test passes.

[JohnnyFFM]

can you share the command you've used to start engraver? what file-system are you running on your drive? will have a look then.

[golyalpha]

The filesystem should have been either ext4 or NTFS , but ext4 more likely. The command: ./engraver_cpu -n 18922400 -s 0 -i 5373760232911902050 -p /media/golyalpha/Plot1 It's the same for both the precompiled binary and the locally built one (except for the executable name).

[golyalpha]

Well, so, now that we manage to start hashing (on an ext4 partition, NTFS still crashes), this happens (I'm suspecting it's at the start of a write, might be wrong, not a rust developer):

./engraver -n 20775476 -s 0 -i 5373760232911902050 -p /media/golyalpha/Plot
Engraver 2.2.0 - PoC2 Plotter

CPU: Intel(R) Core(TM) i3-3110M CPU @ 2.40GHz [using 4 of 4 cores + AVX]
RAM: Total=7.68 GiB, Free=2.15 GiB, Usage=2.09 GiB
Numeric ID:  5373760232911902050
Start Nonce: 0
Nonces:      20775424 (rounded to sector size for fast direct i/o)
Output File: /media/golyalpha/Plot/5373760232911902050_0_20775424

Fast file pre-allocation...OK
Starting plotting...

Hashing: │█░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░│ 0.02 % 18.29 MB/s 4732m 
Writing: │░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░░│ 0.00 % 0 B/s  
thread '<unnamed>' panicked at 'called `Result::unwrap()` on an `Err` value: Os { code: 22, kind: InvalidInput, message: "Invalid argument" }', libcore/result.rs:1009:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
stack backtrace:
   0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
   1: std::sys_common::backtrace::print
   2: std::panicking::default_hook::{{closure}}
   3: std::panicking::default_hook
   4: std::panicking::rust_panic_with_hook
   5: std::panicking::continue_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::result::unwrap_failed

Precompiled binaries don't suffer from this.

[JohnnyFFM]

Almost there! It’s directi i/o again. Invalid argument = direct i/o fails.

Please try the -d switch to disable direct i/o.

[JohnnyFFM]

And one more thing: plotting ntfs on a *nix system is not working well. Ntfs has some security features not allowing quick file allocation until you have special rights. These right can only be obtained under windows. As a result, ntfs file allocation on nix takes ages (file will be created and zeros will be written, 8TB can take 12h+)

[golyalpha]

Is it possible to skip fast file allocation if FS is NTFS and platform is *nix? Also, that crash I showed you (my last comment) is happening on an ext4 partition.

Also, about fast file allocation on NTFS, that file in the screenshot in the OP took about the same time as it does for 5.4 TB file on an ext4 partition.

[JohnnyFFM]

very difficult for ntfs under *nix. I think the driver will automatically use the slow allocation. The only way to have it fast is to pre-allocate it on a windows machine, stop the plotting and resume on nix.

will double check the creation on ext4.

[JohnnyFFM]

Hi, update on this issue: in the current release, direct i/o is bugged for linux only. I‘ve created a fix in master and will soon compile a new release. The current linux release only works with direct i/o disabled.

[JohnnyFFM]

fixed in 2.4.0