search

ArthurBrussee / brush

3D Reconstruction for all
Apache License 2.0
917 stars 25 forks source link

Spontanious validation error #26

Open Poly2it opened 1 week ago

Poly2it commented 1 week ago

After running for some time, this program may run into fatal errors. The rest of my system was running in a faultless state the whole time.

thread 'main' panicked at /home/bach/.cargo/git/checkouts/wgpu-f9afa33caa1e84c9/ffb4852/wgpu/src/backend/wgpu_core.rs:23
Error in Queue::submit: Validation Error

Caused by:
  Parent device is lost

note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
ArthurBrussee commented 1 week ago

Unfortunately that error is basically what happens if anything goes wrong, so it's reallly hard to say :/

What platforms is this on? Is this during training or rendering? How many steps had you run? Did it work ok before that?

Poly2it commented 1 week ago

Unfortunately that error is basically what happens if anything goes wrong, so it's reallly hard to say :/

What platforms is this on? Is this during training or rendering? How many steps had you run? Did it work ok before that?

During training, on Linux. In my experience it is independent of previous iterations, and seems not to be an issue of a lack of system resources. Maybe there should be some printouts?

ArthurBrussee commented 6 days ago

Does this by any chance happen at the same time as https://github.com/ArthurBrussee/brush/issues/28 ?

Poly2it commented 4 days ago

Does this by any chance happen at the same time as #28 ?

No, this leaves a trace, #28 is a hard termination.