[BUG] included madmax plotter does not continue plotting without waiting for copy

[randomisresistance]

Describe the bug The now integrated madmax plotter does wait to start plotting the next plot until current plot is copied to the final directory. This happens even the -w or --waitforcopy is not set.

Steps to reproduce the behavior: $ chia plotters madmax --count=-1 -r 6 -t /mnt/ssd/ -2 /mnt/ssd/ -d /mnt/storage/ -c foo -f bar [...]
Phase 3 took 969.503 sec, wrote 21876873898 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 77.7661 sec, final plot size is 108833667393 bytes Total plot creation time was 3482.33 sec (58.0389 min) Started copy to /mnt/storage/plot-k32-2021-11-06-14-28-foobar.plot Copy to /mnt/storage/plot-k32-2021-11-06-14-28-foobar.plot finished, took 622.973 sec, 166.607 MB/ s avg. Crafting plot 10 out of 100 (2021/11/06 15:37:04) [...]

Expected behavior Plotter should continue plotting while copying final plot to final direcory, like it did in the standalone version of madmax

Desktop Distributor ID: Ubuntu Description: Ubuntu 20.04.3 LTS Release: 20.04 Codename: focal Intel NUC Kit NUC10i7FNH2 Intel i7-10710U

Additional context Works as expected with the standalone plotter version of madmax cloned from github.

[randomisresistance]

My explation for this is that the chia virtual environment does not hand over the correct default values to the madmax plotter. The same goes with the 2nd temp directory, it must explicitly be set otherwise it will use the current $PATH, which is ~/home/chia/chia-blockchain/chia/plotters/madmax/ in the most cases.

[lasers8oclockday1]

+1 confirming the same behavior as described above, though I am running Chia v1.2.11 on Windows.

Have been running the madmax plotter on Windows for some time now (latest version as built by stotiks/chia-plotter) in Powershell 7.

In addition, I use two temp drives, and have been making use of the -G option to toggle between them to even out ware.

The bundled version of madmax in Chia v1.2.11 doesn't seem to respect either of -G or the longform --tmptoggle equivalent.

Logs show the following: .\chia.exe plotters madmax -n -1 -r 4 -K 4 -t E:\ -2 F:\ -d H:\ -c foo -f bar -G | tee my_log_file [...] Working Directory: E:\ Working Directory 2: F:\ [...] Working Directory: E:\ Working Directory 2: F:\ [...] ... and so on.

Using CrystalDiskInfo, I can confirm that the toggling is definitely not happening.

There are obvious workarounds for this issue, but it would be really nice if it worked like the versions from GitHub.

I will gladly test a specific scenario locally if requested to do so.

Many thanks!

[madMAx43v3r]

I assume chia only does a single plot at a time, ie. -n 1, which will make the process wait for copy. And in turn chia waits for the current process to finish before starting the next plot.

This is quite a tricky problem. I suppose chia has to do the final copy itself, in parallel to the next plot.

EDIT: Oh I see you are not using the GUI... that's a different issue then..

[assasinmaxy]

I've got a very slow pooling when 1 embedded madmux plot cteating is going (core i7 - 10gen + 16 GB RAM). And alomost no pooling when 2 madmax plotting processes are going. In logs I saw an increased delay of proofing. It takes <1s 10% of all cases, >1s 90%, >3s 60%, so on. Even if ploting 98% and plot is just copying to my NAS and madmax (probably) doing nothing, pooling are very slow.

[DrakeFS]

Not really the place for this but:

I've got a very slow pooling when 1 embedded madmux plot cteating is going (core i7 - 10gen + 16 GB RAM). And alomost no pooling when 2 madmax plotting processes are going. In logs I saw an increased delay of proofing. It takes <1s 10% of all cases, >1s 90%, >3s 60%, so on. Even if ploting 98% and plot is just copying to my NAS and madmax (probably) doing nothing, pooling are very slow.

You are probably overloading your hardware (especially trying to run to MMplotters in parallel on your system specs). As for slow "pooling" (I assume you mean farming) while copying, you are probably overloading your NAS' I/O or Network bandwidth.

[paninaro]

Thank you for the reports @randomisresistance and @lasers8oclockday1!

I can repro the improper behaviors with the tmptoggle (-G), waitforcopy (-w), and tmpdir2 (-2) options when plotting from the command line. Fixes should be coming shortly.

The (don't) waitforcopy behavior is more difficult to implement in the GUI. As @madMAx43v3r correctly mentioned, the code for managing GUI plotting generally handles each plot one at a time (-n 1). We'll look at some options for fixing this.

[paninaro]

CLI fixes have been merged into main. An exploratory change for the GUI (don't) waitforcopy behavior has been sent out for comments but we haven't committed to a fix yet.