Plots no longer being moved to storage drive

[glukkon24]

Hoping someone could shed some light on this. I have had little issue with the plotter but now all of a sudden the files are staying on my ssd and not being put on storage drive. Before it always moved them between phase 1 table 3 and table 4. I haven't changed anything so not sure why this is occurring. Any help would be greatly appreciated.

[madMAx43v3r]

can you post the output at beginning of plot

[salmancursher]

I get the same issue, my SSDs were 3.5tb filled out of 4tb combined, and plots were not moved after compleition.

I got an error copy to ........ failed with fopen() failed

[manuel-ruiz-23]

i think i have the same issue on the pool puzzle branch

This is the options im using on a rizen 2200g, the storage drives are on a fat format if that helps

!/bin/bash

sudo ./chia-plotter/build/chia_plot -n 1 -r 4 -t /home/the-farm/Desktop/plots_temp/ -d /media/the-farm/genesis/ -c -f

on a fresh install of pop_os

[manuel-ruiz-23]

Multi-threaded pipelined Chia k32 plotter - 7c6d195
Final Directory: /media/the-farm/genesis/
Number of Plots: 1
Crafting plot 1 out of 1
Process ID: 14923
Number of Threads: 4
Number of Buckets P1:    2^8 (256)
Number of Buckets P3+P4: 2^8 (256)
Pool Puzzle Hash:  8eddfb24851519245cedc4e6770fb20b79fd80a863a9fd1137d839427ed86eba
Farmer Public Key: 81944d9facaa8b3c0d21fcb9a3f2deb6c271b5ce6534c4207be6d27eee0b3be19f31a5635ea7d311fa4bc775a91fcd84
Working Directory:   /home/the-farm/Desktop/plots_temp/
Working Directory 2: /home/the-farm/Desktop/plots_temp/
Plot Name: plot-k32-2021-07-11-18-31-64415009d6e1366af7c8d128936b0e72c047fa4fec99148e44e9aa8b061422cb
[P1] Table 1 took 50.7742 sec
[P1] Table 2 took 429.812 sec, found 4295141046 matches
[P1] Table 3 took 494.751 sec, found 4295126572 matches
[P1] Table 4 took 572.672 sec, found 4295204247 matches
[P1] Table 5 took 565.645 sec, found 4295167428 matches
[P1] Table 6 took 535.018 sec, found 4295142429 matches
[P1] Table 7 took 431.151 sec, found 4295175772 matches
Phase 1 took 3079.87 sec
[P2] max_table_size = 4295204247
[P2] Table 7 scan took 30.3431 sec
[P2] Table 7 rewrite took 79.6773 sec, dropped 0 entries (0 %)
[P2] Table 6 scan took 140.234 sec
[P2] Table 6 rewrite took 251.155 sec, dropped 581375843 entries (13.5357 %)
[P2] Table 5 scan took 134.632 sec
[P2] Table 5 rewrite took 244.911 sec, dropped 762153318 entries (17.7444 %)
[P2] Table 4 scan took 132.278 sec
[P2] Table 4 rewrite took 237.688 sec, dropped 829046095 entries (19.3017 %)
[P2] Table 3 scan took 131.763 sec
[P2] Table 3 rewrite took 235.627 sec, dropped 855214396 entries (19.9113 %)
[P2] Table 2 scan took 131.434 sec
[P2] Table 2 rewrite took 236.052 sec, dropped 865762966 entries (20.1568 %)
Phase 2 took 1990.96 sec
Wrote plot header with 252 bytes
[P3-1] Table 2 took 107.927 sec, wrote 3429378080 right entries
[P3-2] Table 2 took 101.396 sec, wrote 3429378080 left entries, 3429378080 final
[P3-1] Table 3 took 141.612 sec, wrote 3439912176 right entries
[P3-2] Table 3 took 98.3201 sec, wrote 3439912176 left entries, 3439912176 final
[P3-1] Table 4 took 144.687 sec, wrote 3466158152 right entries
[P3-2] Table 4 took 99.5501 sec, wrote 3466158152 left entries, 3466158152 final
[P3-1] Table 5 took 145.796 sec, wrote 3533014110 right entries
[P3-2] Table 5 took 101.687 sec, wrote 3533014110 left entries, 3533014110 final
[P3-1] Table 6 took 152.436 sec, wrote 3713766586 right entries
[P3-2] Table 6 took 107.542 sec, wrote 3713766586 left entries, 3713766586 final
[P3-1] Table 7 took 127.23 sec, wrote 4295175772 right entries
[P3-2] Table 7 took 122.568 sec, wrote 4294967296 left entries, 4294967296 final
Phase 3 took 1452.24 sec, wrote 21877196400 entries to final plot
[P4] Starting to write C1 and C3 tables
[P4] Finished writing C1 and C3 tables
[P4] Writing C2 table
[P4] Finished writing C2 table
Phase 4 took 133.596 sec, final plot size is 108835705190 bytes
Total plot creation time was 6656.71 sec (110.945 min)
Started copy to /media/the-farm/genesis/plot-k32-2021-07-11-18-31-64415009d6e1366af7c8d128936b0e72c047fa4fec99148e44e9aa8b061422cb.plot
Copy to /media/the-farm/genesis/plot-k32-2021-07-11-18-31-64415009d6e1366af7c8d128936b0e72c047fa4fec99148e44e9aa8b061422cb.plot failed with: fwrite() failed

[belomarkovicdev]

Same here Im using stotiks plotter for windows

Multi-threaded pipelined Chia k32 plotter - 82f92a9 Build 0.1.1 for Windows. Check for latest updates: https://stotiks.github.io/chia-plotter/

Final Directory: G:\ Number of Plots: 4 Crafting plot 1 out of 4 Process ID: 8780 Number of Threads: 6 Number of Buckets P1: 2^8 (256) Number of Buckets P3+P4: 2^8 (256) Pool Puzzle Hash: 714edbece1589406da826c9df5ee34062b3e9871e7e62ca0d2db7a0ace990c07 Farmer Public Key: ac87f3b326905065413d8fb535f0ae8512d576bdd27f6f125e54ff170db7fcbfeff7823f83bcc4fb808c382871272c41 Working Directory: D:\ Working Directory 2: D:\ Plot Name: plot-k32-2021-07-13-07-36-b9b74d34bbf065caaa19fe3f90fa55124281b00bee53c4d21cb759e87aac3784 [P1] Table 1 took 19.8534 sec [P1] Table 2 took 229.581 sec, found 4294910145 matches [P1] Table 3 took 270.283 sec, found 4294806307 matches [P1] Table 4 took 354.237 sec, found 4294694750 matches [P1] Table 5 took 340.864 sec, found 4294533176 matches [P1] Table 6 took 321.835 sec, found 4294241520 matches [P1] Table 7 took 241.669 sec, found 4293499264 matches Phase 1 took 1778.56 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 15.7163 sec [P2] Table 7 rewrite took 28.6632 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 49.9901 sec [P2] Table 6 rewrite took 81.042 sec, dropped 581338020 entries (13.5376 %) [P2] Table 5 scan took 47.0918 sec [P2] Table 5 rewrite took 77.4978 sec, dropped 762024595 entries (17.7441 %) [P2] Table 4 scan took 46.5234 sec [P2] Table 4 rewrite took 76.5011 sec, dropped 828873015 entries (19.2999 %) [P2] Table 3 scan took 45.9568 sec [P2] Table 3 rewrite took 75.2904 sec, dropped 855090812 entries (19.9099 %) [P2] Table 2 scan took 46.1183 sec [P2] Table 2 rewrite took 75.9336 sec, dropped 865639694 entries (20.155 %) Phase 2 took 669.201 sec Wrote plot header with 252 bytes [P3-1] Table 2 took 64.0573 sec, wrote 3429270451 right entries [P3-2] Table 2 took 53.9649 sec, wrote 3429270451 left entries, 3429270451 final [P3-1] Table 3 took 94.9797 sec, wrote 3439715495 right entries [P3-2] Table 3 took 55.2826 sec, wrote 3439715495 left entries, 3439715495 final [P3-1] Table 4 took 94.0258 sec, wrote 3465821735 right entries [P3-2] Table 4 took 55.5479 sec, wrote 3465821735 left entries, 3465821735 final [P3-1] Table 5 took 96.885 sec, wrote 3532508581 right entries [P3-2] Table 5 took 58.263 sec, wrote 3532508581 left entries, 3532508581 final [P3-1] Table 6 took 100.172 sec, wrote 3712903500 right entries [P3-2] Table 6 took 59.5409 sec, wrote 3712903500 left entries, 3712903500 final [P3-1] Table 7 took 68.7765 sec, wrote 4293499264 right entries [P3-2] Table 7 took 69.792 sec, wrote 4293499264 left entries, 4293499264 final Phase 3 took 872.266 sec, wrote 21873719026 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 111.476 sec, final plot size is 108815050523 bytes Total plot creation time was 3431.57 sec (57.1928 min) Started copy to G:\plot-k32-2021-07-13-07-36-b9b74d34bbf065caaa19fe3f90fa55124281b00bee53c4d21cb759e87aac3784.plot Crafting plot 2 out of 4 Process ID: 8780 Number of Threads: 6 Number of Buckets P1: 2^8 (256) Number of Buckets P3+P4: 2^8 (256) Pool Puzzle Hash: 714edbece1589406da826c9df5ee34062b3e9871e7e62ca0d2db7a0ace990c07 Farmer Public Key: ac87f3b326905065413d8fb535f0ae8512d576bdd27f6f125e54ff170db7fcbfeff7823f83bcc4fb808c382871272c41 Working Directory: D:\ Working Directory 2: D:\ Plot Name: plot-k32-2021-07-13-08-33-bc1899407c52ede1f73ffb74170a34e420c905f21fbd5358e40bf5a14e6b2755 [P1] Table 1 took 22.0643 sec [P1] Table 2 took 241.894 sec, found 4294892674 matches [P1] Table 3 took 295.591 sec, found 4294943229 matches [P1] Table 4 took 370.742 sec, found 4294837767 matches Copy to G:\plot-k32-2021-07-13-07-36-b9b74d34bbf065caaa19fe3f90fa55124281b00bee53c4d21cb759e87aac3784.plot finished, took 1140.99 sec, 90.951 MB/s avg. [P1] Table 5 took 357.858 sec, found 4294647167 matches [P1] Table 6 took 321.91 sec, found 4294408360 matches [P1] Table 7 took 242.167 sec, found 4293681120 matches Phase 1 took 1852.36 sec [P2] max_table_size = 4294967296 [P2] Table 7 scan took 16.0124 sec [P2] Table 7 rewrite took 31.0222 sec, dropped 0 entries (0 %) [P2] Table 6 scan took 49.956 sec [P2] Table 6 rewrite took 81.2241 sec, dropped 581391793 entries (13.5383 %) [P2] Table 5 scan took 47.0961 sec [P2] Table 5 rewrite took 77.9601 sec, dropped 762017952 entries (17.7434 %) [P2] Table 4 scan took 46.3043 sec [P2] Table 4 rewrite took 76.4187 sec, dropped 828916621 entries (19.3003 %) [P2] Table 3 scan took 46.4177 sec [P2] Table 3 rewrite took 75.5315 sec, dropped 855127144 entries (19.9101 %) [P2] Table 2 scan took 46.0472 sec [P2] Table 2 rewrite took 78.9283 sec, dropped 865581692 entries (20.1537 %) Phase 2 took 675.862 sec Wrote plot header with 252 bytes [P3-1] Table 2 took 65.3463 sec, wrote 3429310982 right entries [P3-2] Table 2 took 55.0009 sec, wrote 3429310982 left entries, 3429310982 final [P3-1] Table 3 took 94.8452 sec, wrote 3439816085 right entries [P3-2] Table 3 took 54.81 sec, wrote 3439816085 left entries, 3439816085 final [P3-1] Table 4 took 96.3398 sec, wrote 3465921146 right entries [P3-2] Table 4 took 54.9329 sec, wrote 3465921146 left entries, 3465921146 final [P3-1] Table 5 took 96.4551 sec, wrote 3532629215 right entries [P3-2] Table 5 took 56.54 sec, wrote 3532629215 left entries, 3532629215 final [P3-1] Table 6 took 100.401 sec, wrote 3713016567 right entries [P3-2] Table 6 took 59.5958 sec, wrote 3713016567 left entries, 3713016567 final [P3-1] Table 7 took 69.4107 sec, wrote 4293681120 right entries [P3-2] Table 7 took 69.1649 sec, wrote 4293681120 left entries, 4293681120 final Phase 3 took 873.946 sec, wrote 21874375115 entries to final plot [P4] Starting to write C1 and C3 tables [P4] Finished writing C1 and C3 tables [P4] Writing C2 table [P4] Finished writing C2 table Phase 4 took 117.051 sec, final plot size is 108818525763 bytes Total plot creation time was 3519.29 sec (58.6548 min) Started copy to G:\plot-k32-2021-07-13-08-33-bc1899407c52ede1f73ffb74170a34e420c905f21fbd5358e40bf5a14e6b2755.plot Crafting plot 3 out of 4 Process ID: 8780 Number of Threads: 6 Number of Buckets P1: 2^8 (256) Number of Buckets P3+P4: 2^8 (256) Pool Puzzle Hash: 714edbece1589406da826c9df5ee34062b3e9871e7e62ca0d2db7a0ace990c07 Farmer Public Key: ac87f3b326905065413d8fb535f0ae8512d576bdd27f6f125e54ff170db7fcbfeff7823f83bcc4fb808c382871272c41 Working Directory: D:\ Working Directory 2: D:\ Plot Name: plot-k32-2021-07-13-09-32-533b93287196ed06e2c0416699909009e3d8401a47f903f098b8d6a24c44406b [P1] Table 1 took 22.7933 sec [P1] Table 2 took 247.795 sec, found 4294952526 matches [P1] Table 3 took 295.816 sec, found 4294942096 matches Copy to G:\plot-k32-2021-07-13-08-33-bc1899407c52ede1f73ffb74170a34e420c905f21fbd5358e40bf5a14e6b2755.plot finished, took 705.514 sec, 0 MB/s avg.

after this in cmd i have to press enter and then program close, with plot on SSD, and ~200gb tmp files of new plot that failed to proceed after failing to copy finished plot.

[vegaix9]

Same issue here - the problem started in the last week. The machine experiencing this problem is running Ubuntu 21.04 with 4 NVMEs on 4 PCIEx16s transferring plots to 4x14TB drives. Transfers to the storage drives will randomly fail and the drives are no longer usable until a restart. However, this problem also occurs when transferring the plots manually from the NVMEs to the HDDs, so the issue may not be with Madmax and instead with Ubuntu/Linux.

[Dardanet]

The best solutions so far is to use a SSD as buffer. You copy to -d SSD and then use a bash script for moving from ssd to hdd.. Copy to HDD can never be linear.. With this method the used -d ssd has enough time to copy to hdd. And even if a new plot is done, a ssd can take both jobs without problem.

[salmancursher]

I think I found a solution.

When you are plotting, put it in root directory, but in CHIA, dont add this directory to the plots.

Make a new directory like, Ready, and then put plotted plots inside it and then add this directory to chia.

I think chia process when running on Root of the drive, creates fwrite issues for the plotter

I did this for 3 days, and the problem is gone.

[vegaix9]

The best solutions so far is to use a SSD as buffer. You copy to -d SSD and then use a bash script for moving from ssd to hdd.. Copy to HDD can never be linear.. With this method the used -d ssd has enough time to copy to hdd. And even if a new plot is done, a ssd can take both jobs without problem.

This makes the most sense to me. I will attempt this and see if it solves the issue.

[salmancursher]

this above solution is also same to what I said, you have to avoid putting plots inside a directory thats actively used by CHia. that will stop all the issues

On Fri, Jul 16, 2021 at 6:24 PM vegaix9 @.***> wrote:

The best solutions so far is to use a SSD as buffer. You copy to -d SSD and then use a bash script for moving from ssd to hdd.. Copy to HDD can never be linear.. With this method the used -d ssd has enough time to copy to hdd. And even if a new plot is done, a ssd can take both jobs without problem.

This makes the most sense to me. I will attempt this and see if it solves the issue.

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/madMAx43v3r/chia-plotter/issues/777#issuecomment-881425821, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR2WFXZYIRYWCD6OMILBBLLTYATYFANCNFSM5ACF5BLQ .