Deletions

[Wizzkidy]

So, great tool but when I setup a folder to delete plots automatically, it deletes the data but not before the transfers stop with a Out of Space error, I then have to click try again, not very automated.

[Ron-ski]

That is odd, I replotted 322TiB recently and I didn't have that problem.

Can you post a screen shot of the main screen with the error message to one side please?

Also a screenshot of the drive setup tab as well.

Another thought, is anything else writing to the destination drives?

[Wizzkidy]

Nothing else is writing to that drive, it is a NAS drive mapped to a letter. I will attempt to get SS it may take awhile as I have cleared a load of space just so it doesn't stop.

[Ron-ski]

Unless I've broken something in a recent update, which is unlikely, something odd is going on.

Basically when there is a plot to move, it looks at the next drive and checks if there is enough free space, if there isn't it checks to see if plots are being deleted, and if so, it then will delete enough plots to make enough space for the new plot.

So it shouldn't initiate a plot move unless there is enough free space, and by the sounds of it, it is the plot move causing the error, so the program determined there was enough free space, and then there wasn't for some reason.

Does it do this all the time, or just occasionally?

[Wizzkidy]

It's done it multiple times, i wake up in the morning and it's sitting on not enough free space with a GUI message to try again. But when I look in plot mover it's deleted some space

[Wizzkidy]

Ok, so I currently have 100GB free on another RAID, I am setting up a folder for deletedplots, which I have moved some plots into. I will select the delete tick box

I will start a move of the 30 new plotted ones to this drive and see if it does it again and take screenshots

[Wizzkidy]

Here is the SS before I start

[Wizzkidy]

So I assume, when it moves the first 56Gb plot, it should then start to delete some of the plots, because it will know that it wont have enough space for the next one?

[Ron-ski]

Yes, you currently have 100GB free, so it will move the first plot, which should then result in 44GB free, it will then delete a plot, and only once enough free space has been created it will trigger the actual move. Its as if something else is using space.

As an experiment you could also reserve some space on the problematic drive, try reserving 115GB.

I would think if this was a common problem, I'd be getting a lot of complaints, so perhaps its something specific to the NAS setup.

[Wizzkidy]

I have tried to reserve space before, it had the same outcome with it deleting space but only after stopping with an out of space error.

I will start this off now and see what happens

Oh and I can safely say that no other sata is being written to this RAID

[Ron-ski]

Can you login to the NAS and see if the free space reported there matches what Windows is reporting?

[Wizzkidy]

Yes it does, (its truenas) but hard to get SS as it changing quickly, and webpage refresh etc

Almost done with this plot now, so see what it does, it cant move another without deleting :)

[Wizzkidy]

Ok, it finished but it deleted the space too late because by the time it had deleted, I already got this message - see SS

But as you can see, it has deleted the plots out of the deletedplots folder, just too late it seems

Oh forgot to say this is Windows 11 22H2

[Wizzkidy]

Another thought came to me, this is running onto a ZFS file system, but it is using SMB as normal. Don't know if that has anything to do with it.

When it deleted the plots, the out of space message appeared a fraction before the space was deleted but it was almost both at the same time

Maybe you can add some sort of monitor that looks at the plot size, then deletes the space way before it gets too low, I don't know im no good with powershell :)

[Ron-ski]

Something odd is going on, the program checks the free space, and it should never initiate a move if there isn't enough free space. Your error screen shows 278GB free, when you started with 100GB free, and my program shows 356GB free, also the error dialogue shows the destination as \flukenas rather than D:.

What is liveplots?

Reserving space will also keep that amount of space free, its why I suggested it earlier. So if you reserved 200GB it would never fill that last 200GB, so in theory you'd never get an out of space warning.

I honestly have no idea why its doing it.

[Wizzkidy]

Yes I just noticed that as well. \flukenas is the NAS, the plotting machine is a physical on the network. Your program is right as I checked the NAS

Windows is reporting less space, and I think it's because the error appeared just prior to it deleting the space, so windows at that millisecond saw it as 278GB

You said earlier "it will then delete a plot, and only once enough free space has been created it will trigger the actual move"

Well it seems to be trigging a move before enough space has been deleted, but at 278GB free even reported by windows, thats more than enough to take the next plot...

confusing..

[Wizzkidy]

Oh also if I click "try again" on the error, it just starts working lol

[Ron-ski]

Yes it is very strange, and even stranger that reserving space doesn't work, I built in that feature as one of the drives I use has our TV recording on it, so I needed to ensure it didn't get filled up, so if you reserve space it subtracts that from the available space, so in theory it just wouldn't run out of space.

My program won't attempt to delete any plots on a given drive whilst there is an ongoing move to it - the drive is marked as busy so it looks for the next available drive.

[Ron-ski]

As a thought, does the NAS drive have compression enabled?

[Wizzkidy]

Good question but no it doesn't, also no deduplication.

I didn't consider 32GB of RAM enough for the amount of data

To confirm

[Ron-ski]

What about snapshots?

As you have 315GB free, can you cut and paste 5 plots manually to the mapped drive - make sure my program is paused/closed though.

[Wizzkidy]

No snapshots, yes I can copy 5 plots over no problem manually with your tool stopped. (that size was during a copy as I had clicked try again) but yea they start copying fine and im sure they would finish but I can do that if you want me to test

I'm convinced it's a windows problem to be honest, did you use Windows 11 to do yours? I assume so

[Ron-ski]

I'm running Windows 10 Pro.

It would be a worthwhile experiment manually copying over a batch of plots.

Whenever I update Chia, it nearly always complains there are still processes running, even though I know there isn't, I just click try again and off it goes and finishes the install. Windows weirdness.

[Wizzkidy]

Hang on Powerrshell has a different size to windows explorer?

Isn't that 339.2GB ?

[Ron-ski]

Nope, its 315GB

[Wizzkidy]

Ok, barking up the wrong tree then. I'm stumped at the moment. These 5 are copying anyway. If the tool dont work for me ill just delete a load and check it each morning.

No worries

[Wizzkidy]

FYI, those 5 plots moved over fine so shrug

[Ron-ski]

If I get chance tonight I'll see if I can add some logging, so we know what free space the program is seeing, I think its the only way we will get any idea on what's happening.