MrPig91 / PSChiaPlotter

A repo for powershell module that helps Chia Plotting
MIT License
182 stars 47 forks source link

MadMax Support? #117

Closed wallacepf closed 3 years ago

wallacepf commented 3 years ago

Hello guys, first of all, thanks for this amazing work!

Would be very nice to support MadMax plotter (even as experimental feature). Looking forward to this!!!

Thanks again

Jacek-ghub commented 3 years ago

@phpfrog,

"They are both plotting software (competitors in a loose sense)"

Actually, I think that you are not right. PSChiaPlotter is rather equivalent to Chia.exe (capital C) - a plotting manager, where MadMax is equivalent to chia.exe (lower case C) - the actual plotter.

Main difference is that MM is heavily parallelized, where chia.exe is barely using just one core at a time. However, all the comparisons that I saw are rather ridiculous, as people compare a single run, and say chia.exe takes 5 hours, but MM just 1 hour. However, during that 5 hours, you can run 5 chia.exe plotters in parallel, where MM can either run one at a time (serializing those 5 plotters), or all 5 at the same time. My take is that the end results would be very similar.

Also, to me, the main problem with PSCP are the legacy issues - initial delays, copy collisions, ... MM doesn't have those issues when runs synchronous jobs (or just a few instances). Of course, the big question is whether MM can fully utilize 24+ cores, but if not, you can still run two, three, ... instances at a time to fully max out all physical cores.

As far as logistics, MM is targeting the Linux platform, and to get it on Windows, either you need to create a Windows project and compile it, or you can use stotiks builds (I think those binaries may be produced in Russia, although he also provides sources, so just compile time would be needed). As MM is just getting off the ground, one problem is with parameters either changing or being added, as such a nightmare for PSCP, as it may not use parameters that are supported by the underlying MM installation

So, you can look at MM as a drop-in replacement for chia.exe, as such using the same settings as are right now for chia.exe, you can run MM in exactly the same way.

The hype is that MM implementation at some point will be able to use graphics card. Although, that may benefit mostly big plotting farms (where your plotters run 24/7 for long months at a time), and you have a factory of disks.

So, for an average user, at the moment there may not be a difference between those two. However, if PSCP would add MM support, potentially the user base would grow dramatically.

Again, the biggest promise of PSCP at the moment is that you can just click Plot! and PSCP will take care of the rest (figure out delays, manage collisions, properly allocate cores/HW threads, handle for you dest folders, ...). Having MM support gives PSCP more options to run potentially better (or maybe not).

Thank you, Jacek

phpfrog commented 3 years ago

Ok, if that's the case, my apologize and disregard my previous comment.

Jacek-ghub commented 3 years ago

Hey, no sweat, all that stuff is new to all of us. If not @wallacepf's post, I would also know nothing about MadMax.

MrPig91 commented 3 years ago

I do want to add the ability to use the Window's version of MadMax, which I think it is "stotiks" like Jacek mentioned, if the user has it available on the computer already. The only issue is I have not tested out the software to see where I need to make the necessary changes. I think it generates shorter logs files based on what I have seen so I would need to write a different Get-ChiaPlotProgress function for that. I also think it tries to conform to the same parameter flags as the official chia plotter, so integration would be pretty easy in that regard. I would just add a text box to the "Basic Plotting" groupbox that people can enter the path to the the exe file. Given that this plotter usually runs one at a time PSChiaPlotter plot manager seems like overkill, but it would be nice to have the plot progress I suppose.

wv-krmkz commented 3 years ago

Windows MadMax support would be very useful and much appreciated! The ability to manage the running of more than one job with MM (even if not in parallel within each job) would be super helpful. Plus, tracking progress and performance, of course.

Jacek-ghub commented 3 years ago

@MrPig91,

I would not be stuck on the notion that MM "usually runs one at a time." Yes, when you have 2-4 physical cores in your machine that may be a preferred solution. However, on hi-phys-core-count CPUs that puts a lot of stress on just one/two TMP folder, as you cannot parallelize them anymore (I guess, that is the reason that under Linux, they ask for ~250GB RAM drive - yikes). Therefore, still running few MM instances may be beneficial, so the need for PSCP.

There is a new post that asks for a bit more granularity that is relevant to SAS drives for tmp folders. Basically, running just one/few instances of MM puts those people with SAS drives out of luck. There is really nothing wrong to run MM in the same way as chia.exe-s are being run right now (one instance per physical core). It will utilize the full power of MM, plus better mitigate resource contentions.

Also, my hopes are that PSCP at some point will do all calculations internally, so the end user will not need to think about core count, how many parallel plotters are running there, etc. whether the plotter is chia.exe or MM. Therefore, I would not try to underestimate the power of PSCP.

cayolblake commented 3 years ago

What is Windows!?

What does it have to do with Chia or the tech industry as a whole!?

:)

Jacek-ghub commented 3 years ago

I just run MadMax for 24 hours (just one instance) and it outperformed chia.exe by about 25%. Again, that was a single instance, running on the same H/W setup. There is still plenty of room for fine tuning MadMax, so potentially it could be more than that 25%.

That to me implies that there is no point to use chia.exe as a ploter at least for some time. Although Bram has recently had either a dumb comment, or rather has opened a rift between Chia.net and MadMax - whatever it is, it is not good for chia community.

I hope that PSChiaPlotter will get support for MadMax soon.

MrPig91 commented 3 years ago

MadMax has now been added. You can provide the path to the madmax exe file in the alternative plotter field. Please note I have not modified the log stats function to parse the log file created by madmax, but I will add that in a later update. I did update the Get-ChiaPlotProgess function so it will show the plot progress for madmax when plotting.