search

ericaltendorf / plotman

Chia plotting manager
Apache License 2.0
911 stars 280 forks source link

plotman keeps running plots pass limit #307

Open eric62451 opened 3 years ago

eric62451 commented 3 years ago 
tmpdir_stagger_phase_major: 1
tmpdir_stagger_phase_minor: 2
tmpdir_stagger_phase_limit: 2

tmpdir_max_jobs: 8
global_max_jobs: 8
global_stagger_m: 90
polling_time_s: 20

This is my current config, only have 1 tmp and 1 dest set, no overrides/archive

After running "plotman plot" it shows that it started running 1 plot after 20 seconds, it spawns another one.

based on settings, it should no longer spawn anymore, but instead it keeps spawning more (I know I have only set major 1 and minor 2, but the 2 plots have definitely not reach beyond that stage in 40 seconds) and while checking for plotman status, nothing is showing up.

I am running ubuntu 20.04

I am suspecting it doesnt know how many plots it has ran because the plotman status is not showing anything... but I dont know why it is not showing anything

bradsquicciarini commented 3 years ago

https://github.com/ericaltendorf/plotman/pull/309

bradsquicciarini commented 3 years ago

Did you install plotman with the experimental GUI installer?

eric62451 commented 3 years ago

yea, I just found out now that there is a different way to install chia on ubuntu...

altendky commented 3 years ago

plotman doesn't have a gui nor experimental installer. There are issues with some of the installation options for chia that are resolved in the development branch that result in this. If you install per the readme instructions except with @development instead of @main then you should get these fixes. Let us know if this resolves your issue.

adrienaggery commented 3 years ago

I confirm the issue @eric62451 raises

Here, running on debian My conf:

tmpdir_stagger_phase_major: 2
tmpdir_stagger_phase_minor: 1
tmpdir_stagger_phase_limit: 5

And as you can see, it does not limit spawning new plot processes beyond that limit.

(venv) adrien@fractal7xl:~/chia-blockchain$ plotman status
 plot id    k                      tmp                  dst    wall   phase    tmp     pid   stat      mem   user    sys     io
ccbda017   32      /volumes/nvme/disk1   /volumes/hdd/disk3    0:01     1:1    22G   17721    SLP   393.5M   0:01    22s     0s
1273f80c   32      /volumes/nvme/disk2   /volumes/hdd/disk1    0:31     1:3    97G   15675    SLP     5.5G   0:34   0:01     5s
9f18859e   32      /volumes/nvme/disk1   /volumes/hdd/disk2    1:01     1:3   145G   13525    SLP     5.5G   1:03   0:02     6s
27bcb1d3   32      /volumes/nvme/disk2   /volumes/hdd/disk3    1:31     1:4   163G   11189    SLP     5.5G   1:33   0:04    14s
299991f0   32      /volumes/nvme/disk1   /volumes/hdd/disk1    2:01     1:5   172G    8712    SLP     5.6G   2:02   0:05    15s
eae484e8   32      /volumes/nvme/disk2   /volumes/hdd/disk2    2:48     1:6   178G    5638    SLP     5.6G   2:48   0:06    16s
ba2333bd   32      /volumes/nvme/disk1   /volumes/hdd/disk1    3:20     1:7   175G    3439    SLP     5.4G   3:23   0:08    25s
4a02612a   32      /volumes/nvme/disk2   /volumes/hdd/disk2    3:52     2:2   184G    1099    DSK     1.3G   3:52   0:09   0:05
62e0c3af   32      /volumes/nvme/disk1   /volumes/hdd/disk1    4:24     2:3   217G   30987    RUN     1.5G   4:21   0:10   0:09
ee21ba80   32      /volumes/nvme/disk2   /volumes/hdd/disk2    4:56     3:1   240G   28709    DSK     5.9G   4:50   0:11   0:14
81ca0135   32      /volumes/nvme/disk1   /volumes/hdd/disk1    5:28     3:2   206G   26537    RUN     5.3G   5:11   0:12   0:27
43b3204c   32      /volumes/nvme/disk2   /volumes/hdd/disk2    6:00     3:3   186G   24262    RUN     5.3G   5:33   0:13   0:35
2f9afd04   32      /volumes/nvme/disk1   /volumes/hdd/disk1    6:33     3:5   159G   22083    RUN     5.3G   5:53   0:15   0:48
5b573ff9   32      /volumes/nvme/disk2   /volumes/hdd/disk2    7:05     3:6   162G   19788    DSK     5.3G   6:15   0:16   0:58
d33eb38a   32   /volumes/old_hdd/disk1   /volumes/hdd/disk2   10:21     4:0   165G    6517    DSK     5.3G   6:08   0:17   3:12
eric62451 commented 3 years ago

yea the issue happens when you use the chia gui experimental installer, if you follow the installation process from chia github, plotman will work

adrienaggery commented 3 years ago

No, I'm running debian without desktop environment, so I never used any guy installer. Only the github instructions

altendky commented 3 years ago

@adrienaggery, are you expecting the stagger limit to apply globally? Or per tmp dir?

fabsau commented 3 years ago

I am having the same issue as @adrienaggery. Running plotman 0.3 (main branch). I expect that globally it does not launch a new plot as long there are more (or same number of) plots in major:minor than phase limit allows.

Edit: #191 solved my issue. I was running both plot create and plot interactive.

adrienaggery commented 3 years ago

@altendky I guess the logical behavior would be the limit being applied globally as we see plot time increasing when too much plots are on phase 1. In my case, (Ryzen 9 5900X 12C/24T), the sweet spot is around 6 jobs max on phase 1.

So I'm not really sure why that config is not being taken in account

adrienaggery commented 3 years ago

Small update, I found out the tmpdir_stagger_phase_* works per tmp dir and not globally (as the names states, didn't understand that before) However as I explained in my previous comment, CPU can be a bottleneck and really increase plot times when there's too much concurrency during phase 1. Maybe a global parameter for that could be useful.

altendky commented 3 years ago

Let's control the total process and thread count directly if that is what we want. The phase limit is a very confusing way to do that. We already have a global process limit. I have considered a global thread limit but have not implemented it yet.