added a config option to reduce the number of full_node worker processes to reduce ram usage

[grayfallstown]

The Issue: The biggest problem chia fork farmers face is ram usage. Many farmers simply do not have the ram required to run all the forks and my 64GB machine is at its limits as well

One Cause: By default, chia and its forks create number of logical cores - 2 start_full_node process. Each worker process takes 50MB RAM. On a typical 8 core, 16 threads CPU this is 700MB of RAM per chia fork. For 38 Forks this is about 26GB of ram for the full_nodes alone

One Solution: Give farmers the ability to limit the number of worker processes. Make this configurable, so pure, designated full_nodes can still use all cpu cores for their task, but allow farmers to reduce the CPU and RAM load of running a full_node

Pro:

• On an 8 core / 16 thread cpu this can easily save up to 600MB RAM per fork
• On an 32 core / 64 thread cpu this can easily save 3GB RAM per fork
• The ram saving is especially noticable on dual core machines, which likely have little ram available
• Users can choose to reduce ram usage, which allows them to farm more forks including yours if they there previously unable to

Cons: Reducing the number of worker processes can reduce sync speed. This does not affect ongoing synchronization of synced nodes, only a node that is way behind the chain really benefits from many worker processes. The user has full control over that and can reduce the number of worker processes once synced or set them to a good compromise between resource consumption and sync speed, tailored for his machine

Changes:

• 5 Lines of code (two of those line are imports). In a place where the chia team already limits the number of worker processes to 61 I added a config check and apply the configured limit if one is set
• 1 new config option in config.yaml

Who is effected by this change:

• This change only effects users who add the config option multiprocessing_limit to their config.yaml

[ariegrossman]

Question, which section of the config.yaml file should one add the "multiprocessing_limit" paramater?

[grayfallstown]

Question, which section of the config.yaml file should one add the "multiprocessing_limit" paramater?

Just throw it in the first line without any whitespace before it.

[johnnyde94]

Hello there is there a way to add this tweak in windows to other forks of chia?