search

nothub / mrpack-install

Modrinth Modpack server deployment
https://pkg.go.dev/github.com/nothub/mrpack-install
MIT License
128 stars 9 forks source link

Large modpacks trigger a SIGABRT #45

Closed mmtawous closed 3 months ago

mmtawous commented 3 months ago

Currently, large modpacks such as the Versatile v7.5 pack triggers a SIGABRT caused by the following

runtime/cgo: pthread_create failed: Resource temporarily unavailable

This is caused by the value in the --dl-threads flag not being used in the Downloader. Here is a snippet of the relevant code from main.

https://github.com/nothub/mrpack-install/blob/91991df13f22fef5a2275027fffd47ec44e20290/web/download/multi.go#L20-L38

nothub commented 3 months ago

Resolved :rocket:

nothub commented 3 months ago

On a sidenote, I tried to reproduce the issue once on my system but I was not able to reproduce it. Can you post the DefaultTasksMax (systemctl show --property=DefaultTasksMax) on the system you experienced the issue on?

mmtawous commented 3 months ago

I am running on an Apple Silicon M2 Macbook Air running MacOS 13.5.1. MacOS doesn't have systemctl so here is what I think is the closest thing to what you are referring to. Oddly enough I don't think that modpack had 4096 things to download so maybe this isn't what I am looking for.

~  ulimit -u
2666

~  sysctl kern.num_threads
kern.num_threads: 20480

~  sysctl kern.num_taskthreads
kern.num_taskthreads: 4096

Adding print statements to the code before the fix doesn't give any more useful info. All 261 routines (there are 261 dependencies) appear to be started before it crashes. I am unfamiliar with how Go handles threads, so I can't offer any more insight into what's happening.

nothub commented 3 months ago

Thanks for the infos, I gonna test around in a ressource constrained VM.