error: can't start new thread

[lllama]

This is under Windows. Runs for a bit and then errors. I've adjusted the number of threads (even down to 1) and still run into it. I'll dig into it a little more and update this issue if I find anything else.

[bdamele]

Same issue on Linux Kali

[x4v13r64]

Same on Ubuntu:

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/home/j4v/Git/aws-inventory/aws_inventory/invoker.py", line 97, in _probe_services
    num_threads=config.MAX_THREADS)
  File "/home/j4v/Git/aws-inventory/aws_inventory/invoker.py", line 188, in thread_work
    worker.start()
  File "/usr/lib/python2.7/threading.py", line 736, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

[ghost]

Use Ubuntu Desktop

[x4v13r64]

@chilithomas I've run it in Ubuntu but the issue is with python. @bitsandsalsa have you had the time to look into this? For my part it's a blocker.

[Electronickss]

Ran into this on MacOS Sierra as well

[bitsandsalsa]

We use a pool of worker threads to call APIs for a given service in a region. The threads were never stopping because they did not know when they were done. The result would be too many threads or too many open connections for the process.

[Electronickss]

Command with branch in shell ~/gitProjects/aws-inventory/ develop$: python aws_inventory.py --profile MYPROFILE . . .

Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 801, in __bootstrap_inner
    self.run()
  File "/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 754, in run
    self.__target(*self.__args, **self.__kwargs)
  File "/Users/37272/gitProjects/aws-inventory/aws_inventory/invoker.py", line 97, in _probe_services
    num_threads=config.MAX_THREADS)
  File "/Users/37272/gitProjects/aws-inventory/aws_inventory/invoker.py", line 188, in thread_work
    worker.start()
  File "/usr/local/Cellar/python/2.7.14/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 736, in start
    _start_new_thread(self.__bootstrap, ())
error: can't start new thread

From my testing with the develop branch. Let me know if there's any logs I can send over

[bitsandsalsa]

I'm having trouble reproducing this. In a Windows 10 VM, I could hit it when I get to around 100,000 threads (as reported by threading.active_acount()) if I force a worker pool of around 10,000 threads and wait a few services, but even if somehow your system was creating a thread per API call or even a whole worker pool per API, I don't see how you could hit that many threads. This assumes you are using the default operation blacklist.

Try setting the MAX_THREADS variable in config.py to 1. I am not sure if the bug is caused by something in your AWS account or your execution environment.

1. Are you sure you were on the develop branch? Try a new clean clone of the develop branch.
2. Could you run with the --debug commandline option and add some of that output here.
3. Are you running in some really constrained or minimal environment (e.g., Docker, virtual machine, sandbox)? How much memory is available in the environment? I don't think Homebrew is causing this.
4. Are you using the default operation blacklist file? Since we retrieve ALL data from APIs that have pagination (not just the first page), I wonder if botocore is spawning a ton of threads to parse all of the pages of responses.
5. What permissions does your AWS user have? I tried to reproduce the bug using PowerUserAccess policy.

[Electronickss]

MAX_THREADS: I'd be happy to try that out

1. Yes I am positive. I can try a clean clone of develop next. However I did not make any changes. That doesn't mean I couldn't have had a PEBCAK
2. I will try that first and add it in as an edit
3. I am running directly on MacOS Sierra. I just upgraded to High Sierra since attempting this however I highly doubt that would change anything. I have 16 GB of memory in my laptop
4. To be honest I just cloned it down. I did not set or change any blacklists. If one comes configured, that is what I am using. If it does not than I am not using one.
5. That I am not sure. I will do my best to verify if the other tests are unsuccessful

[Electronickss]

After removing my directory and recloning for specifically the develop branch, my run did complete. I apologize about the incorrect report

[bitsandsalsa]

Thanks for working through this. @j4v @inquisb @lllama I think the bug may be fixed. If you have a chance, see if this works for you too.

[ncc-erik-steringer]

Cleaning out issues. It looks like this was already resolved, so I'm closing this. Thank you for your contribution.