search

ansible / galaxy

Legacy Galaxy still available as read-only on https://old-galaxy.ansible.com - looking for the new galaxy -> https://github.com/ansible/galaxy_ng
Apache License 2.0
855 stars 329 forks source link

Too many files open in system issues during ansible-galaxy import #61

Open chouseknecht opened 7 years ago

chouseknecht commented 7 years ago

From @sadsfae on February 22, 2017 18:17

Hi Folks,

I'm trying to add a few of my ansible playbooks via github to galaxy and I keep hitting this error:

ERROR! Unexpected Exception: [Errno 23] Too many open files in system: '/tmp/tmplaNlBj'

System: Fedora 24 Ansible version: ansible-2.2.1.0-1.fc24.noarch

I have even upped the open file limits on my system to 65535 and I still get the error and lsof states that I only have about 16k files open for my user.

-=>>ulimit -Hn ; ulimit -Sn
65535
65535
-=>>lsof -u wfoster | wc -l
15618

Here's the command I am using:

ansible-galaxy import sadsfae ansible-elk -vvv

This runs for a few hours then spits out the below traceback.

-=>>ansible-galaxy import sadsfae ansible-elk -vvv
Using /etc/ansible/ansible.cfg as config file
Opened /home/wfoster/.ansible_galaxy
https://galaxy.ansible.com/api/v1/imports/
Successfully submitted import request 60772
https://galaxy.ansible.com/api/v1/imports/?id=60772
Starting import 60772: role_name=ansible-elk repo=sadsfae/ansible-elk
https://galaxy.ansible.com/api/v1/imports/?id=60772
Accessing branch: master
Parsing and validating meta data.
-- snip - several thousand lines --
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
https://galaxy.ansible.com/api/v1/imports/?id=60783
ERROR! Unexpected Exception: [Errno 23] Too many open files in system: '/tmp/tmplaNlBj'
the full traceback was:

Traceback (most recent call last):
  File "/usr/bin/ansible-galaxy", line 103, in <module>
  File "/usr/lib/python2.7/site-packages/ansible/cli/galaxy.py", line 133, in run
  File "/usr/lib/python2.7/site-packages/ansible/cli/__init__.py", line 145, in execute
  File "/usr/lib/python2.7/site-packages/ansible/cli/galaxy.py", line 611, in execute_import
  File "/usr/lib/python2.7/site-packages/ansible/galaxy/api.py", line 58, in wrapped
  File "/usr/lib/python2.7/site-packages/ansible/galaxy/api.py", line 170, in get_import_task
  File "/usr/lib/python2.7/site-packages/ansible/galaxy/api.py", line 58, in wrapped
  File "/usr/lib/python2.7/site-packages/ansible/galaxy/api.py", line 94, in __call_galaxy
  File "/usr/lib/python2.7/site-packages/ansible/module_utils/urls.py", line 908, in open_url
  File "/usr/lib64/python2.7/urllib2.py", line 154, in urlopen
  File "/usr/lib64/python2.7/urllib2.py", line 427, in open
  File "/usr/lib/python2.7/site-packages/ansible/module_utils/urls.py", line 671, in http_request
  File "/usr/lib/python2.7/site-packages/ansible/module_utils/urls.py", line 601, in get_ca_certs
  File "/usr/lib64/python2.7/tempfile.py", line 314, in mkstemp
  File "/usr/lib64/python2.7/tempfile.py", line 244, in _mkstemp_inner
OSError: [Errno 23] Too many open files in system: '/tmp/tmplaNlBj'

Here's my ../meta/main.yml

---
# Ansible galaxy requirements file
#
# Role to setup ELK/EFK stack on a server (CentOS/RHEL only)
- src: https://github.com/sadsfae/ansible-elk
  name: ansible-elk
  version: master

I have also set forks = 25 in my /etc/ansible/ansible.cfg

Copied from original issue: ansible/galaxy-issues#236

chouseknecht commented 7 years ago

From @sadsfae on February 22, 2017 23:29

I've kicked up my system-wide file limits to 818354 and the galaxy import process just seems to run forever (hours and hours) and my open file count slowly ticks upwards.

For what it's worth I'm using a Github access token.

This just repeats over and over in verbose mode:

https://galaxy.ansible.com/api/v1/imports/?id=60943
https://galaxy.ansible.com/api/v1/imports/?id=60943
https://galaxy.ansible.com/api/v1/imports/?id=60943

e.g.

-=>>time ansible-galaxy import sadsfae ansible-nagios -vvv
Using /etc/ansible/ansible.cfg as config file
Opened /home/wfoster/.ansible_galaxy
https://galaxy.ansible.com/api/v1/imports/
Successfully submitted import request 60943
https://galaxy.ansible.com/api/v1/imports/?id=60943
Starting import 60943: role_name=ansible-nagios repo=sadsfae/ansible-nagios
Accessing branch: master
https://galaxy.ansible.com/api/v1/imports/?id=60943
Parsing and validating meta data.
https://galaxy.ansible.com/api/v1/imports/?id=60943
https://galaxy.ansible.com/api/v1/imports/?id=60943

Any idea what I might be doing wrong?

chouseknecht commented 7 years ago

From @sadsfae on February 23, 2017 0:8

Hi Folks, I was using an out-of-date main.yml format. Using the updated one worked, curious why this wouldn't have given me an error instead? Either way this issue is fixed so closing this. Thanks.