Supervisor / supervisor

Supervisor process control system for Unix (supervisord)
http://supervisord.org
Other
8.55k stars 1.25k forks source link

unix:///var/run/supervisor.sock no such file #480

Closed behnampmdg3 closed 10 years ago

behnampmdg3 commented 10 years ago

I don't seem to be able to get supervisor to run! Here are the details:

vagrant@homestead:/etc/supervisor$ lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 14.04 LTS Release: 14.04 Codename: trusty vagrant@homestead:/etc/su

vagrant@homestead:/etc/supervisor$ sudo apt-get install supervisor Reading package lists... Done Building dependency tree
Reading state information... Done supervisor is already the newest version. 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded.

vagrant@homestead:/etc/supervisor$ sudo service supervisor restart Restarting supervisor: supervisord.

vagrant@homestead:/etc/supervisor$ sudo supervisord /usr/lib/python2.7/dist-packages/supervisor/options.py:295: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security. 'Supervisord is running as root and it is searching '

vagrant@homestead:/etc/supervisor$ sudo supervisorctl unix:///var/run/supervisor.sock no such file supervisor> help

default commands (type help ):

add clear fg open quit remove restart start stop update avail exit maintail pid reload reread shutdown status tail version

supervisor> reread error: <class 'socket.error'>, [Errno 2] No such file or directory: file: /usr/lib/python2.7/socket.py line: 224

supervisor> update error: <class 'socket.error'>, [Errno 2] No such file or directory: file: /usr/lib/python2.7/socket.py line: 224

Here are some more details:

vagrant@homestead:/etc/supervisor$ sudo apt-cache show supervisor Package: supervisor Priority: extra Section: universe/admin Installed-Size: 1485 Maintainer: Ubuntu Developers ubuntu-devel-discuss@lists.ubuntu.com Original-Maintainer: Qijiang Fan fqj1994@gmail.com Architecture: all Version: 3.0b2-1 Depends: python, python-meld3, python-pkg-resources (>= 0.6c7) Filename: pool/universe/s/supervisor/supervisor_3.0b2-1_all.deb Size: 313972 MD5sum: 1e5ee03933451a0f4fc9ff391404f292 SHA1: d9dc47366e99e77b6577a9a82abd538c4982c58e SHA256: f83f89a439cc8de5f2a545edbf20506695e4b477c579a5824c063fbaf94127c1 Description-en: A system for controlling process state Supervisor is a system for controlling and maintaining process state, similar to what init does, but not intended as an init replacement. . It will manage individual processes or groups of processes that need to be started and stopped in order, and it is possible to control individual process state via an rpc mechanism, thus allowing ordinary users to restart processes. Description-md5: b18ffbeaa3a697e8ccaee9cc104ec380 Homepage: http://supervisord.org/ Bugs: https://bugs.launchpad.net/ubuntu/+filebug Origin: Ubuntu

Thanks

msabramo commented 10 years ago

You shouldn't have to do sudo supervisord. It should've already been started. I would try connecting after doing the restart.

behnampmdg3 commented 10 years ago

Same thing:

vagrant@homestead:/etc/supervisor$ sudo service supervisor restart Restarting supervisor: supervisord. vagrant@homestead:/etc/supervisor$ sudo supervisorctl unix:///var/run/supervisor.sock no such file supervisor>

behnampmdg3 commented 10 years ago

Anyone knows about this?

msabramo commented 10 years ago

Is supervisord running?

pgrep -fl supervisor

If it's not, you need to figure out why; maybe look at tail -f /var/log/supervisor/supervisord.log

If it is running, then probably the config is not as expected. I would try using lsof:

vagrant@vagrant-ubuntu-trusty-64:~$ sudo lsof -c supervisord | egrep 'COMMAND|sock'
COMMAND    PID USER   FD   TYPE             DEVICE SIZE/OFF   NODE NAME
superviso 2240 root    5u  unix 0xffff8800374fa300      0t0 171865 /var/run/supervisor.sock.2233

Make sure that supervisord and supervisorctl are both using the same config file. They both support a -c option to explicitly specify the config file; this may come in handy.

behnampmdg3 commented 10 years ago

Hey

This file does not exist on my computer: /var/run/supervisor.sock

Here is the code you sent:

vagrant@homestead:/etc/supervisor$ pgrep -fl supervisor vagrant@homestead:/etc/supervisor$ tail -f /var/log/supervisor/supervisord.log 2014-08-21 06:42:44,699 INFO RPC interface 'supervisor' initialized 2014-08-21 06:42:44,699 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2014-08-21 06:42:44,709 INFO daemonizing the supervisord process 2014-08-21 06:42:44,721 INFO supervisord started with pid 32042 2014-08-21 06:46:24,288 CRIT Supervisor running as root (no user in config file) 2014-08-21 06:46:24,288 WARN Included extra file "/etc/supervisor/conf.d/queue.conf" during parsing 2014-08-21 06:46:24,353 INFO RPC interface 'supervisor' initialized 2014-08-21 06:46:24,353 CRIT Server 'unix_http_server' running without any HTTP authentication checking 2014-08-21 06:46:24,364 INFO daemonizing the supervisord process 2014-08-21 06:46:24,372 INFO supervisord started with pid 32081

vagrant@homestead:/etc/supervisor$ sudo lsof -c supervisord | egrep 'COMMAND|sock' vagrant@homestead:/etc/supervisor$

This is /etc/supervisor/supervisord.conf

vagrant@homestead:/etc/supervisor$ cat supervisord.conf ; supervisor config file

[unix_http_server] file=/var/run/supervisor.sock ; (the path to the socket file) chmod=0700 ; sockef file mode (default 0700)

[supervisord] logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log) pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid) childlogdir=/var/log/supervisor ; ('AUTO' child log dir, default $TEMP)

; the below section must remain in the config file for RPC ; (supervisorctl/web interface) to work, additional interfaces may be ; added by defining them in separate rpcinterface: sections [rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl] serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL for a unix socket

; The [include] section can just contain the "files" setting. This ; setting can list multiple files (separated by whitespace or ; newlines). It can also contain wildcards. The filenames are ; interpreted as relative to this file. Included files cannot ; include files themselves.

[include] files = /etc/supervisor/conf.d/*.conf vagrant@homestead:/etc/supervisor$

mnaberez commented 10 years ago

~Please ask questions about installing or using Supervisor on the mailing list.~

The mailing list is no longer available. Please see potential solutions below. Various scenarios where this may occur and solutions have been posted below.

kovacshuni commented 9 years ago

Same problem here, don't know how to solve it yet. :(

ethought commented 9 years ago

Same problem here

rodrigobendia commented 9 years ago

This should solve this problem:

sudo touch /var/run/supervisor.sock
sudo chmod 777 /var/run/supervisor.sock
sudo service supervisor restart
srikiraju commented 9 years ago

Make sure you have a [unix_http_server] and [rpcinterface:supervisor] section in your conf file

xzyaoi commented 9 years ago

@rodrigobendia it doesn't work.

rex commented 9 years ago

@rodrigobendia I'm having mixed results with those commands as well. As far as I can see, they work. I'm just working inside a Docker container and something keeps killing the /var/run/supervisor.sock file :) I'll try and remember to update here if/when I figure it out.

jartaud commented 9 years ago

@rodrigobendia works for me. Thanks!

vikas-sharma1 commented 9 years ago

I found the fix. Please try.

please add below 2 section to your supervisord.conf -

[unix_http_server] file = /var/tmp/supervisor.sock ;

[rpcinterface:supervisor] supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

Thanks Vikas

codingwithoutcomments commented 9 years ago

@rex did you figure out why the the sock file was being killed inside the docker container?

rex commented 9 years ago

@codingwithoutcomments I didn't, unfortunately. I was working on it for a contract gig at the time that has since ended and I no longer have access to the machine in question. :-/

liutxer commented 8 years ago

may be you conf file raise this error. you should check you conf

raphaelsoul commented 8 years ago

I have had the same problem both on my Ubuntu 14.04 Lts and CentOS 7.0.1 server. It runs well untill last week.

After viewing the issues and checking the command outputting on my server. It should be caused by the permissions of directory.

The configure I used before the outputting of command echo_supervisord_conf,which indicate the file in [unix_http_server] and serverurl in [supervisorctl] section to /tmp/supervisor.sock. permission is 0770.

And I got

unix:///tmp/supervisor.sock no such file

Now I configure it to /var/run/supervisor.sock. And it works now again.

Hope this will be helpful

lucian303 commented 8 years ago

Still getting this problem in docker with Ubuntu 14.04 and supervisord 3.0b2

This issue also seems to be a duplicate of: https://github.com/Supervisor/supervisor/issues/121

I'm going to open a new issue since both are closed.

mnaberez commented 8 years ago

I'm going to open a new issue since both are closed.

Please do not open duplicate issues.

lucian303 commented 8 years ago

@mnaberez I'll just post this here then:

Test system: Clean Ubuntu 14.04 install with supervisord 3.0b2 inside a docker container.

To save time:

Previous issues: https://github.com/Supervisor/supervisor/issues/480 https://github.com/Supervisor/supervisor/issues/121

supervisor.log:

2016-03-30 00:55:52,592 CRIT Supervisor running as root (no user in config file)
2016-03-30 00:55:52,603 INFO supervisord started with pid 6
2016-03-30 00:55:53,605 INFO spawned: 'nginx' with pid 9
2016-03-30 00:55:53,606 INFO spawned: 'php-fpm' with pid 10
2016-03-30 00:55:53,609 INFO spawned: 'newrelic-sysmond' with pid 11
2016-03-30 00:55:53,701 INFO success: newrelic-sysmond entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2016-03-30 00:55:54,919 INFO success: nginx entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-03-30 00:55:54,919 INFO success: php-fpm entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2016-03-30 00:55:54,919 INFO exited: newrelic-sysmond (exit status 0; expected)

conf.d/supervisord.conf:

[supervisord]
nodaemon=true
logfile=/var/log/supervisor/supervisord.log
pidfile=/var/run/supervisord.pid
childlogdir=/var/log/supervisor

[program:nginx]
command=/usr/sbin/nginx -g "daemon off;"
user = root
autostart=true
autorestart=true
priority=10
stdout_events_enabled=true
stderr_events_enabled=true

[program:php-fpm]
command=/usr/sbin/php-fpm7.0 --nodaemonize --fpm-config /etc/php/7.0/fpm/php-fpm.conf
user = root
autostart = true
autorestart = true

[program:newrelic-sysmond]
command=service newrelic-sysmond start
user = root
autostart = true
startsecs=0
mnaberez commented 8 years ago

The configuration file above is missing a [unix_http_server] section, a [supervisorctl] section, and an [rpcinterface:supervisor] section. supervisord is not creating a socket file because it has not been configured to do so in the configuration file. See the configuration guide and the output of echo_supervisord_conf for examples of these sections.

lucian303 commented 8 years ago

@mnaberez Sorry I wasn't clearer. The supervisord config file at /etc/supervisor/supervisord.conf is the default one. That's just the one in conf.d (it has the same name cause that's the only way I could get supervisor to see it).

; supervisor config file

[unix_http_server]
file=/var/run/supervisor.sock   ; (the path to the socket file)
chmod=0700                       ; sockef file mode (default 0700)

[supervisord]
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
childlogdir=/var/log/supervisor            ; ('AUTO' child log dir, default $TEMP)

; the below section must remain in the config file for RPC
; (supervisorctl/web interface) to work, additional interfaces may be
; added by defining them in separate rpcinterface: sections
[rpcinterface:supervisor]
supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

[supervisorctl]
serverurl=unix:///var/run/supervisor.sock ; use a unix:// URL  for a unix socket

; The [include] section can just contain the "files" setting.  This
; setting can list multiple files (separated by whitespace or
; newlines).  It can also contain wildcards.  The filenames are
; interpreted as relative to this file.  Included files *cannot*
; include files themselves.

[include]
files = /etc/supervisor/conf.d/*.conf
mnaberez commented 8 years ago

If this configuration was being used, your log file should contain these lines:

INFO RPC interface 'supervisor' initialized
CRIT Server 'unix_http_server' running without any HTTP authentication checking

The first line is logged when [rpcinterface:supervisor] is read. The second line is logged when a [unix_http_server] section is specified without a username/password.

Since these lines are absent from your log, it does appear that supervisord hasn't been configured correctly. I would check the configuration again, e.g. perhaps supervisord is not using the configuration file(s) you think it is.

lucian303 commented 8 years ago

@mnaberez You're right. That was the problem. I had accidentally explicitly specified the config file as the one in conf.d, which apparently leads to the above also. facepalm Thanks so much for your help!

Longbingfeihua commented 8 years ago

yes itis!

timothy12387 commented 8 years ago

@rodrigobendia it works for me.. thanks

lifeofguenter commented 8 years ago

Why is this being closed with work arounds but no real fix? Does not work here as well (Debian 8).

mnaberez commented 8 years ago

Why is this being closed with work arounds but no real fix?

There's no single fix because there are many reasons why the file might not be found, and we can't control most of them: supervisord might not be running and thus didn't create it, supervisord might be running but the file may have been deleted by something else, there may be a misconfiguration, or it may even be a problem in distribution packaging. We've troubleshot this problem on the mailing list and several issues, but in all the cases I'm aware of, it ended up being one of those issues. We certainly try to fix bugs we are aware of, but at this time we don't know a bug related to this that we can fix.

bbayles commented 8 years ago

For what it's worth, multiple people in this thread have mentioned having problems when running in Docker containers.

Do see this issue in the Docker tracker. There is a problem with Unix sockets inside Docker. I ran into the problem myself some time ago and had to switch to a TCP socket.

lifeofguenter commented 8 years ago

@mnaberez, suggestion: would it then not be possible to introduce some kind of config-check or similar to make sure that the environment and configuration would allow supervisord to run properly? And in doubt would barf something into the logs?

I have been using supervisor successfully on rhel - so if it works, it works pretty well, but if it doesn't, then it gets a real pita to debug.

Most prolly supervisord will be deprecated very soon in favour of systemd which features basically everything (and more) and just works. So not sure how much has to be still invested into supervisord.

mnaberez commented 8 years ago

would it then not be possible to introduce some kind of config-check or similar to make sure that the environment and configuration would allow supervisord to run properly? And in doubt would barf something into the logs?

I think it already does what we know how to do, but maybe we can add a "Troubleshooting" section to the documentation.

0x7061 commented 8 years ago

I have the weird behaviour that under Ubuntu 16.04 the permissons (0700) for /var/run/supervisor.sock are always being reset after reboot which leads to the error in the issue's title. I always have to manually assign mod 0700 after reboot so supervisor can work properly again. Any clues?

raagin commented 8 years ago
sudo systemctl enable supervisor
sudo systemctl start supervisor

I found solution here

StalkAlex commented 8 years ago

Had the same error. Run just supervisord in console and it showed that I simply forgot to create log file for my command, after creating it problem has gone.

nodermann2 commented 8 years ago

sudo service supervisor start

dariogomez commented 8 years ago

thanks @rodrigobendia! It's works for me

dkavraal commented 7 years ago

I believe there are differences between distributor packages and pip package. The error yields of that. (first installing with yum/apt then pip upgrade or vice versa)

aldee07 commented 7 years ago

I know this question is kinda old but for the sake of others who happen to land to this problem, starting the supervisor daemon works for me.

sudo service supervisor start

A more detailed explanation is that in general, when you encounter a "unix:///var/run/blabla.sock no such file" error, most likely the issue is that the daemon of the program in subject (supervisord in this case) wasn't started and thus wasn't able to generate the expected unix socket supervisor.sock file. This file is the communication endpoint for the supervisor foreground commands (such as supervisorctl) that acts as a tunnel/middleman responsible for relaying user-issued commands (ex. supervisorctl reread) to the supervisor service running in background.

You can refer to Unix Domain Socket and to this stackoverflow answer.

fusion2222 commented 7 years ago

In my case, this error was caused by missing log files. Check it if path with log files really exist. If not, create it.

chuanjin commented 7 years ago

I am using Ubuntu 16.04 For me, the issue is that if I use pip to install the latest version (3.3.2), I alway get this "No such file" issue, and I have basically tried all the solutions above but without any luck. However when I remove the latest version and install via apt, and I end up with version 3.2.0, and everything works fine now.

frutality commented 7 years ago

For me it was just sudo systemctl start supervisor command, but for some reason I had to type it twice. First time I type it nothing happens - workers don't start and I can't start them manually. But second time I type it, workers start fine.

I installed supervisor with pip on Ubuntu 16.04. Same behaviour at vagrant machine and Digital Ocean droplet.

farialima commented 7 years ago

To summarize: this problem occurs mainly in two cases:

In the second case, the issue can be complex to understand, because when no configuration is provided for supervisorctl to connect to the server, it will try to connect on http://localhost:9001, which will work if inet_http_server is configured on the server. So, adding a configuration setting for supervisorctl (e.g. connecting as socket in a [supervisorctl] section of your config file) may make it fail even though it worked before.

KeithTt commented 6 years ago

meet this problem...

I was trying alpine OS, in my case, error output this:

# supervisord
/usr/lib/python2.7/site-packages/supervisor/options.py:298: UserWarning: Supervisord is running as root and it is searching for its configuration file in default locations (including its current working directory); you probably want to specify a "-c" argument specifying an absolute path to a configuration file for improved security.
  'Supervisord is running as root and it is searching '
Error: /var/log/supervisor is not an existing directory

Then I create this dir /var/log/supervisor and restart supervisord, it works...

sigurdurb commented 6 years ago

Fixed it by seeing that there still were some processes running check netstat -lp Thats why it was refusing the socket connection for me. So killing those proccesses and Then supervisorctl reload worked.

Edit: Now I think the problem was environment variables, so I set NODE_PATH like this environment=NODE_PATH=%(ENV_NODE_PATH)s while I had already done export NODE_PATH=/usr/lib/node_modules and doing env displayed it. but only the "%(ENV_NODE_PATH)s" part specifically did not work, it always crashed. Removing that and just hardcoding the variable worked however. sudo systemctl start supervisor was also helpful.

homeworkprod commented 6 years ago

Ran into "unix:///var/run/supervisor.sock no such file" as well, on Debian Stretch.

In my case that symptom was caused by program configuration files (/etc/supervisor/conf.d/*.conf) copied from another server that included user directives without the specified user names actually existing on the system in question.

I discovered the helpful error message in the output of systemd's # journalctl -xn, but not in supervisor's own logfile (/var/log/supervisor/supervisor.log in my case).

So now I can use supervisorctl without the error message (see first paragraph) appearing, and it seems to work. And that without setting username and password anywhere and/or switching from unix_http_server to inet_http_server (like I did on the other server for whatever reason years ago).

avinashkuv commented 6 years ago

Replace this [unix_http_server] file=/var/run/supervisor.sock

with

[unix_http_server] file=/tmp/supervisor.sock

then restart the supervisor it will work. it throws an error because the user doesn't have permission create sock file

ercancavusoglu commented 6 years ago

touch /var/run/supervisor.sock sudo supervisord -c /etc/supervisor/supervisord.conf and after , supervisorctl restart all

wtorcaso commented 6 years ago

I had this problem on a fresh Ubuntu, hosted on AWS.

The workaround I found it this:

function sctl() { sudo -H -E supervisorctl -c /etc/supervisor/supervisord.conf "$@"; }

I checked the version of Python, and the executable of supervisorctl, and the value of $PATH, and I grep everything in the /etc/tree. Nothing made a difference. But referencing the config file by its full path made the problem go away.

My theory is that a bare 'supervisorctl' command was not reading the config file, and that "unix:///tmp/supervisor.sock" is a built-in default value for the socket file. This explains the observed behavior, but I have no explanation of why that might be happening.

sdanbury commented 6 years ago

I was hitting this also, so I just ran the following manually again and again until it wasn't complaining anymore (usually it was to do with log directories not existing):

/usr/bin/supervisord -n -c /etc/supervisor/supervisord.conf