docker-archive / for-azure

27 stars 18 forks source link

editions_logger Operation not permitted #54

Open djeeg opened 6 years ago

djeeg commented 6 years ago

On a fresh 17.12.0 swarm I have found editions_logger not writing logs.

Couple of other related issues: https://forums.docker.com/t/log-files-are-not-generated-in-azure-storage-account/45103 https://forums.docker.com/t/no-logs-are-showing-in-the-log-storage-account/42266

My investigation below:

Container is running

$ docker ps | grep "editions_logger"

e333f1665895        docker4x/logger-azure:17.12.0-ce-azure1                           "python /server.py"      22 hours ago         Up 22 hours             0.0.0.0:514->514/udp                                  editions_logger

Using Azure Storage Explorer I can see the docker4azurelogs directory in the storage account, so I assume that the container is partial working, ie create_file_service creates the root directory.

Try to see if there are any errors in the container

$ docker attach --sig-proxy=false $(docker ps | grep "editions_logger" | awk '{print $1}')

Nothing is output

Try to re-run the python aggregation script, fails (guess that makes sense, cant run two services at the same time)

$ docker exec -ti editions_logger bash

bash-4.3# cat server.py
bash-4.3# python /server.py
Traceback (most recent call last):
  File "/server.py", line 192, in <module>
    server = SocketServer.UDPServer((HOST,PORT), SyslogUDPHandler)
  File "/usr/lib/python2.7/SocketServer.py", line 417, in __init__
    self.server_bind()
  File "/usr/lib/python2.7/SocketServer.py", line 431, in server_bind
    self.socket.bind(self.server_address)
  File "/usr/lib/python2.7/socket.py", line 228, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 98] Address in use

Try to rerun the container so i can view the logs

$ docker kill $(docker ps | grep "editions_logger" | awk '{print $1}')

Extract the environment variables

$ docker inspect $(docker ps -a | grep "editions_logger" | awk '{print $1}')

Now restart the container

$ docker run -ti --rm -e "SWARM_LOGS_STORAGE_ACCOUNT=..." -e "ACCOUNT_ID=..." -e "APP_ID=..." -e "APP_SECRET=..." -e "TENANT_ID=..." -e "SWARM_FILE_SHARE=swarm-manager000000" -e "ROLE=MANAGER" -e "GROUP_NAME=..." -e "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" docker4x/logger-azure:17.12.0-ce-azure1 python /server.py

Throws this error:

Unable to set current process capabilities: Operation not permitted
mount: permission denied (are you root?)
Traceback (most recent call last):
  File "/server.py", line 191, in <module>
    azure_log = AzureLog()
  File "/server.py", line 81, in __init__
    self.mount_log_share(self.account_name, self.__key, LOGS_SHARE_NAME)
  File "/server.py", line 106, in mount_log_share
    subprocess.check_call(mount_cmd, shell=True)
  File "/usr/lib/python2.7/subprocess.py", line 186, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'mount -t cifs //....file.core.windows.net/docker4azurelogs /logmnt/ -o vers=2.1,username=...,password=...,file_mode=0777,dir_mode=0777,uid=0,gid=0' returned non-zero exit status 1