gwu-libraries / TweetSets

Service for creating Twitter datasets for research and archiving.
MIT License
25 stars 2 forks source link

Update installation instructions and/or example.docker-compose.yml #90

Closed kerchner closed 2 years ago

kerchner commented 3 years ago

Installing with the current example.docker-compose.yml results first in this error, due to a container definition that links to a spark-master service which is not defined:

ERROR: Service 'loader' has a link to service 'spark-master' which is undefined.

Borrowing spark-master from the production docker-compose.yml as:

spark-master:
      image: gwul/tweetsets-spark:2.0
      command: bin/spark-class org.apache.spark.deploy.master.Master -h spark-master
      hostname: spark-master
      logging:
          driver: json-file
          options:
              max-size: ${DOCKER_LOG_MAX_SIZE}
              max-file: ${DOCKER_LOG_MAX_FILE}
      environment:
        - MASTER=spark://spark-master:7101
        - SPARK_PUBLIC_DNS=localhost
        - SPARK_MASTER_PORT=7101
        - SPARK_MASTER_WEBUI_PORT=7102
      ports:
        - 7101:7101
        - 7102:7102
      networks:
        - tweetsets
      restart: always

results in:

Traceback (most recent call last):
  File "urllib3/connectionpool.py", line 677, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1252, in request
  File "http/client.py", line 1298, in _send_request
  File "http/client.py", line 1247, in endheaders
  File "http/client.py", line 1026, in _send_output
  File "http/client.py", line 966, in send
  File "docker/transport/unixconn.py", line 43, in connect
PermissionError: [Errno 13] Permission denied

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "requests/adapters.py", line 449, in send
  File "urllib3/connectionpool.py", line 727, in urlopen
  File "urllib3/util/retry.py", line 403, in increment
  File "urllib3/packages/six.py", line 734, in reraise
  File "urllib3/connectionpool.py", line 677, in urlopen
  File "urllib3/connectionpool.py", line 392, in _make_request
  File "http/client.py", line 1252, in request
  File "http/client.py", line 1298, in _send_request
  File "http/client.py", line 1247, in endheaders
  File "http/client.py", line 1026, in _send_output
  File "http/client.py", line 966, in send
  File "docker/transport/unixconn.py", line 43, in connect
urllib3.exceptions.ProtocolError: ('Connection aborted.', PermissionError(13, 'Permission denied'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "docker/api/client.py", line 205, in _retrieve_server_version
  File "docker/api/daemon.py", line 181, in version
  File "docker/utils/decorators.py", line 46, in inner
  File "docker/api/client.py", line 228, in _get
  File "requests/sessions.py", line 543, in get
  File "requests/sessions.py", line 530, in request
  File "requests/sessions.py", line 643, in send
  File "requests/adapters.py", line 498, in send
requests.exceptions.ConnectionError: ('Connection aborted.', PermissionError(13, 'Permission denied'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "bin/docker-compose", line 3, in <module>
  File "compose/cli/main.py", line 67, in main
  File "compose/cli/main.py", line 123, in perform_command
  File "compose/cli/command.py", line 69, in project_from_options
  File "compose/cli/command.py", line 132, in get_project
  File "compose/cli/docker_client.py", line 43, in get_client
  File "compose/cli/docker_client.py", line 170, in docker_client
  File "docker/api/client.py", line 188, in __init__
  File "docker/api/client.py", line 213, in _retrieve_server_version
docker.errors.DockerException: Error while fetching server API version: ('Connection aborted.', PermissionError(13, 'Permission denied'))
[20372] Failed to execute script docker-compose
kerchner commented 2 years ago

This is no longer occurring, using the current example.docker-compose.yml which pulls from image: gwul/tweetsets-spark. Closing ticket.