Running datahub on AWS (docker compose)

[alberttwong]

I'm trying to run datahub on AWS via docker compose. I've set it up with the security group that it's open inbound and open outbound for the EC2 instance. Running into this issue.

(datahub) ubuntu@ip-172-31-30-229:~$ datahub docker quickstart
[2024-11-18 18:51:23,661] INFO     {datahub.cli.quickstart_versioning:79} - Unable to connect to GitHub, using default quickstart version mapping config.
[2024-11-18 18:51:23,662] INFO     {datahub.cli.docker_cli:649} - Using quickstart plan: composefile_git_ref='master' docker_tag='head' mysql_tag='8.2'
[2024-11-18 18:51:23,683] INFO     {datahub.cli.docker_cli:667} - compose file name /home/ubuntu/.datahub/quickstart/docker-compose.yml
[2024-11-18 18:51:23,694] INFO     {datahub.cli.docker_cli:879} - Fetching docker-compose file https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose-without-neo4j.quickstart.yml from GitHub
[2024-11-18 18:51:23,816] ERROR    {datahub.entrypoints:218} - Command failed: 403 Client Error: Forbidden for url: https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
Traceback (most recent call last):
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/entrypoints.py", line 205, in main
    sys.exit(datahub(standalone_mode=False, **kwargs))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/upgrade/upgrade.py", line 427, in async_wrapper
    ret = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/telemetry/telemetry.py", line 462, in wrapper
    raise e
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/telemetry/telemetry.py", line 411, in wrapper
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/cli/docker_cli.py", line 668, in quickstart
    download_compose_files(
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/datahub/cli/docker_cli.py", line 882, in download_compose_files
    quickstart_download_response.raise_for_status()
  File "/home/ubuntu/datahub/lib/python3.12/site-packages/requests/models.py", line 1024, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

wget however works just fine.

(datahub) ubuntu@ip-172-31-30-229:~$ wget https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
--2024-11-18 18:55:43--  https://raw.githubusercontent.com/datahub-project/datahub/master/docker/quickstart/docker-compose-without-neo4j.quickstart.yml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.111.133, 185.199.108.133, 185.199.109.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.111.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 10259 (10K) [text/plain]
Saving to: ‘docker-compose-without-neo4j.quickstart.yml’

docker-compose-without-neo4j.quickstart.yml                      100%[==========================================================================================================================================================>]  10.02K  --.-KB/s    in 0.001s  

2024-11-18 18:55:44 (16.9 MB/s) - ‘docker-compose-without-neo4j.quickstart.yml’ saved [10259/10259]

[alberttwong]

running

datahub docker quickstart --quickstart-compose-file /path/to/your/custom-docker-compose.yml

[alberttwong]

the github token (fine grained and classic) didn't work for me.