search

getsentry / self-hosted

Sentry, feature-complete and packaged up for low-volume deployments and proofs-of-concept
https://develop.sentry.dev/self-hosted/
Other
7.83k stars 1.76k forks source link

Backup Restore Process Fails #874

Closed erfansahaf closed 3 years ago

erfansahaf commented 3 years ago

Version Information

Version: 20.10.1

Steps to Reproduce

Take a quick backup on a working sentry instance.

Takedown all the containers. Remove all volumes and try to install the sentry again by running the ./install.sh script.

Restore the taken backup from the first step with the following command:

docker-compose run --rm -T web import /etc/sentry/backup.json

Expected Result

Restore the backup file without any error.

Actual Result

Facing the following error:

12:29:13 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
/usr/local/lib/python2.7/site-packages/OpenSSL/crypto.py:12: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography import x509
12:29:17 [INFO] sentry.plugins.github: apps-not-configured
Traceback (most recent call last):
  File "/usr/local/bin/sentry", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/__init__.py", line 166, in main
    cli(prog_name=get_prog(), obj={}, max_content_width=100)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 722, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 697, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 1066, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 895, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/decorators.py", line 30, in inner
    return ctx.invoke(f, *args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/click/core.py", line 535, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python2.7/site-packages/sentry/runner/commands/backup.py", line 15, in import_
    for obj in serializers.deserialize("json", src, stream=True, use_natural_keys=True):
  File "/usr/local/lib/python2.7/site-packages/django/core/serializers/json.py", line 88, in Deserializer
    six.reraise(DeserializationError, DeserializationError(e), sys.exc_info()[2])
  File "/usr/local/lib/python2.7/site-packages/django/core/serializers/json.py", line 82, in Deserializer
    for obj in PythonDeserializer(objects, **options):
  File "/usr/local/lib/python2.7/site-packages/django/core/serializers/python.py", line 181, in Deserializer
    raise base.DeserializationError.WithData(e, d['model'], d.get('pk'), field_value)
django.core.serializers.base.DeserializationError: Expecting value: line 1 column 2 (char 1): (sentry.apitoken:pk=2) field_value was '[u'event:admin', u'event:read', u'member:read', u'org:read', u'project:read', u'project:releases', u'team:read']'

Logs

I entered the No answer for creating default user/project prompt in the installation process to keep the database empty with executed migrations.

Checking minimum requirements...

Creating volumes for persistent storage...
Created sentry-data.
Created sentry-postgres.
Created sentry-redis.
Created sentry-zookeeper.
Created sentry-kafka.
Created sentry-clickhouse.
Created sentry-symbolicator.

Creating sentry/sentry.conf.py...
Creating sentry/config.yml...
Creating sentry/requirements.txt...
Creating symbolicator/config.yml...
Creating relay/config.yml...

Generating secret key...
Secret key written to sentry/config.yml

Fetching and updating Docker images...

Some service image(s) must be built from source by running:
    docker-compose build cron snuba-cleanup sentry-cleanup symbolicator-cleanup web worker ingest-consumer post-process-forwarder
20.10.1: Pulling from getsentry/sentry
8d691f585fa8: Pulling fs layer
3fd6980f9df6: Pulling fs layer
c4f890363b98: Pulling fs layer
b8cfb9853b0a: Pulling fs layer
03a65e68bd5f: Pulling fs layer
74fd7792bf9a: Pulling fs layer
bb2024cd6bbc: Pulling fs layer
e74fa5de680f: Pulling fs layer
dae0eea108a7: Pulling fs layer
07cc196e1a97: Pulling fs layer
1881e2b0218c: Pulling fs layer
0b6965ef0fc9: Pulling fs layer
b8cfb9853b0a: Waiting
5df780d49856: Pulling fs layer
ac2e6fded9c0: Pulling fs layer
03a65e68bd5f: Waiting
74fd7792bf9a: Waiting
dae0eea108a7: Waiting
07cc196e1a97: Waiting
bb2024cd6bbc: Waiting
1881e2b0218c: Waiting
ac2e6fded9c0: Waiting
e74fa5de680f: Waiting
5df780d49856: Waiting
c4f890363b98: Verifying Checksum
c4f890363b98: Download complete
b8cfb9853b0a: Verifying Checksum
b8cfb9853b0a: Download complete
03a65e68bd5f: Verifying Checksum
03a65e68bd5f: Download complete
74fd7792bf9a: Verifying Checksum
74fd7792bf9a: Download complete
bb2024cd6bbc: Verifying Checksum
bb2024cd6bbc: Download complete
e74fa5de680f: Verifying Checksum
e74fa5de680f: Download complete
dae0eea108a7: Verifying Checksum
dae0eea108a7: Download complete
3fd6980f9df6: Retrying in 5 seconds
8d691f585fa8: Retrying in 5 seconds
3fd6980f9df6: Retrying in 4 seconds
8d691f585fa8: Retrying in 4 seconds
3fd6980f9df6: Retrying in 3 seconds
8d691f585fa8: Retrying in 3 seconds
3fd6980f9df6: Retrying in 2 seconds
8d691f585fa8: Retrying in 2 seconds
07cc196e1a97: Verifying Checksum
07cc196e1a97: Download complete
3fd6980f9df6: Retrying in 1 second
8d691f585fa8: Retrying in 1 second
1881e2b0218c: Verifying Checksum
1881e2b0218c: Download complete
3fd6980f9df6: Verifying Checksum
3fd6980f9df6: Download complete
0b6965ef0fc9: Verifying Checksum
0b6965ef0fc9: Download complete
5df780d49856: Download complete
8d691f585fa8: Download complete
ac2e6fded9c0: Verifying Checksum
ac2e6fded9c0: Download complete
8d691f585fa8: Pull complete
3fd6980f9df6: Pull complete
c4f890363b98: Pull complete
b8cfb9853b0a: Pull complete
03a65e68bd5f: Pull complete
74fd7792bf9a: Pull complete
bb2024cd6bbc: Pull complete
e74fa5de680f: Pull complete
dae0eea108a7: Pull complete
07cc196e1a97: Pull complete
1881e2b0218c: Pull complete
0b6965ef0fc9: Pull complete
5df780d49856: Pull complete
ac2e6fded9c0: Pull complete
Digest: sha256:c7f37d609b43fdafcb643655ab0d48c21d503e2b390928a5db67253bee4781e2
Status: Downloaded newer image for getsentry/sentry:20.10.1
docker.io/getsentry/sentry:20.10.1

Building and tagging Docker images...

Building web
Step 1/5 : ARG SENTRY_IMAGE
Step 2/5 : ARG SENTRY_PYTHON3
Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}
 ---> b81fbb0a7446
Step 4/5 : COPY . /usr/src/sentry
 ---> 2b93bb261aa6
Step 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fi
 ---> Running in b29b2ba2204b
/usr/local/lib/python2.7/site-packages/OpenSSL/crypto.py:12: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography import x509
DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support
Removing intermediate container b29b2ba2204b
 ---> 58c970ecbd37

Successfully built 58c970ecbd37
Successfully tagged sentry-onpremise-local:latest
smtp uses an image, skipping
memcached uses an image, skipping
redis uses an image, skipping
postgres uses an image, skipping
zookeeper uses an image, skipping
kafka uses an image, skipping
clickhouse uses an image, skipping
snuba-api uses an image, skipping
snuba-consumer uses an image, skipping
snuba-outcomes-consumer uses an image, skipping
snuba-sessions-consumer uses an image, skipping
snuba-transactions-consumer uses an image, skipping
snuba-replacer uses an image, skipping
symbolicator uses an image, skipping
relay uses an image, skipping
nginx uses an image, skipping
Building snuba-cleanup          ... 
Building symbolicator-cleanup   ... 
Building web                    ... 
Building cron                   ... 
Building worker                 ... 
Building ingest-consumer        ... 
Building post-process-forwarder ... 
Building sentry-cleanup         ... 
Building worker
Building web
Building sentry-cleanup
Building post-process-forwarder
Building cron
Step 1/5 : ARG BASE_IMAGE
Step 2/5 : FROM ${BASE_IMAGE}
 ---> 58c970ecbd37
Step 3/5 : RUN apt-get update && apt-get install -y --no-install-recommends cron &&     rm -r /var/lib/apt/lists/*
Step 1/5 : ARG SENTRY_IMAGE
Step 2/5 : ARG SENTRY_PYTHON3
Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}
 ---> b81fbb0a7446
Step 4/5 : COPY . /usr/src/sentry
 ---> Using cache
 ---> 2b93bb261aa6
Step 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fi
 ---> Using cache
 ---> 58c970ecbd37

Successfully built 58c970ecbd37
Successfully tagged sentry-onpremise-local:latest
Building worker                 ... done
Building snuba-cleanup
Step 1/5 : ARG SENTRY_IMAGE
Step 1/5 : ARG SENTRY_IMAGE
Step 2/5 : ARG SENTRY_PYTHON3Step 1/5 : ARG SENTRY_IMAGEStep 2/5 : ARG SENTRY_PYTHON3

Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}

 ---> b81fbb0a7446
 ---> b81fbb0a7446
Step 4/5 : COPY . /usr/src/sentryStep 4/5 : COPY . /usr/src/sentry

 ---> Using cache
 ---> Using cache
 ---> 2b93bb261aa6
 ---> 2b93bb261aa6
Step 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fiStep 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fi

 ---> Using cache
 ---> Using cache
 ---> 58c970ecbd37
 ---> 58c970ecbd37

Successfully built 58c970ecbd37
Successfully built 58c970ecbd37
Successfully tagged sentry-onpremise-local:latest
Step 2/5 : ARG SENTRY_PYTHON3Building cron                   ... done
Successfully tagged sentry-onpremise-local:latest
Building symbolicator-cleanup

Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}
 ---> b81fbb0a7446
Step 4/5 : COPY . /usr/src/sentry
 ---> Using cache
 ---> 2b93bb261aa6
Step 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fi
 ---> Using cache
 ---> 58c970ecbd37

Successfully built 58c970ecbd37
Successfully tagged sentry-onpremise-local:latest
Building web                    ... done
Building post-process-forwarder ... done
Building ingest-consumer
 ---> Running in 84096731edf9
Step 1/5 : ARG BASE_IMAGE
Step 2/5 : FROM ${BASE_IMAGE}
 ---> 1563b0e59ae1
Step 3/5 : RUN apt-get update && apt-get install -y --no-install-recommends cron &&     rm -r /var/lib/apt/lists/*
Step 1/5 : ARG BASE_IMAGE
Step 2/5 : FROM ${BASE_IMAGE}
 ---> f91ead8550da
Step 3/5 : RUN apt-get update && apt-get install -y --no-install-recommends cron &&     rm -r /var/lib/apt/lists/*
Step 1/5 : ARG SENTRY_IMAGE
Step 2/5 : ARG SENTRY_PYTHON3
Step 3/5 : FROM ${SENTRY_IMAGE}${SENTRY_PYTHON3:+-py3}
 ---> b81fbb0a7446
Step 4/5 : COPY . /usr/src/sentry
 ---> Using cache
 ---> 2b93bb261aa6
Step 5/5 : RUN if [ -s /usr/src/sentry/requirements.txt ]; then pip install -r /usr/src/sentry/requirements.txt; fi
 ---> Using cache
 ---> 58c970ecbd37

Successfully built 58c970ecbd37
Successfully tagged sentry-onpremise-local:latest
Building ingest-consumer        ... done
 ---> Running in 1456c4481b8f
 ---> Running in 0a106b6ad16a
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Ign:1 http://deb.debian.org/debian stretch InRelease
Get:2 http://security.debian.org/debian-security stretch/updates InRelease [53.0 kB]
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:3 http://deb.debian.org/debian stretch-updates InRelease [93.6 kB]
Get:4 http://deb.debian.org/debian stretch Release [118 kB]
Get:2 http://security.debian.org/debian-security buster/updates/main amd64 Packages [268 kB]
Get:3 http://deb.debian.org/debian buster InRelease [122 kB]
Get:5 http://deb.debian.org/debian stretch Release.gpg [2410 B]
Get:2 http://deb.debian.org/debian buster InRelease [122 kB]
Get:6 http://security.debian.org/debian-security stretch/updates/main amd64 Packages [656 kB]
Get:3 http://security.debian.org/debian-security buster/updates/main amd64 Packages [268 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:7 http://deb.debian.org/debian stretch-updates/main amd64 Packages [2596 B]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:8 http://deb.debian.org/debian stretch/main amd64 Packages [7080 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [9504 B]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [9504 B]
Fetched 8006 kB in 2s (2721 kB/s)
Reading package lists...Fetched 8423 kB in 3s (2480 kB/s)
Reading package lists...
Reading package lists...Fetched 8423 kB in 4s (2239 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...

Suggested packages:
  anacron logrotate checksecurity
Recommended packages:
  exim4 | postfix | mail-transport-agent
The following NEW packages will be installed:
  cron
Reading package lists...
0 upgraded, 1 newly installed, 0 to remove and 5 not upgraded.
Need to get 95.4 kB of archives.
After this operation, 257 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 cron amd64 3.0pl1-128+deb9u1 [95.4 kB]
Building dependency tree...
Reading state information...
debconf: delaying package configuration, since apt-utils is not installed
Fetched 95.4 kB in 0s (379 kB/s)
Selecting previously unselected package cron.
(Reading database ... 6661 files and directories currently installed.)
Preparing to unpack .../cron_3.0pl1-128+deb9u1_amd64.deb ...
Unpacking cron (3.0pl1-128+deb9u1) ...
The following additional packages will be installed:
  lsb-base sensible-utils
Suggested packages:
  anacron logrotate checksecurity
Recommended packages:
  default-mta | mail-transport-agent
Setting up cron (3.0pl1-128+deb9u1) ...
The following NEW packages will be installed:
  cron lsb-base sensible-utils
Adding group `crontab' (GID 101) ...
Done.
0 upgraded, 3 newly installed, 0 to remove and 35 not upgraded.
Need to get 143 kB of archives.
After this operation, 383 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 sensible-utils all 0.0.12 [15.8 kB]

Building dependency tree...Get:2 http://deb.debian.org/debian buster/main amd64 lsb-base all 10.2019051400 [28.4 kB]
Get:3 http://deb.debian.org/debian buster/main amd64 cron amd64 3.0pl1-134+deb10u1 [99.0 kB]
update-rc.d: warning: start and stop actions are no longer supported; falling back to defaults
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.

Reading state information...
The following additional packages will be installed:
  lsb-base sensible-utils
Suggested packages:
  anacron logrotate checksecurity
Recommended packages:
  default-mta | mail-transport-agent
The following NEW packages will be installed:
  cron lsb-base sensible-utils
debconf: delaying package configuration, since apt-utils is not installed
Fetched 143 kB in 0s (422 kB/s)
Selecting previously unselected package sensible-utils.
(Reading database ... 11918 files and directories currently installed.)
Preparing to unpack .../sensible-utils_0.0.12_all.deb ...
Unpacking sensible-utils (0.0.12) ...
0 upgraded, 3 newly installed, 0 to remove and 13 not upgraded.
Need to get 143 kB of archives.
After this operation, 383 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian buster/main amd64 sensible-utils all 0.0.12 [15.8 kB]
Selecting previously unselected package lsb-base.
Preparing to unpack .../lsb-base_10.2019051400_all.deb ...
Unpacking lsb-base (10.2019051400) ...
Selecting previously unselected package cron.
Preparing to unpack .../cron_3.0pl1-134+deb10u1_amd64.deb ...
Unpacking cron (3.0pl1-134+deb10u1) ...
Get:2 http://deb.debian.org/debian buster/main amd64 lsb-base all 10.2019051400 [28.4 kB]
Removing intermediate container 0a106b6ad16a
 ---> 2e61c0d35a9b
Step 4/5 : COPY entrypoint.sh /entrypoint.sh
Get:3 http://deb.debian.org/debian buster/main amd64 cron amd64 3.0pl1-134+deb10u1 [99.0 kB]
Setting up lsb-base (10.2019051400) ...
Setting up sensible-utils (0.0.12) ...
Setting up cron (3.0pl1-134+deb10u1) ...
Adding group `crontab' (GID 101) ...
Done.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
 ---> cb810dae5180
Step 5/5 : ENTRYPOINT ["/entrypoint.sh"]
debconf: delaying package configuration, since apt-utils is not installed
 ---> Running in 15d30a2bcf30
Fetched 143 kB in 0s (423 kB/s)
Selecting previously unselected package sensible-utils.
(Reading database ... 6829 files and directories currently installed.)
Preparing to unpack .../sensible-utils_0.0.12_all.deb ...
Unpacking sensible-utils (0.0.12) ...
Selecting previously unselected package lsb-base.
Preparing to unpack .../lsb-base_10.2019051400_all.deb ...
Unpacking lsb-base (10.2019051400) ...
Selecting previously unselected package cron.
Preparing to unpack .../cron_3.0pl1-134+deb10u1_amd64.deb ...
Unpacking cron (3.0pl1-134+deb10u1) ...
Removing intermediate container 15d30a2bcf30
 ---> c13ac2c61f76

Successfully built c13ac2c61f76
Successfully tagged symbolicator-cleanup-onpremise-local:latest
Building symbolicator-cleanup   ... done
Setting up lsb-base (10.2019051400) ...
Setting up sensible-utils (0.0.12) ...
Setting up cron (3.0pl1-134+deb10u1) ...
Adding group `crontab' (GID 101) ...
Done.
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Removing intermediate container 84096731edf9
 ---> 684774de32bc
Step 4/5 : COPY entrypoint.sh /entrypoint.sh
 ---> 2d1727993619
Step 5/5 : ENTRYPOINT ["/entrypoint.sh"]
 ---> Running in e27d47e7086c
Removing intermediate container e27d47e7086c
 ---> 498a309c4741

Successfully built 498a309c4741
Successfully tagged sentry-cleanup-onpremise-local:latest
Building sentry-cleanup         ... done
Removing intermediate container 1456c4481b8f
 ---> d1ca587f74e2
Step 4/5 : COPY entrypoint.sh /entrypoint.sh
 ---> 5c1c7fd52732
Step 5/5 : ENTRYPOINT ["/entrypoint.sh"]
 ---> Running in d1dfa13b8f59
Removing intermediate container d1dfa13b8f59
 ---> 93e325c502c0

Successfully built 93e325c502c0
Successfully tagged snuba-cleanup-onpremise-local:latest
Building snuba-cleanup          ... done

Docker images built.
Removing network onpremise_default
Network onpremise_default not found.
Removing network sentry_onpremise_default
Network sentry_onpremise_default not found.
Creating network "sentry_onpremise_default" with the default driver
Creating volume "sentry_onpremise_sentry-secrets" with default driver
Creating volume "sentry_onpremise_sentry-smtp" with default driver
Creating volume "sentry_onpremise_sentry-zookeeper-log" with default driver
Creating volume "sentry_onpremise_sentry-kafka-log" with default driver
Creating volume "sentry_onpremise_sentry-smtp-log" with default driver
Creating volume "sentry_onpremise_sentry-clickhouse-log" with default driver
Bootstrapping and migrating Snuba...
Creating sentry_onpremise_zookeeper_1 ... 
Creating sentry_onpremise_clickhouse_1 ... 
Creating sentry_onpremise_redis_1      ... 
Creating sentry_onpremise_redis_1      ... done
Creating sentry_onpremise_clickhouse_1 ... done
Creating sentry_onpremise_zookeeper_1  ... done
Creating sentry_onpremise_kafka_1      ... 
Creating sentry_onpremise_kafka_1      ... done
+ '[' b = - ']'
+ snuba bootstrap --help
+ set -- snuba bootstrap --no-migrate --force
+ set gosu snuba snuba bootstrap --no-migrate --force
+ exec gosu snuba snuba bootstrap --no-migrate --force
%3|1614428472.194|FAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 2ms in state CONNECT)
%3|1614428473.192|FAIL|rdkafka#producer-1| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)
2021-02-27 12:21:13,192 Connection to Kafka failed (attempt 0)
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 58, in bootstrap
    client.list_topics(timeout=1)
cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
%3|1614428474.195|FAIL|rdkafka#producer-2| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 0ms in state CONNECT)
%3|1614428475.195|FAIL|rdkafka#producer-2| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 1ms in state CONNECT, 1 identical error(s) suppressed)
2021-02-27 12:21:15,197 Connection to Kafka failed (attempt 1)
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 58, in bootstrap
    client.list_topics(timeout=1)
cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
%3|1614428476.199|FAIL|rdkafka#producer-3| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 0ms in state CONNECT)
%3|1614428477.199|FAIL|rdkafka#producer-3| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 0ms in state CONNECT, 1 identical error(s) suppressed)
2021-02-27 12:21:17,200 Connection to Kafka failed (attempt 2)
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 58, in bootstrap
    client.list_topics(timeout=1)
cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
%3|1614428478.203|FAIL|rdkafka#producer-4| [thrd:kafka:9092/bootstrap]: kafka:9092/bootstrap: Connect to ipv4#172.27.0.5:9092 failed: Connection refused (after 0ms in state CONNECT)
2021-02-27 12:21:19,204 Connection to Kafka failed (attempt 3)
Traceback (most recent call last):
  File "/usr/src/snuba/snuba/cli/bootstrap.py", line 58, in bootstrap
    client.list_topics(timeout=1)
cimpl.KafkaException: KafkaError{code=_TRANSPORT,val=-195,str="Failed to get metadata: Local: Broker transport failure"}
2021-02-27 12:21:21,094 Topic cdc created
2021-02-27 12:21:21,095 Topic events created
2021-02-27 12:21:21,095 Topic outcomes created
2021-02-27 12:21:21,095 Topic errors-replacements created
2021-02-27 12:21:21,095 Topic event-replacements created
2021-02-27 12:21:21,095 Topic snuba-commit-log created
2021-02-27 12:21:21,096 Topic ingest-sessions created
Starting sentry_onpremise_redis_1 ... 
Starting sentry_onpremise_clickhouse_1 ... 
Starting sentry_onpremise_clickhouse_1 ... done
Starting sentry_onpremise_redis_1      ... done
Starting sentry_onpremise_zookeeper_1  ... 
Starting sentry_onpremise_zookeeper_1  ... done
Starting sentry_onpremise_kafka_1      ... 
Starting sentry_onpremise_kafka_1      ... done
+ '[' m = - ']'
+ snuba migrations --help
+ set -- snuba migrations migrate --force
+ set gosu snuba snuba migrations migrate --force
+ exec gosu snuba snuba migrations migrate --force
Finished running migrations

Creating additional Kafka topics...
Starting sentry_onpremise_zookeeper_1 ... 
Starting sentry_onpremise_zookeeper_1 ... done
Created topic ingest-attachments.

Creating additional Kafka topics...
Starting sentry_onpremise_zookeeper_1 ... 
Starting sentry_onpremise_zookeeper_1 ... done
Created topic ingest-transactions.

Creating additional Kafka topics...
Starting sentry_onpremise_zookeeper_1 ... 
Starting sentry_onpremise_zookeeper_1 ... done
Created topic ingest-events.

Setting up database...
Starting sentry_onpremise_redis_1 ... 
Starting sentry_onpremise_redis_1 ... done
Creating sentry_onpremise_smtp_1  ... 
Creating sentry_onpremise_postgres_1 ... 
Starting sentry_onpremise_clickhouse_1 ... 
Starting sentry_onpremise_clickhouse_1 ... done
Creating sentry_onpremise_symbolicator_1 ... 
Starting sentry_onpremise_zookeeper_1    ... 
Starting sentry_onpremise_zookeeper_1    ... done
Creating sentry_onpremise_memcached_1    ... 
Starting sentry_onpremise_kafka_1        ... 
Starting sentry_onpremise_kafka_1        ... done
Creating sentry_onpremise_snuba-sessions-consumer_1 ... 
Creating sentry_onpremise_snuba-replacer_1          ... 
Creating sentry_onpremise_snuba-api_1               ... 
Creating sentry_onpremise_snuba-consumer_1          ... 
Creating sentry_onpremise_snuba-outcomes-consumer_1 ... 
Creating sentry_onpremise_snuba-transactions-consumer_1 ... 
Creating sentry_onpremise_memcached_1                   ... done
Creating sentry_onpremise_symbolicator_1                ... done
Creating sentry_onpremise_smtp_1                        ... done
Creating sentry_onpremise_snuba-sessions-consumer_1     ... done
Creating sentry_onpremise_postgres_1                    ... done
Creating sentry_onpremise_snuba-api_1                   ... done
Creating sentry_onpremise_snuba-outcomes-consumer_1     ... done
Creating sentry_onpremise_snuba-consumer_1              ... done
Creating sentry_onpremise_snuba-replacer_1              ... done
Creating sentry_onpremise_snuba-transactions-consumer_1 ... done
12:21:52 [WARNING] sentry.utils.geo: settings.GEOIP_PATH_MMDB not configured.
/usr/local/lib/python2.7/site-packages/OpenSSL/crypto.py:12: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in a future release.
  from cryptography import x509
12:21:57 [INFO] sentry.plugins.github: apps-not-configured
Operations to perform:
  Apply all migrations: admin, auth, contenttypes, jira_ac, nodestore, sentry, sessions, sites, social_auth
Running migrations:
  Applying sentry.0001_initial... OK
  Applying contenttypes.0001_initial... OK
  Applying admin.0001_initial... OK
  Applying admin.0002_logentry_remove_auto_add... OK
  Applying contenttypes.0002_remove_content_type_name... OK
  Applying auth.0001_initial... OK
  Applying auth.0002_alter_permission_name_max_length... OK
  Applying auth.0003_alter_user_email_max_length... OK
  Applying auth.0004_alter_user_username_opts... OK
  Applying auth.0005_alter_user_last_login_null... OK
  Applying auth.0006_require_contenttypes_0002... OK
  Applying auth.0007_alter_validators_add_error_messages... OK
  Applying auth.0008_alter_user_username_max_length... OK
  Applying jira_ac.0001_initial... OK
  Applying nodestore.0001_initial... OK
  Applying sentry.0002_912_to_recent... OK
  Applying sentry.0003_auto_20191022_0122... OK
  Applying sentry.0004_bitfieldtestmodel_blankjsonfieldtestmodel_callabledefaultmodel_jsonfieldtestmodel_jsonfieldwithdefau... OK
  Applying sentry.0005_fix_content_types... OK
  Applying sentry.0006_sentryapp_date_published... OK
  Applying sentry.0007_auto_20191029_0131... OK
  Applying sentry.0008_auto_20191030_0016... OK
  Applying sentry.0009_auto_20191101_1608... OK
  Applying sentry.0010_auto_20191104_1641... OK
  Applying sentry.0011_remove_pagerdutyservice_service_id_from_state... OK
  Applying sentry.0012_remove_pagerdutyservice_service_id... OK
  Applying sentry.0013_auto_20191111_1829... OK
  Applying sentry.0014_delete_sentryappwebhookerror... OK
  Applying sentry.0015_delete_sentryappwebhookerror_db... OK
  Applying sentry.0016_delete_alert_rule_deprecated_fields... OK
  Applying sentry.0017_incident_aggregation... OK
  Applying sentry.0018_discoversavedquery_version... OK
  Applying sentry.0019_auto_20191114_2040... OK
  Applying sentry.0020_auto_20191125_1420... OK
  Applying sentry.0021_auto_20191203_1803... OK
  Applying sentry.0021_auto_20191202_1716... OK
  Applying sentry.0022_merge... OK
  Applying sentry.0023_hide_environment_none_20191126... OK
  Applying sentry.0024_auto_20191230_2052...Nothing to do, skipping migration.

 OK
  Applying sentry.0025_organizationaccessrequest_requester... OK
  Applying sentry.0026_delete_event... OK
  Applying sentry.0027_exporteddata... OK
  Applying sentry.0028_user_reports... OK
  Applying sentry.0029_discover_query_upgrade... OK
  Applying sentry.0030_auto_20200201_0039... OK
  Applying sentry.0031_delete_alert_rules_and_incidents... OK
  Applying sentry.0032_delete_alert_email... OK
  Applying sentry.0033_auto_20200210_2137... OK
  Applying sentry.0034_auto_20200210_2311... OK
  Applying sentry.0035_auto_20200127_1711... OK
  Applying sentry.0036_auto_20200213_0106... OK
  Applying sentry.0037_auto_20200213_0140... OK
  Applying sentry.0038_auto_20200213_1904... OK
  Applying sentry.0039_delete_incidentsuspectcommit... OK
  Applying sentry.0040_remove_incidentsuspectcommittable... OK
  Applying sentry.0041_incidenttrigger_date_modified... OK
  Applying sentry.0042_auto_20200214_1607... OK
  Applying sentry.0043_auto_20200218_1903... OK
  Applying sentry.0044_auto_20200219_0018... OK
  Applying sentry.0045_remove_incidentactivity_event_stats_snapshot... OK
  Applying sentry.0046_auto_20200221_1735... OK
  Applying sentry.0047_auto_20200224_2319... OK
  Applying sentry.0048_auto_20200302_1825... OK
  Applying sentry.0049_auto_20200304_0254... OK
  Applying sentry.0050_auto_20200306_2346... OK
  Applying sentry.0051_fix_auditlog_pickled_data... OK
  Applying sentry.0052_organizationonboardingtask_completion_seen... OK
  Applying sentry.0053_migrate_alert_task_onboarding... OK
  Applying sentry.0054_create_key_transaction... OK
  Applying sentry.0055_query_subscription_status... OK
  Applying sentry.0056_remove_old_functions... OK
  Applying sentry.0057_remove_unused_project_flag... OK
  Applying sentry.0058_project_issue_alerts_targeting... OK
  Applying sentry.0059_add_new_sentry_app_features... OK
  Applying sentry.0060_add_file_eventattachment_index... OK
  Applying sentry.0061_alertrule_partial_index... OK
  Applying sentry.0062_key_transactions_unique_with_owner... OK
  Applying sentry.0063_drop_alertrule_constraint... OK
  Applying sentry.0064_project_has_transactions... OK
  Applying sentry.0065_add_incident_status_method... OK
  Applying sentry.0066_alertrule_manager... OK
  Applying sentry.0067_migrate_rules_alert_targeting... OK
  Applying sentry.0068_project_default_flags... OK
  Applying sentry.0069_remove_tracked_superusers... OK
  Applying sentry.0070_incident_snapshot_support... OK
  Applying sentry.0071_add_default_fields_model_subclass... OK
  Applying sentry.0072_alert_rules_query_changes... OK
  Applying sentry.0073_migrate_alert_query_model... OK
  Applying sentry.0074_add_metric_alert_feature... OK
  Applying sentry.0075_metric_alerts_fix_releases... OK
  Applying sentry.0076_alert_rules_disable_constraints... OK
  Applying sentry.0077_alert_query_col_drop_state... OK
  Applying sentry.0078_incident_field_updates... OK
  Applying sentry.0079_incidents_remove_query_field_state... OK
  Applying sentry.0080_alert_rules_drop_unused_tables_cols... OK
  Applying sentry.0081_add_integraiton_upgrade_audit_log... OK
  Applying sentry.0082_alert_rules_threshold_float... OK
  Applying sentry.0083_add_max_length_webhook_url... OK
  Applying sentry.0084_exported_data_blobs... OK
  Applying sentry.0085_fix_error_rate_snuba_query... OK
  Applying sentry.0086_sentry_app_installation_for_provider... OK
  Applying sentry.0087_fix_time_series_data_type... OK
  Applying sentry.0088_rule_level_resolve_threshold_type... OK
  Applying sentry.0089_rule_level_fields_backfill... OK
  Applying sentry.0090_fix_auditlog_pickled_data_take_2... OK
  Applying sentry.0091_alertruleactivity... OK
  Applying sentry.0092_remove_trigger_threshold_type_nullable... OK
  Applying sentry.0093_make_identity_user_id_textfield... OK
  Applying sentry.0094_cleanup_unreferenced_event_files... OK
  Applying sentry.0095_ruleactivity... OK
  Applying sentry.0096_sentry_app_component_skip_load_on_open... OK
  Applying sentry.0097_add_sentry_app_id_to_sentry_alertruletriggeraction... OK
  Applying sentry.0098_add-performance-onboarding... OK
  Applying sentry.0099_fix_project_platforms... OK
  Applying sentry.0100_file_type_on_event_attachment... OK
  Applying sentry.0101_backfill_file_type_on_event_attachment... OK
  Applying sentry.0102_collect_relay_analytics... OK
  Applying sentry.0103_project_has_alert_filters... OK
  Applying sentry.0104_collect_relay_public_key_usage... OK
  Applying sentry.0105_remove_nullability_of_event_attachment_type... OK
  Applying sentry.0106_service_hook_project_id_nullable... OK
  Applying sentry.0107_remove_spaces_from_slugs... OK
  Applying sentry.0108_update_fileblob_action... OK
  Applying sentry.0109_sentry_app_creator... OK
  Applying sentry.0110_sentry_app_creator_backill... OK
  Applying sentry.0111_snuba_query_event_type... OK
  Applying sentry.0112_groupinboxmodel... OK
  Applying sentry.0113_add_repositoryprojectpathconfig... OK
  Applying sentry.0114_add_unhandled_savedsearch... OK
  Applying sessions.0001_initial... OK
  Applying sites.0001_initial... OK
  Applying sites.0002_alter_domain_unique... OK
  Applying social_auth.0001_initial... OK
12:23:33 [WARNING] sentry: Cannot initiate onboarding for organization (1) due to missing owners
Created internal Sentry project (slug=internal, id=1)

Would you like to create a user account now? [Y/n]: n

Run `sentry createuser` to do this later.

Creating missing DSNs
Correcting Group.num_comments counter

Generating Relay credentials...
Relay credentials written to relay/credentials.json

----------------
You're all done! Run the following command to get Sentry running:

  docker-compose up -d
chadwhitacre commented 3 years ago

Looks like something is messed up in the GitHub plugin. As a workaround, can you install the plugin before exporting (or expunge it from the export) before importing?

chadwhitacre commented 3 years ago

Or if that's not it, can you otherwise whittle down what exactly in your export is causing the error on import? Can you send us a minimal test case to repro the bug?

erfansahaf commented 3 years ago

I deleted the previous project and installed a new fresh Sentry with the latest version (21) so I don't have access to logs/database anymore. All I have is the exported backup file.

chadwhitacre commented 3 years ago

Can you bisect the export file to find the offending bit? Whittle it down that way?

erfansahaf commented 3 years ago

@chadwhitacre Since it contains sensitive data (password, etc.) I prefer to share it privately with you, if possible.

chadwhitacre commented 3 years ago

Did you figure out which part of the export file is choking the import process? Which is it?

github-actions[bot] commented 3 years ago

This issue has gone three weeks without activity. In another week, I will close it.

But! If you comment or otherwise update it, I will reset the clock, and if you label it Status: Backlog or Status: In Progress, I will leave it alone ... forever!

"A weed is but an unloved flower." ― Ella Wheeler Wilcox 🥀

BYK commented 3 years ago

This should be fixed with getsentry/sentry#24461.