export includes debug output and error output in json

rrauenza commented 2 years ago

Environment

self-hosted (onpremise deployment)

Version

21.11.0

Steps to Reproduce

docker-compose run web export -q > /tmp/foo

(-q should really be a default, otherwise you get << skip messages intermixed with the output.)

>> Skipping model <AppConnectBuild>
>> Skipping model <Activity>
>> Skipping model <GroupAssignee>
>> Skipping model <GroupBookmark>

Even with -q, you get:

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
/usr/local/lib/python3.8/site-packages/sentry/utils/email/list_resolver.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is depr
ecated since Python 3.3, and in 3.10 it will stop working
  from collections import Mapping
/usr/local/lib/python3.8/site-packages/memcache.py:1303: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key is '':
/usr/local/lib/python3.8/site-packages/memcache.py:1304: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key_extra_len is 0:
05:31:17 [INFO] sentry.plugins.github: apps-not-configured
[
{
(export continues)

Expected Result

clean json

Actual Result

>> Skipping model <AppConnectBuild>
>> Skipping model <Activity>
>> Skipping model <GroupAssignee>
>> Skipping model <GroupBookmark>

Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
/usr/local/lib/python3.8/site-packages/sentry/utils/email/list_resolver.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is depr
ecated since Python 3.3, and in 3.10 it will stop working
  from collections import Mapping
/usr/local/lib/python3.8/site-packages/memcache.py:1303: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key is '':
/usr/local/lib/python3.8/site-packages/memcache.py:1304: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key_extra_len is 0:
05:31:17 [INFO] sentry.plugins.github: apps-not-configured
[
{
(export continues)

rrauenza commented 2 years ago

Tried this workaround, but we still get messages intermixed... so the internal docker container output is all stdout:

sudo docker-compose run web bash -c "sentry export -q > /tmp/export 2>/dev/stderr && cat /tmp/export " > /tmp/export

$ less /tmp/export
Updating certificates in /etc/ssl/certs...
0 added, 0 removed; done.
Running hooks in /etc/ca-certificates/update.d...
done.
/usr/local/lib/python3.8/site-packages/sentry/utils/email/list_resolver.py:3: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
  from collections import Mapping
/usr/local/lib/python3.8/site-packages/memcache.py:1303: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key is '':
/usr/local/lib/python3.8/site-packages/memcache.py:1304: SyntaxWarning: "is" with a literal. Did you mean "=="?
  if key_extra_len is 0:
05:40:12 [INFO] sentry.plugins.github: apps-not-configured
[
{
  "model": "sites.site",

Only solution I can think of is for there to be added a --output=PATH to sentry export and then the file is cp'ed out.

getsentry-release commented 2 years ago

Routing to @getsentry/open-source for triage. ⏲️

chadwhitacre commented 2 years ago

Putting this on the backlog. Import/export is a bit under-maintained, tbh. :-/

SyntaxWarning: "is" with a literal.

This particular output should be gone now (see https://github.com/getsentry/self-hosted/issues/1162).

getsentry / sentry