Unable to read PEM from enviroment

Hi!

Adding a new incident creates the following error seemingly connected to the Google G Suite plugin. I tried loading the private key in a simple python script (which doesn't raise the Value exception -> https://gist.github.com/unbelauscht/f347546cccedce06076aafe5a06ae89f), so its safe to assume the private key is fine.

INFO:     10.10.250.135:42228 - "POST /api/v1/incidents/ HTTP/1.1" 200 OK
web_1        | WARNING:dispatch.plugin.service:Attempted to fetch active plugin, but none were found. PluginType: participant
web_1        | ERROR:dispatch.incident.flows:Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.
web_1        | Traceback (most recent call last):
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 486, in incident_create_flow
web_1        |     tactical_group, notification_group = create_participant_groups(
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 186, in create_participant_groups
web_1        |     tactical_group = plugin.instance.create(
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 58, in wrapper
web_1        |     result = func(*args, **kwargs)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 74, in wrapper
web_1        |     return func(*args, **kwargs)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_google/groups/plugin.py", line 138, in create
web_1        |     client = get_service("admin", "directory_v1", self.scopes)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_google/common.py", line 39, in get_service
web_1        |     credentials = service_account.Credentials.from_service_account_file(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/oauth2/service_account.py", line 233, in from_service_account_file
web_1        |     info, signer = _service_account_info.from_filename(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/_service_account_info.py", line 74, in from_filename
web_1        |     return data, from_dict(data, require=require)
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/_service_account_info.py", line 55, in from_dict
web_1        |     signer = crypt.RSASigner.from_service_account_info(data)
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/crypt/base.py", line 113, in from_service_account_info
web_1        |     return cls.from_string(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/crypt/_cryptography_rsa.py", line 133, in from_string
web_1        |     private_key = serialization.load_pem_private_key(
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 20, in load_pem_private_key
web_1        |     return backend.load_pem_private_key(data, password)
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1217, in load_pem_private_key
web_1        |     return self._load_key(
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1448, in _load_key
web_1        |     self._handle_key_loading_error()
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1490, in _handle_key_loading_error
web_1        |     raise ValueError(
web_1        | ValueError: Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.
web_1        | WARNING:dispatch.plugin.service:Attempted to fetch active plugin, but none were found. PluginType: storage
web_1        | WARNING:dispatch.plugin.service:Attempted to fetch active plugin, but none were found. PluginType: conference
web_1        | WARNING:dispatch.plugin.service:Attempted to fetch active plugin, but none were found. PluginType: conversation
web_1        | WARNING:dispatch.plugin.service:Attempted to fetch active plugin, but none were found. PluginType: document
web_1        | ERROR:dispatch.decorators:Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.
web_1        | Traceback (most recent call last):
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 37, in wrapper
web_1        |     result = func(*args, **kwargs)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/incident/flows.py", line 694, in incident_create_flow
web_1        |     send_incident_created_notifications(incident, db_session)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/incident/messaging.py", line 281, in send_incident_created_notifications
web_1        |     notification_service.filter_and_send(
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/notification/service.py", line 123, in filter_and_send
web_1        |     send(
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/notification/service.py", line 92, in send
web_1        |     plugin.instance.send(
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 58, in wrapper
web_1        |     result = func(*args, **kwargs)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/decorators.py", line 74, in wrapper
web_1        |     return func(*args, **kwargs)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_google/gmail/plugin.py", line 79, in send
web_1        |     client = get_service("gmail", "v1", self.scopes)
web_1        |   File "/usr/local/lib/python3.8/site-packages/dispatch/plugins/dispatch_google/common.py", line 39, in get_service
web_1        |     credentials = service_account.Credentials.from_service_account_file(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/oauth2/service_account.py", line 233, in from_service_account_file
web_1        |     info, signer = _service_account_info.from_filename(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/_service_account_info.py", line 74, in from_filename
web_1        |     return data, from_dict(data, require=require)
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/_service_account_info.py", line 55, in from_dict
web_1        |     signer = crypt.RSASigner.from_service_account_info(data)
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/crypt/base.py", line 113, in from_service_account_info
web_1        |     return cls.from_string(
web_1        |   File "/usr/local/lib/python3.8/site-packages/google/auth/crypt/_cryptography_rsa.py", line 133, in from_string
web_1        |     private_key = serialization.load_pem_private_key(
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/primitives/serialization/base.py", line 20, in load_pem_private_key
web_1        |     return backend.load_pem_private_key(data, password)
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1217, in load_pem_private_key
web_1        |     return self._load_key(
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1448, in _load_key
web_1        |     self._handle_key_loading_error()
web_1        |   File "/usr/local/lib/python3.8/site-packages/cryptography/hazmat/backends/openssl/backend.py", line 1490, in _handle_key_loading_error
web_1        |     raise ValueError(
web_1        | ValueError: Could not deserialize key data. The data may be in an incorrect format or it may be encrypted with an unsupported algorithm.

It seems like failing to read the public key prevents dispatch from communicating with G Suite.

Dispatch Web ENV:

(please note some values are redacted)

affinity:container==f515c979dd7f143670526a77d9a37d573ee62e643b18099e65108ebe396bd733
INCIDENT_RESOURCE_INVESTIGATION_DOCUMENT=google-docs-investigation-document
INCIDENT_RESOURCE_TACTICAL_GROUP=google-group-participant-tactical-group
INCIDENT_RESOURCE_FAQ_DOCUMENT=google-docs-faq-document
INCIDENT_FAQ_DOCUMENT_ID=INCIDENT_FAQ_DOCUMENT_ID
HOSTNAME=1672b6ad74d5
PYTHON_VERSION=3.8.8
POSTGRES_PASSWORD=**redacted**
DATABASE_CREDENTIALS=**redacted**
SLACK_API_BOT_TOKEN=
INCIDENT_ONCALL_SERVICE_ID=None
COMPOSE_PROJECT_NAME=dispatch
INCIDENT_RESOURCE_CONVERSATION_COMMANDS_REFERENCE_DOCUMENT=google-docs-conversation-commands-reference-document
PWD=/
INCIDENT_CONVERSATION_COMMANDS_REFERENCE_DOCUMENT_ID=INCIDENT_CONVERSATION_COMMANDS_REFERENCE_DOCUMENT_ID
DATABASE_NAME=dispatch
INCIDENT_STORAGE_FOLDER_ID=INCIDENT_STORAGE_FOLDER_ID
JIRA_USERNAME=
GOOGLE_SERVICE_ACCOUNT_PRIVATE_KEY_ID=**redacted**
DISPATCH_CONF=/etc/dispatch
ANNUAL_COST_EMPLOYEE=50000
GOOGLE_SERVICE_ACCOUNT_CLIENT_ID=**redacted**
HOME=/root
LANG=C.UTF-8
GOOGLE_SERVICE_ACCOUNT_PROJECT_ID=netflix-dispatch-**redacted**
INCIDENT_RESOURCE_INCIDENT_REVIEW_DOCUMENT=google-docs-incident-review-document
SLACK_SIGNING_SECRET=
SECRET_KEY=**redacted**
DATABASE_PORT=5432
GPG_KEY=**redacted**
INCIDENT_RESOURCE_INCIDENT_TASK=google-docs-incident-task
DATABASE_HOSTNAME=postgres
INCIDENT_DOCUMENT_INVESTIGATION_SHEET_ID=INCIDENT_DOCUMENT_INVESTIGATION_SHEET_ID
GOOGLE_SERVICE_ACCOUNT_CLIENT_EMAIL=nf-dispatch@**redacted**.iam.gserviceaccount.com
JIRA_PASSWORD=
GOOGLE_SERVICE_ACCOUNT_PRIVATE_KEY="-----BEGIN PRIVATE KEY-----\nMIIEvgIBADANBgkqhkiG9w0BAQEFAASCBKgwggSkAgEAAoIBAQDSj1tv0Ds2HxKV\nxPOfDWmNbC9glDwZdikv2t6E/YRsrJPkvMxpgZPkDIhRaSvdmh3DWhufJ9qD/k6l\nAVjdmXUnZs6LYMY5yRZxjq/ftz+gVaeGxUEwz1rF6GWlaR1X0NdF/X3Hlr4LvK24\nu1NuOd3Ucu/10dvsLU/uwDuPMb9uTTfOuSHmdhFf7FVQG5Prn5yMIa3vs5UHPCwT\nJByLlQs//jUigyA010MWsDyinYyKoR6w3EV/NnMYZJhv1efVHyzw6yBcuG+6kNHI\nsvzWcXERNxWpfcE+t6Nm5S0eMuk0zKJmn5/O2UMK6CkYbfRK3VtmeEFJPTtnWY1s\n6PmuPfDtAgMBAAECggEAHpZ9yX3+BAw6hnBQldP8d/N15HlUS7ybGh0ZA0fS43ik\nmW+F4HPW+KOLcAwvt6cKWNUvuwyulUstwJG0B6kh2EH6kkvSMBXSwVdhnEXCI/gd\neJoSMNWGkcaDcaaWASpRA5ViHsr6u7jCodnwskwMui6iRlsgsCwa9wuM+23WKd8h\nM7cohO2FVyHL/RBs1NXzr2Aljf7IFtmF74R6oCt7ZRV7y+QdidVNSRalF+oCH0N/\nRpT0+Pdui0+yPyY21ewwh0D5exKL4ZRAGx9d6nLD+FkpUlGSYgyYWnKWC5J8FGiS\n**redacted**\nBWKsPtUVF1fNMKUwqomYhpLG\n-----END PRIVATE KEY-----\n"
INCIDENT_RESOURCE_INVESTIGATION_SHEET=google-docs-investigation-sheet
TERM=xterm
GOOGLE_SERVICE_ACCOUNT_DELEGATED_ACCOUNT=nf-dispatch@**redacted**.iam.gserviceaccount.com
PIP_DISABLE_PIP_VERSION_CHECK=1
GOOGLE_DOMAIN=**redacted**
SHLVL=1
POSTGRES_USER=dispatch
PAGERDUTY_API_FROM_EMAIL=
PYTHON_PIP_VERSION=21.0.1
GOOGLE_DEVELOPER_KEY=**redacted**
JIRA_PROJECT_KEY=
INCIDENT_RESOURCE_NOTIFICATIONS_GROUP=google-group-participant-notifications-group
PYTHON_GET_PIP_SHA256=c3b81e5d06371e135fb3156dc7d8fd6270735088428c4a9a5ec1f342e2024565
JIRA_ISSUE_TYPE_ID=
JIRA_API_URL=
SLACK_APP_USER_SLUG=
PYTHON_GET_PIP_URL=https://github.com/pypa/get-pip/raw/b60e2320d9e8d02348525bd74e871e466afdf77c/get-pip.py
DISPATCH_JWT_SECRET=**redacted**
PATH=/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
PAGERDUTY_API_KEY=
PIP_NO_CACHE_DIR=off
JIRA_BROWSER_URL=
POSTGRES_DB=dispatch
BUSINESS_HOURS_YEAR=2080
SLACK_WORKSPACE_NAME=
_=/usr/bin/env

pip freeze:

aiofiles==0.6.0
aiohttp==3.7.4.post0
alembic==1.5.7
arrow==1.0.3
async-timeout==3.0.1
attrs==20.3.0
bcrypt==3.2.0
blis==0.7.4
cachetools==3.1.1
catalogue==1.0.0
certifi==2020.12.5
cffi==1.14.5
chardet==3.0.4
click==7.1.2
cryptography==3.4.6
cssselect==1.1.0
cssutils==2.2.0
cymem==2.0.5
decorator==4.4.2
defusedxml==0.7.1
dispatch @ file:///tmp/dist/dispatch-0.1.0.dev0-py38-none-any.whl
dnspython==2.1.0
ecdsa==0.14.1
email-validator==1.1.2
emails==0.6
fastapi==0.63.0
google-api-core==1.26.1
google-api-python-client==2.0.2
google-auth==1.28.0
google-auth-httplib2==0.1.0
google-auth-oauthlib==0.4.3
googleapis-common-protos==1.53.0
h11==0.9.0
httpcore==0.12.3
httplib2==0.19.0
httpx==0.17.1
idna==2.10
Jinja2==2.11.3
jira==2.0.0
joblib==1.0.1
lxml==4.6.2
Mako==1.1.4
MarkupSafe==1.1.1
multidict==5.1.0
murmurhash==1.0.5
numpy==1.20.1
oauth2client==4.1.3
oauthlib==3.1.0
packaging==20.9
pandas==1.2.3
patsy==0.5.1
pbr==5.5.1
pdpyras==4.1.3
plac==1.1.3
premailer==3.7.0
preshed==3.0.5
protobuf==3.15.6
psycopg2-binary==2.8.6
pyasn1==0.4.8
pyasn1-modules==0.2.8
pycparser==2.20
pydantic==1.8.1
PyJWT==2.0.1
pyparsing==2.4.7
python-dateutil==2.8.1
python-editor==1.0.4
python-jose==3.2.0
python-multipart==0.0.5
pytz==2021.1
requests==2.25.1
requests-oauthlib==1.3.0
requests-toolbelt==0.9.1
rfc3986==1.4.0
rsa==4.0
schedule==1.0.0
scipy==1.6.1
sentry-asgi==0.2.0
sentry-sdk==1.0.0
sh==1.14.1
six==1.15.0
slack-sdk==3.4.2
sniffio==1.2.0
spacy==2.3.5
SQLAlchemy==1.3.23
sqlalchemy-filters==0.12.0
SQLAlchemy-Searchable==1.2.0
SQLAlchemy-Utils==0.36.8
srsly==1.0.5
starlette==0.13.6
statsmodels==0.12.2
tabulate==0.8.9
tenacity==7.0.0
thinc==7.4.5
tqdm==4.59.0
typing-extensions==3.7.4.3
uritemplate==3.0.1
urllib3==1.26.4
uvicorn==0.13.4
validators==0.18.2
wasabi==0.8.2
yarl==1.6.3

Thanks for your help!

Screenshot:

Netflix / dispatch

Unable to read PEM from enviroment #992