search

sul-dlss / dlme-airflow

This is a new repository to capture the work related to the DLME ETL Pipeline and establish airflow
Apache License 2.0
1 stars 0 forks source link

`harvest_send_report` task failing #417

Closed jacobthill closed 11 months ago

jacobthill commented 1 year ago

I believe this task is failing for all DAGs but definitely for koch:


Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 175, in execute
    return_value = self.execute_callable()
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/operators/python.py", line 193, in execute_callable
    return self.python_callable(*self.op_args, **self.op_kwargs)
  File "/opt/airflow/dlme_airflow/tasks/send_mapping_report.py", line 13, in email_callback
    send_email(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/email.py", line 59, in send_email
    return backend(
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/email.py", line 120, in send_email_smtp
    send_mime_email(e_from=mail_from, e_to=recipients, mime_msg=msg, conn_id=conn_id, dryrun=dryrun)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/email.py", line 231, in send_mime_email
    smtp_conn = _get_smtp_connection(smtp_host, smtp_port, smtp_timeout, smtp_ssl)
  File "/home/airflow/.local/lib/python3.10/site-packages/airflow/utils/email.py", line 265, in _get_smtp_connection
    else smtplib.SMTP(host=host, port=port, timeout=timeout)
  File "/usr/local/lib/python3.10/smtplib.py", line 255, in __init__
    (code, msg) = self.connect(host, port)
  File "/usr/local/lib/python3.10/smtplib.py", line 341, in connect
    self.sock = self._get_socket(host, port, self.timeout)
  File "/usr/local/lib/python3.10/smtplib.py", line 312, in _get_socket
    return socket.create_connection((host, port), timeout,
  File "/usr/local/lib/python3.10/socket.py", line 845, in create_connection
    raise err
  File "/usr/local/lib/python3.10/socket.py", line 833, in create_connection
    sock.connect(sa)```
aaron-collier commented 1 year ago

i don't thing these values got migrated from AWS.

We had the following values applied to ECS:

    AIRFLOW__SMTP__SMTP_HOST : var.smtp_host
    AIRFLOW__SMTP__SMTP_USER : aws_iam_access_key.dlme-ses.id
    AIRFLOW__SMTP__SMTP_PASSWORD : aws_iam_access_key.dlme-ses.ses_smtp_password_v4
    AIRFLOW__SMTP__SMTP_MAIL_FROM : var.smtp_from

We'll need these values for local deployment. I'll reach out to ops.