Open CarlosEpia opened 1 month ago
The task heat_demand_timeseries.district-heating is failing with this error message:
heat_demand_timeseries.district-heating
Found local files: * /home/powerd/egon100re-run/airflow/logs/dag_id=egon-data-processing-pipeline/run_id=manual2024-05-03T07:42:22.499342+00:00/task_id=heat_demand_timeseries.district-heating/attempt=2.log [2024-05-13, 06:40:33 UTC] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [queued]> [2024-05-13, 06:40:33 UTC] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [queued]> [2024-05-13, 06:40:33 UTC] {taskinstance.py:1361} INFO - Starting attempt 2 of 2 [2024-05-13, 06:40:33 UTC] {taskinstance.py:1382} INFO - Executing <Task(HeatTimeSeries (versioned)): heat_demand_timeseries.district-heating> on 2024-05-03 07:42:22.499342+00:00 [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:57} INFO - Started process 1751280 to run task [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:84} INFO - Running: ['airflow', 'tasks', 'run', 'egon-***-processing-pipeline', 'heat_demand_timeseries.district-heating', 'manual2024-05-03T07:42:22.499342+00:00', '--job-id', '231', '--raw', '--subdir', 'DAGS_FOLDER/dags/pipeline.py', '--cfg-path', '/tmp/tmpgows_y6z'] [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:85} INFO - Job 231: Subtask heat_demand_timeseries.district-heating [2024-05-13, 06:40:33 UTC] {task_command.py:416} INFO - Running <TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [running]> on host at31 [2024-05-13, 06:40:33 UTC] {taskinstance.py:1662} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='airflowstatsd_on = False' AIRFLOW_CTX_DAG_ID='egon-***-processing-pipeline' AIRFLOW_CTX_TASK_ID='heat_demand_timeseries.district-heating' AIRFLOW_CTX_EXECUTION_DATE='2024-05-03T07:42:22.499342+00:00' AIRFLOW_CTX_TRY_NUMBER='2' AIRFLOW_CTX_DAG_RUN_ID='manual2024-05-03T07:42:22.499342+00:00' [2024-05-13, 06:42:10 UTC] {logging_mixin.py:154} INFO - 0:01:36.668452 [2024-05-13, 06:42:10 UTC] {taskinstance.py:1937} ERROR - Task failed with exception Traceback (most recent call last): File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/init.py", line 216, in skip_task result = super(type(task), task).execute(*xs, *ks) File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/operators/python.py", line 192, in execute return_value = self.execute_callable() File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/operators/python.py", line 209, in execute_callable return self.python_callable(self.op_args, self.op_kwargs) File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/heat_demand_timeseries/init.py", line 799, in district_heating create_district_heating_profile_python_like(scenario) File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/heat_demand_timeseries/init.py", line 346, in create_district_heating_profile_python_like slice_df.idp.str[hour] File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/pandas/core/ops/init.py", line 189, in flex_wrapper return self._binop(other, op, level=level, fill_value=fill_value) File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/pandas/core/series.py", line 3070, in _binop result = func(this_vals, other_vals) TypeError: unsupported operand type(s) for : 'float' and 'NoneType' [2024-05-13, 06:42:10 UTC] {taskinstance.py:1400} INFO - Marking task as FAILED. dag_id=egon--processing-pipeline, task_id=heat_demand_timeseries.district-heating, execution_date=20240503T074222, start_date=20240513T064033, end_date=20240513T064210 [2024-05-13, 06:42:10 UTC] {standard_task_runner.py:104} ERROR - Failed to execute job 231 for task heat_demand_timeseries.district-heating (unsupported operand type(s) for *: 'float' and 'NoneType'; 1751280) [2024-05-13, 06:42:10 UTC] {local_task_job_runner.py:228} INFO - Task exited with return code 1 [2024-05-13, 06:42:10 UTC] {logging_mixin.py:154} WARNING - /home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/models/baseoperator.py:1203 AirflowProviderDeprecationWarning: Call to deprecated class PostgresOperator. (Please use airflow.providers.common.sql.operators.sql.SQLExecuteQueryOperator.Also, you can provide hook_params={'schema': <***base>}.) [2024-05-13, 06:42:10 UTC] {taskinstance.py:2778} INFO - 0 downstream tasks scheduled from follow-on schedule check
airflow.providers.common.sql.operators.sql.SQLExecuteQueryOperator
hook_params={'schema': <***base>}
The task
heat_demand_timeseries.district-heatingis failing with this error message:Found local files: * /home/powerd/egon100re-run/airflow/logs/dag_id=egon-data-processing-pipeline/run_id=manual2024-05-03T07:42:22.499342+00:00/task_id=heat_demand_timeseries.district-heating/attempt=2.log [2024-05-13, 06:40:33 UTC] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=non-requeueable deps ti=<TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [queued]> [2024-05-13, 06:40:33 UTC] {taskinstance.py:1159} INFO - Dependencies all met for dep_context=requeueable deps ti=<TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [queued]> [2024-05-13, 06:40:33 UTC] {taskinstance.py:1361} INFO - Starting attempt 2 of 2 [2024-05-13, 06:40:33 UTC] {taskinstance.py:1382} INFO - Executing <Task(HeatTimeSeries (versioned)): heat_demand_timeseries.district-heating> on 2024-05-03 07:42:22.499342+00:00 [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:57} INFO - Started process 1751280 to run task [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:84} INFO - Running: ['airflow', 'tasks', 'run', 'egon-***-processing-pipeline', 'heat_demand_timeseries.district-heating', 'manual2024-05-03T07:42:22.499342+00:00', '--job-id', '231', '--raw', '--subdir', 'DAGS_FOLDER/dags/pipeline.py', '--cfg-path', '/tmp/tmpgows_y6z'] [2024-05-13, 06:40:33 UTC] {standard_task_runner.py:85} INFO - Job 231: Subtask heat_demand_timeseries.district-heating [2024-05-13, 06:40:33 UTC] {task_command.py:416} INFO - Running <TaskInstance: egon-data-processing-pipeline.heat_demand_timeseries.district-heating manual2024-05-03T07:42:22.499342+00:00 [running]> on host at31 [2024-05-13, 06:40:33 UTC] {taskinstance.py:1662} INFO - Exporting env vars: AIRFLOW_CTX_DAG_OWNER='airflowstatsd_on = False' AIRFLOW_CTX_DAG_ID='egon-***-processing-pipeline' AIRFLOW_CTX_TASK_ID='heat_demand_timeseries.district-heating' AIRFLOW_CTX_EXECUTION_DATE='2024-05-03T07:42:22.499342+00:00' AIRFLOW_CTX_TRY_NUMBER='2' AIRFLOW_CTX_DAG_RUN_ID='manual2024-05-03T07:42:22.499342+00:00' [2024-05-13, 06:42:10 UTC] {logging_mixin.py:154} INFO - 0:01:36.668452 [2024-05-13, 06:42:10 UTC] {taskinstance.py:1937} ERROR - Task failed with exception Traceback (most recent call last): File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/init.py", line 216, in skip_task result = super(type(task), task).execute(*xs, *ks) File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/operators/python.py", line 192, in execute return_value = self.execute_callable() File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/operators/python.py", line 209, in execute_callable return self.python_callable(self.op_args, self.op_kwargs) File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/heat_demand_timeseries/init.py", line 799, in district_heating create_district_heating_profile_python_like(scenario) File "/home/powerd/egon100re-run/powerd-data/src/egon/data/datasets/heat_demand_timeseries/init.py", line 346, in create_district_heating_profile_python_like slice_df.idp.str[hour] File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/pandas/core/ops/init.py", line 189, in flex_wrapper return self._binop(other, op, level=level, fill_value=fill_value) File "/home/powerd/egon100re-run/venv/lib/python3.8/site-packages/pandas/core/series.py", line 3070, in _binop result = func(this_vals, other_vals) TypeError: unsupported operand type(s) for : 'float' and 'NoneType' [2024-05-13, 06:42:10 UTC] {taskinstance.py:1400} INFO - Marking task as FAILED. dag_id=egon--processing-pipeline, task_id=heat_demand_timeseries.district-heating, execution_date=20240503T074222, start_date=20240513T064033, end_date=20240513T064210 [2024-05-13, 06:42:10 UTC] {standard_task_runner.py:104} ERROR - Failed to execute job 231 for task heat_demand_timeseries.district-heating (unsupported operand type(s) for *: 'float' and 'NoneType'; 1751280) [2024-05-13, 06:42:10 UTC] {local_task_job_runner.py:228} INFO - Task exited with return code 1 [2024-05-13, 06:42:10 UTC] {logging_mixin.py:154} WARNING - /home/powerd/egon100re-run/venv/lib/python3.8/site-packages/airflow/models/baseoperator.py:1203 AirflowProviderDeprecationWarning: Call to deprecated class PostgresOperator. (Please use
airflow.providers.common.sql.operators.sql.SQLExecuteQueryOperator.Also, you can providehook_params={'schema': <***base>}.) [2024-05-13, 06:42:10 UTC] {taskinstance.py:2778} INFO - 0 downstream tasks scheduled from follow-on schedule check