search

DouweM / pixbyt

Pixbyt is a self-hosted Tidbyt app server for advanced apps that aren't supported by the official community app server that you can access through Tidbyt's mobile app.
https://pixbyt.dev
MIT License
93 stars 10 forks source link

Improve Error Logging #12

Closed EricWF closed 6 months ago

EricWF commented 12 months ago

Each time I try to modify my configuration, or add an app, I manage to break something. IDK what. There is little to no discernible log output. The most notable logs I get from the docker container are:

INFO - Resetting orphaned tasks for active dag runs {scheduler_job_runner.py:1611} INFO - Marked 1 SchedulerJob instances as failed

Which isn't helpful.

There are no app logs under logs/ anymore (since I tried switching to the multi tidbyt setup). And the schedule_manager.log provides no useful information apart from:

File Path                                                 PID  Runtime      # DAGs    # Errors  Last Runtime    Last Run
------------------------------------------------------  -----  ---------  --------  ----------  --------------  -------------------
/project/plugins/airflow/dags/meltano_dag_generator.py    247  0.00s             0           1  2.03s           2023-09-15T13:42:12
================================================================================
[2023-09-15T09:43:12.597-0400] {manager.py:888} INFO - 
================================================================================

Please either document or surface more information about failures that occur in the pipeline.

DouweM commented 6 months ago

@EricWF I'm facing the same issue, I'm starting to look into it.

DouweM commented 6 months ago

https://github.com/DouweM/pixbyt/commit/bca2f4301b15898b64a8f61eddfc8939d45c9f41 routes the output of plugins/airflow/dags/meltano_dag_generator.py to logs/scheduler/*.log, which should tell us why logs/schedule_manager.log reports Errors: 1.