Error Accessing Executions List

[dominik-mukrecki]

Describe the bug

When attempting to access the executions list in the n8n interface, the application fails to load the list and displays HTTP 500 (Internal Server Error) and 502 (Bad Gateway) errors. This prevents the examination and management of workflow executions.

To Reproduce

Steps to reproduce the behavior:

1. Open the n8n interface in a web browser.
2. Navigate to the Executions section in the UI.
3. The list of executions fails to load, and error messages are displayed in the browser's console.

Expected behavior

The expected behavior is to successfully load and display the list of workflow executions in the n8n interface, allowing users to view and manage their workflow executions.

Environment (please complete the following information):

• OS: Docker environment
• n8n Version: 1.22.1 image from DockerHub
• Database system: PostgreSQL
• Operation mode: queue

Additional context

This issue seems to be related to server-side problems, as indicated by the HTTP status codes 500 and 502. The problem might be due to issues with the PostgreSQL database connection, configuration in queue mode, or an internal error in the n8n server. The issue is consistently reproducible when trying to access the executions list, suggesting a persistent problem with this specific functionality.

[dominik-mukrecki]

Additional Context

During runtime, the application encountered a TypeError, specifically related to accessing a 'data' property on a null object. Here are the details from the log entry:

error | TypeError: Cannot read properties of null (reading 'data') "{ file: 'LoggerProxy.js', function: 'exports.error' }"

Analysis of the Log Entry:

• Error Type: The error is a TypeError, indicating an attempt was made to access a property of a value that is not an object (in this case, null).
• Specific Error: The error message "Cannot read properties of null (reading 'data')" suggests that the code is trying to access the 'data' property of an object that is unexpectedly null at the time of access.
• Location: The error seems to be logged from 'LoggerProxy.js' within the 'exports.error' function. This information is crucial for pinpointing where in the codebase the error is originating.

Possible Causes:

• The error might be occurring due to a response object from an API call or a similar operation being null, and subsequent code attempting to access its 'data' property.
• There may be a logical flaw or missing null checks in the code, particularly in the area indicated by 'LoggerProxy.js'.

Recommended Actions:

• Review the code in 'LoggerProxy.js', specifically in the 'exports.error' function, to identify where the 'data' property is being accessed.
• Add null checks before accessing properties of objects that may potentially be null, especially in asynchronous operations or external data fetches.
• If this error is associated with API calls or external data sources, ensure that the responses are being validated and handled correctly for null or unexpected values.
• Consider implementing more robust error handling and logging to provide clearer insights into such issues in the future.

[dominik-mukrecki]

Additional Observations and Attempts to Resolve the Issue

Downgrading the Version

• Attempting to downgrade to previous versions of the software did not resolve the issue. This suggests that the problem may not be directly related to a specific recent update.

Effective Solution: Manual Database Purging

• The only effective solution so far has been manually purging executions from the database using SQL commands and then rebooting the system. This indicates that the issue might be related to the data stored in the execution_entity table rather than the application code itself.

Uncertainty About the Specific Version Causing the Issue

• It is unclear which specific version of the software introduced this issue. The problem may have started around version 1.22.1, but this is not certain. This lack of clarity suggests a need for further investigation into changes made in and around this version.

Need for Further Investigation

• Given these observations, a more in-depth investigation into both the database state and application behavior across different versions is recommended. This may involve reviewing changelogs, database schemas, and application logs for anomalies or changes introduced in recent versions that could affect the execution_entity table.

Recommendations

• Continue to monitor the system’s behavior after manual database purges to identify any patterns or triggers for the issue.
• Consider setting up a test environment to replicate the issue, allowing for safer experimentation with different versions and configurations without impacting the production environment.

[dominik-mukrecki]

Update on the Issue with Version 1.22.3

• The problem still persists in version 1.22.3.
• Temporarily, downgrading to version 1.20.0 and manually purging the database resolves the issue.

[netroy]

@dominikmukrecki Thanks for the detailed report 💟. Unfortunately I'm not able to reproduce this so far. We are also not seeing this issue for any of our cloud users.

I've created a temporary project on Sentry, which should hopefully help us capture more details about this error, which we can then use to reproduce and fix the issue.

Can you please help us debug this by setting N8N_SENTRY_DSN to https://1df1121064ed9da7bafad223a045185f@o1420875.ingest.sentry.io/4506501935398912, and upgrading to 1.22.3?

Please also make sure that N8N_DIAGNOSTICS_ENABLED is either not set, or set to true.

[dominik-mukrecki]

@netroy You're welcome :)

I've noticed an issue in a workflow of mine within n8n, which seems prone to looping, especially when it runs every 5 minutes without new data. This could be causing the memory leak and the problems with the execution list. My proposed solution is to set up conditions to identify such looping workflows and freeze their execution. Additionally, I'm considering adding a feature to allow users to turn loop protection on or off. Sentry is already in place for in-depth debugging, and I've upgraded to n8n version 1.22.3, ensuring the diagnostic features are enabled. I'm hopeful this will address and prevent these kinds of issues in the future.

[MarleTangible]

@dominik-mukrecki can you check if the MAC address of the n8n container is duplicated by any chance? I had an issue where a connections were randomly failing because some containers on the same network had the same MAC address. This is a known issue with Portainer, especially if you're using the recreate feature.