n8n-io / n8n

Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.
https://n8n.io
Other
48.94k stars 7.77k forks source link

Outdated duplicate workflow #10836

Open oumbguet opened 2 months ago

oumbguet commented 2 months ago

Bug Description

I'm working on a self-hosted n8n environment. After making some changes to a workflow and running it some times it triggered twice at the same time, but one were executed on the outdated version of the same workflow, while the first one was up to date.

The two executions happening at the same time:

Screenshot 2024-09-16 155948

The up to date workflow (with an http trigger replacing the mqtt one):

image

The outdated workflow (with the now unused mqtt trigger):

image

The workflows execution before and after this duplication are up to date.

The old workflow is not duplicated and does not show in the workflow list:

image

To Reproduce

  1. Have a workflow with an mqtt trigger
  2. Make some changes to it (change the trigger to a webhook one)
  3. Trigger the workflow using the new trigger node
  4. Observe a potential duplication

Expected behavior

The workflow should only be executed once, and should not be executed using its older versions.

Operating System

Ubuntu 24.04 LTS

n8n Version

1.57.0

Node.js Version

18.20.3

Database

PostgreSQL

Execution mode

main (default)

Joffcom commented 2 months ago

Hey @oumbguet,

We have created an internal ticket to look into this which we will be tracking as "GHC-239"

Joffcom commented 2 months ago

Hey @oumbguet,

It seems unlikely to be related to the MQTT node, Do you have n8n running in queue mode? It could be that 2 main instances of n8n were running and that caused the issue.

Can you also share the full workflow? Does anything show in the n8n console log on the server as well?

oumbguet commented 2 months ago

Hey,

Thanks for the response. I don't think it is related to the type of my triggers either.

I don't think my n8n instance is running in queue mode. If I read it correctly the EXECUTIONS_MODE environment variable should be set, but is not:

image

The only log entry that stands out is the following, but I don't think it is related to this error and cannot check that as it has no datetime:

image

The instance is being run in a docker container that is started only once, and only one instance is running in it:

image

Both workflows are the same, except the trigger nodes:

image

The subworkflow number 1:

image

The subworkflow number 2:

image

The subworkflow number 3:

image

Joffcom commented 2 months ago

@netroy do we have some kind of cache internally that could cause this?

tomi commented 1 month ago

It seems like the there was some scenario where the disabled mqtt trigger node wasn't properly closed, and it kept listening for new events. I couldn't however reproduce this locally. @oumbguet do you see the following line in your logs:

Removed triggers and pollers for workflow [id here]

?

Could you provide the logs starting from the time you made the change up to when the duplicate execution happened? Feel free to redact any sensitive data

oumbguet commented 1 month ago

Hello, sadly the n8n instance was running in a docker container that does not keep track of the logs. I cannot check if this line can be found in it, sorry

Joffcom commented 1 month ago

Hey @oumbguet,

Does this mean the issue is no longer occuring?

oumbguet commented 1 month ago

Hello,

Sorry for the late response. This error has occured again today, each executions is duplicated:

image

The two green ones for example are two runs of two different workflow version started at the same time. The red ones are also grouped by pair of old/new workflow versions.

There is too many sensitive datas to write them all out, but the logs do not mention anything other than the errors that my workflows are raising (ApplicationError, Axios Error, ...)