search

automatisch / automatisch

The open source Zapier alternative. Build workflow automation without spending time and money.
https://automatisch.io
Other
4.68k stars 351 forks source link

Some flows based on RSS keep firing for "new entries" while previously been handled already #1861

Open benjaminvanrenterghem opened 5 months ago

benjaminvanrenterghem commented 5 months ago

Hello,

Hope you all are well. I have noticed some flows based on RSS feeds keep firing as if they contain "new entries".

In my case the flow ends with a discord notification but that should be irrelevant. Worthy of mention that other RSS -> Discord flows are not being fired more than once, leading me to believe the RSS feed and how it's processed are to blame.

This occurs when the event is published, I don't think the manual firing during unpublished state (while editing) takes local state into account.

Example of such an RSS feed: https://ubuntu.com/blog/feed

Thanks

farukaydin commented 5 months ago

@benjaminvanrenterghem thanks for the feedback.

We identify the new RSS entries based on whether the item content has changed or not. Do you think it's possible for that specific RSS feed that they re-publish the same posts but changing some attributes of them in their RSS feed.

benjaminvanrenterghem commented 5 months ago

The flow for the following RSS feed on the right also resulted in refiring.

image

Neither have a guid or id accompanying the item, leading to a hash being generated from the JSON representation of the "raw" item in its entirety.

Perhaps there could be additional else if blocks which try to generate a hash for the link, title and description elements, or a combination thereof? Would seem to me that that'd reduce the chances of misfiring due to an extra character having less chance of slipping through the cracks, so to speak.

https://github.com/automatisch/automatisch/blob/033b15a1585f23cf005d08f3d6c8e3e83bf97828/packages/backend/src/apps/rss/triggers/new-items-in-feed/new-items-in-feed.js#L4-L20