act-now-coalition / can-scrapers

MIT License
9 stars 13 forks source link

Move etag_skip_flag to signal's context to fix terminal state handling. #414

Closed smcclure17 closed 2 years ago

smcclure17 commented 2 years ago

Originally I thought we could just add arbitrary tags/arguments to a state signal. The code I had in place before just created a signal message that looked like: "{'message': 'No new source data. Skipping...', 'etag_skip_flag': True}" (note that it's just one big string 🤦). So, there never was an etag_skip_flag attribute to look for, meaning the final flow state would never be set to Skipped.

To fix this, this now modifies the signal's context and adds the etag_skip_flag to this dictionary.

My local testing must have been slightly different to not catch this :/