segmentio / segment-docs

Segment Documentation. Powered by Jekyll.
https://segment.com/docs/
Creative Commons Attribution 4.0 International
47 stars 341 forks source link

[Content]: Clarify `experimental` warning on Synchronous mode #5728

Open ltsSmitty opened 9 months ago

ltsSmitty commented 9 months ago

What article on segment.com/docs is affected?

https://segment.com/docs/connections/sources/catalog/libraries/server/python/#troubleshooting

What part(s) of the article would you like to see updated?

Based on several PRs from around 2019, the Synchronous Mode field that was added to the Python library was a workaround for applications that used Celery, or something similar, which conflicted with flush() function. This field is still labeled as 'experimental' but appears to be stable and a suitable workaround for customers who have a similar Python configuration.

Additional information

See Segment support conversation here where this came up.

stayseesong commented 8 months ago

@ltsSmitty, just to clarify, are you talking about updating this section specifically, and removing the "experimental feature part?

I don't have access to the Segment support conversation link

ltsSmitty commented 8 months ago

@stayseesong Correct, that section. Here's a relevant part of the support conversation for more context:

We do have a section in our Python docs for utilizing analytics with Django here, but I believe you're correct—this issue is likely associated with how the Python library conflicts with Celery. I was able to find an old pull request detailing a similar issue to what you're experiencing.

It also seems that Synchronous Mode may have been added as a workaround for customers experiencing similar issues. It maybe the case that this option is not recommended for common applications that could utilize flush(), but for customers with a similar application to yours, enabling Synchronous Mode appears to be the only available solution.

Our engineers only left this note in the python client file about the sync_mode option:

On program exit, allow the consumer thread to exit cleanly. This prevents exceptions and a messy shutdown when the interpreter is destroyed before the daemon thread finishes execution. However, it is not the same as flushing the queue! To guarantee all messages have been delivered, you'll still need to call flush().

Sorry for not being able to provide a more definitive solution for your use case. I would recommend checking out the Python and Celery repos for more information that could relate to your implementation.

The customer's response:

Thanks for researching this for us. We’ve been using sync_mode for several days now and as far as I can tell it has been working perfectly. The PR you linked was very helpful. Given that the PR was from 2019, maybe it’s time to update your docs to remove the experimental designation on sync_mode and instead detail the use cases it was designed for?