python / cpython

The Python programming language
https://www.python.org
Other
63.16k stars 30.25k forks source link

Race condition involving SocketServer.TCPServer #79475

Open f9a22689-c58a-4385-931c-4dc55b6ae3a5 opened 5 years ago

f9a22689-c58a-4385-931c-4dc55b6ae3a5 commented 5 years ago
BPO 35294

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'library', 'expert-IO'] title = 'Race condition involving SocketServer.TCPServer' updated_at = user = 'https://bugs.python.org/RuslanDautkhanov' ``` bugs.python.org fields: ```python activity = actor = 'Ruslan Dautkhanov' assignee = 'none' closed = False closed_date = None closer = None components = ['Library (Lib)', 'IO'] creation = creator = 'Ruslan Dautkhanov' dependencies = [] files = [] hgrepos = [] issue_num = 35294 keywords = [] message_count = 1.0 messages = ['330228'] nosy_count = 1.0 nosy_names = ['Ruslan Dautkhanov'] pr_nums = [] priority = 'normal' resolution = None stage = None status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue35294' versions = ['Python 2.7', 'Python 3.6'] ```

f9a22689-c58a-4385-931c-4dc55b6ae3a5 commented 5 years ago

We had a long discussion in https://issues.apache.org/jira/browse/SPARK-26019 regarding a race condition that happens around https://github.com/apache/spark/blob/master/python/pyspark/accumulators.py#L289

The fix I created here - https://github.com/apache/spark/pull/23113/files basically changes bind_andactivate to False in SocketServer.TCPServer.\_init__'s call and manually doing self.server_bind() self.server_activate() self.serve_forever() in that thread instead of main thread.

We have a fix, but we're not sure if this is a bug in Python with multithreading / handling SocketServer.TCPServer ?

Thank you.