search

Bot-detector / bot-detector-scraper

GNU General Public License v2.0
2 stars 2 forks source link

Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. #51

Open extreme4all opened 10 months ago

extreme4all commented 10 months ago 
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - aiokafka.consumer.group_coordinator - _do_commit_offsets - ERROR - OffsetCommit failed for group scraper due to group error ([Error 22] IllegalGenerationError: scraper), will rejoin
2023-12-16 00:10:05,703 - asyncio - default_exception_handler - ERROR - Task exception was never retrieved
future: <Task finished name='Task-22' coro=<receive_messages() done, defined at /project/src/modules/kafka.py:48> exception=CommitFailedError('Commit cannot be completed since the group has already\n            rebalanced and assigned the partitions to another member.\n            This means that the time between subsequent calls to poll()\n            was longer than the configured max_poll_interval_ms, which\n            typically implies that the poll loop is spending too much\n            time message processing. You can address this either by\n            increasing the rebalance timeout with max_poll_interval_ms,\n            or by reducing the maximum size of batches returned in poll()\n            with max_poll_records.\n            ', 'Commit cannot be completed since the group has already rebalanced and may have assigned the partitions to another member')>
Traceback (most recent call last):
  File "/project/src/main_highscore.py", line 149, in <module>
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 953, in commit_offsets
    await asyncio.shield(
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 1066, in _do_commit_offsets
    raise first_error
kafka.errors.IllegalGenerationError: [Error 22] IllegalGenerationError: scraper

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/modules/kafka.py", line 60, in receive_messages
    await consumer.commit()
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/consumer.py", line 568, in commit
    await self._coordinator.commit_offsets(assignment, offsets)
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 958, in commit_offsets
    raise Errors.CommitFailedError(
kafka.errors.CommitFailedError: CommitFailedError: ('Commit cannot be completed since the group has already\n            rebalanced and assigned the partitions to another member.\n            This means that the time between subsequent calls to poll()\n            was longer than the configured max_poll_interval_ms, which\n            typically implies that the poll loop is spending too much\n            time message processing. You can address this either by\n            increasing the rebalance timeout with max_poll_interval_ms,\n            or by reducing the maximum size of batches returned in poll()\n            with max_poll_records.\n            ', 'Commit cannot be completed since the group has already rebalanced and may have assigned the partitions to another member')
2023-12-16 00:10:05,704 - asyncio - default_exception_handler - ERROR - Task exception was never retrieved
future: <Task finished name='Task-22' coro=<receive_messages() done, defined at /project/src/modules/kafka.py:48> exception=CommitFailedError('Commit cannot be completed since the group has already\n            rebalanced and assigned the partitions to another member.\n            This means that the time between subsequent calls to poll()\n            was longer than the configured max_poll_interval_ms, which\n            typically implies that the poll loop is spending too much\n            time message processing. You can address this either by\n            increasing the rebalance timeout with max_poll_interval_ms,\n            or by reducing the maximum size of batches returned in poll()\n            with max_poll_records.\n            ', 'Commit cannot be completed since the group has already rebalanced and may have assigned the partitions to another member')>
Traceback (most recent call last):
  File "/project/src/main_highscore.py", line 149, in <module>
    loop = asyncio.get_running_loop()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: no running event loop

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 953, in commit_offsets
    await asyncio.shield(
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 1066, in _do_commit_offsets
    raise first_error
kafka.errors.IllegalGenerationError: [Error 22] IllegalGenerationError: scraper

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/project/src/modules/kafka.py", line 60, in receive_messages
    await consumer.commit()
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/consumer.py", line 568, in commit
    await self._coordinator.commit_offsets(assignment, offsets)
  File "/usr/local/lib/python3.11/site-packages/aiokafka/consumer/group_coordinator.py", line 958, in commit_offsets
    raise Errors.CommitFailedError(
kafka.errors.CommitFailedError: CommitFailedError: ('Commit cannot be completed since the group has already\n            rebalanced and assigned the partitions to another member.\n            This means that the time between subsequent calls to poll()\n            was longer than the configured max_poll_interval_ms, which\n            typically implies that the poll loop is spending too much\n            time message processing. You can address this either by\n            increasing the rebalance timeout with max_poll_interval_ms,\n            or by reducing the maximum size of batches returned in poll()\n            with max_poll_records.\n            ', 'Commit cannot be completed since the group has already rebalanced and may have assigned the partitions to another member')