search

sergree / DolboNet

🤪 Русскоязычный чат-бот для Discord на архитектуре Transformer
GNU General Public License v3.0
58 stars 17 forks source link

Вечно у меня какие-то проблемы #26

Open AnUser1 opened 1 year ago

AnUser1 commented 1 year ago

В общем, я хочу сделать публичную версию для реплита, но мне выдает вот эту ошибку когда я пингую бота и что-то пишу, что делать?

Он отвечает, но очень долго, даже после того как убрал USE_DELAY, и вылетают эти ошибки ниже

Loop thread traceback (most recent call last):
  File "bot.py", line 31, in <module>
    main_client.run(token)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 828, in run
    asyncio.run(runner())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 409, in _run_event
    await coro(*args, **kwargs)
  File "/home/runner/shxhxhcht1hdud-2/core/main_client.py", line 115, in on_message
    output_tensor = predictor.decode_sequence(
  File "/home/runner/shxhxhcht1hdud-2/core/predictor.py", line 62, in decode_sequence
    output_tokens = model.predict([input_seq, target_seq])
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
    return fn(*args, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/engine/training.py", line 2350, in predict
    tmp_batch_outputs = self.predict_function(iterator)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/util/traceback_utils.py", line 150, in error_handler
    return fn(*args, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 880, in __call__
    result = self._call(*args, **kwds)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/polymorphic_function.py", line 919, in _call
    results = self._variable_creation_fn(*args, **kwds)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/tracing_compiler.py", line 134, in __call__
    return concrete_function._call_flat(
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/monomorphic_function.py", line 1745, in _call_flat
    return self._build_call_outputs(self._inference_function.call(
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/polymorphic_function/monomorphic_function.py", line 378, in call
    outputs = execute.execute(
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/eager/execute.py", line 52, in quick_execute
    tensors = pywrap_tfe.TFE_Py_Execute(ctx._handle, device_name, op_name,

1/1 [==============================] - 10s 10s/step
2023-01-13 19:58:37 WARNING  discord.gateway Shard ID None heartbeat blocked for more than 20 seconds.
Loop thread traceback (most recent call last):
  File "bot.py", line 31, in <module>
    main_client.run(token)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 828, in run
    asyncio.run(runner())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 409, in _run_event
    await coro(*args, **kwargs)
  File "/home/runner/shxhxhcht1hdud-2/core/main_client.py", line 115, in on_message
    output_tensor = predictor.decode_sequence(
  File "/home/runner/shxhxhcht1hdud-2/core/predictor.py", line 62, in decode_sequence
    output_tokens = model.predict([input_seq, target_seq])
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
    return fn(*args, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/engine/training.py", line 2346, in predict
    for _, iterator in data_handler.enumerate_epochs():  # Single epoch.
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/engine/data_adapter.py", line 1304, in enumerate_epochs
    data_iterator = iter(self._dataset)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 499, in __iter__
    return iterator_ops.OwnedIterator(self)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 703, in __init__
    self._create_iterator(dataset)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 742, in _create_iterator
    gen_dataset_ops.make_iterator(ds_variant, self._iterator_resource)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 3409, in make_iterator
    _result = pywrap_tfe.TFE_Py_FastPathExecute(

2023-01-13 19:58:37 WARNING  discord.gateway Shard ID None heartbeat blocked for more than 20 seconds.
Loop thread traceback (most recent call last):
  File "bot.py", line 31, in <module>
    main_client.run(token)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 828, in run
    asyncio.run(runner())
  File "/usr/lib/python3.8/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.8/asyncio/base_events.py", line 603, in run_until_complete
    self.run_forever()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 570, in run_forever
    self._run_once()
  File "/usr/lib/python3.8/asyncio/base_events.py", line 1859, in _run_once
    handle._run()
  File "/usr/lib/python3.8/asyncio/events.py", line 81, in _run
    self._context.run(self._callback, *self._args)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 409, in _run_event
    await coro(*args, **kwargs)
  File "/home/runner/shxhxhcht1hdud-2/core/main_client.py", line 115, in on_message
    output_tensor = predictor.decode_sequence(
  File "/home/runner/shxhxhcht1hdud-2/core/predictor.py", line 62, in decode_sequence
    output_tokens = model.predict([input_seq, target_seq])
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 65, in error_handler
    return fn(*args, **kwargs)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/engine/training.py", line 2346, in predict
    for _, iterator in data_handler.enumerate_epochs():  # Single epoch.
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/keras/engine/data_adapter.py", line 1304, in enumerate_epochs
    data_iterator = iter(self._dataset)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/dataset_ops.py", line 499, in __iter__
    return iterator_ops.OwnedIterator(self)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 703, in __init__
    self._create_iterator(dataset)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/data/ops/iterator_ops.py", line 742, in _create_iterator
    gen_dataset_ops.make_iterator(ds_variant, self._iterator_resource)
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/tensorflow/python/ops/gen_dataset_ops.py", line 3409, in make_iterator
    _result = pywrap_tfe.TFE_Py_FastPathExecute(

1/1 [==============================] - 10s 10s/step
2023-01-13 19:58:42 ERROR    discord.client Attempting a reconnect in 0.03s
Traceback (most recent call last):
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 627, in connect
    await self.ws.poll_event()
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/gateway.py", line 646, in poll_event
    raise ConnectionClosed(self.socket, shard_id=self.shard_id, code=code) from None
discord.errors.ConnectionClosed: Shard ID None WebSocket closed with 1000
2023-01-13 19:58:42 ERROR    discord.client Attempting a reconnect in 0.03s
Traceback (most recent call last):
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/client.py", line 627, in connect
    await self.ws.poll_event()
  File "/opt/virtualenvs/python3/lib/python3.8/site-packages/discord/gateway.py", line 646, in poll_event
    raise ConnectionClosed(self.socket, shard_id=self.shard_id, code=code) from None
discord.errors.ConnectionClosed: Shard ID None WebSocket closed with 1000
sergree commented 1 year ago

Процесс генерации сообщения довольно дорогой по ресурсам, не факт что у воркера реплита достаточно CPU и RAM для этого

AnUser1 commented 1 year ago

Процесс генерации сообщения довольно дорогой по ресурсам, не факт что у воркера реплита достаточно CPU и RAM для этого

в старой версии DolboNet, когда я его запускал, таких проблем не было. Он работал исправно и не жрал много всего На других же хостингах (150% cpu, 2gb ram, 2gb memory) пишет не достаточно памяти и вылетал (не докачав даже), а на реплите я спокойно все это скачиваю без вылетов

sergree commented 1 year ago

Код не менялся, только версии зависимостей, вопрос к разрабам Tensorflow