ostris / ai-toolkit

Various AI scripts. Mostly Stable Diffusion stuff.
MIT License
3.52k stars 376 forks source link

run_modal.py (application protocol failed to receive SSL data) #162

Open Dragoy opened 2 months ago

Dragoy commented 2 months ago

This is for bugs only

Did you already ask in the discord?

Yes

You verified that this is a bug and not a feature request or question by asking in the discord?

Yes

Describe the bug

The modal version doesn't work: modal run run_modal.py --config-file-list-str=/root/ai-toolkit/config/modal_train_lora_flux_24gb.yaml

I run it with this command and after a while I get an error:

application protocol failed to receive SSL data
protocol: <asyncio.sslproto.SSLProtocol object at 0x0000024DF7041FF0>
transport: <_SelectorSocketTransport fd=185608 read=polling write=<idle, bufsize=0>>
Traceback (most recent call last):
  File "C:\Users\Dragoy\AppData\Local\Programs\Python\Python310\lib\asyncio\sslproto.py", line 551, in data_received
    self._app_protocol.data_received(chunk)
  File "D:\Program Files\Stable Diffusion\FLUX_TRAIN\modal_com\ai-toolkit\venv\lib\site-packages\grpclib\protocol.py",
line 714, in data_received
    events = self.connection.feed(data)
  File "D:\Program Files\Stable Diffusion\FLUX_TRAIN\modal_com\ai-toolkit\venv\lib\site-packages\grpclib\protocol.py",
line 189, in feed
    return self._connection.receive_data(data)  # type: ignore
  File "D:\Program Files\Stable Diffusion\FLUX_TRAIN\modal_com\ai-toolkit\venv\lib\site-packages\h2\connection.py", line
1463, in receive_data
    events.extend(self._receive_frame(frame))
  File "D:\Program Files\Stable Diffusion\FLUX_TRAIN\modal_com\ai-toolkit\venv\lib\site-packages\h2\connection.py", line
1487, in _receive_frame
    frames, events = self._frame_dispatch_table[frame.__class__](frame)
AttributeError: 'H2Connection' object has no attribute '_frame_dispatch_table'

158 - the patch also doesn't help....

Dragoy commented 2 months ago

@jaretburkett

tazztone commented 1 month ago

aight this fixed it for me: edit: the run_modal.py file to exclude the venv and pycache folders like this:

code_mount = modal.Mount.from_local_dir(
    "D:\\Program Files\Stable Diffusion\\FLUX_TRAIN\\modal_com\\ai-toolkit", 
    remote_path="/root/ai-toolkit",
    condition=lambda path: not any(segment in ['venv', '__pycache__'] for segment in path.split(os.sep))
)

source for solution: https://modallabscommunity.slack.com/archives/C06GPR10K9V/p1727882458380109?thread_ts=1725407469.736319&cid=C06GPR10K9V