[Bug]: arrow::fs::FinalizeS3 was not called even though S3 was initialized.

[Amazon90]

https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/11827

Extensions tab Click "Apply and restart webUI" C:\arrow\cpp\src\arrow\filesystem\s3fs.cc:2598: arrow::fs::FinalizeS3 was not called even though S3 was initialized. This could lead to a segmentation fault at exit

[AlUlkesh]

This seems to be caused by the current version of pyarrow, see here:

https://github.com/apache/arrow/issues/36069

As suggested there, try installing the previous version. From the a1111 directory:

venv\Scripts\activate.bat
pip install --upgrade --force-reinstall pyarrow==11.0.0

[mikheys]

I have the same problem. Reinstalling pyarrow didn't help. dependency errors appeared during reinstallation.

[AlUlkesh]

@mikheys On my tests I also got these messages, but after that it worked anyway.

Pyarrow is a dependency of ImageReward. Are you even using that? Since this function seems to run into dependency problems from time to time, perhaps I should make it optional.

[coder168]

Same error here, it is very annoying when restarting webui. I downgraded pyarrow to 11.0.0 within venv as follows:

\venv>pip install --upgrade pyarrow==11.0.0

The problem seems to be fixed and the error is gone.

And I think it is a good idea to make ImageReward optional.

[AlUlkesh]

Ok, I think I found a fix for the pyarrow issue.

If you already downgraded, you can upgrade again with:

venv\Scripts\activate.bat
pip install --upgrade pyarrow

[coder168]

Yes, I just try the latest commit and upgrade pyarrow up to 12.0.1 again, it works fine now. Thanks for your great support!

[AlUlkesh]

Great.

I made a separate issue to make ImageReward optional: #219

[vladmandic]

pyarrow is really messy. you placed it in try/except block:

try:
    import ImageReward
    import pyarrow._s3fs
    pyarrow._s3fs.finalize_s3()
    image_reward_installed = True
except ImportError as e:
   ...

but the problem is that pyarrow._s3fs.finalize_s3() executes on first call, but either HANGS or CRASHES on reload. meaning that restart capability in webui gets broken and server either hangs on script load or it simply crashes and exits.

there are sooo many issues related to pyarrow, its up to either removing it as dependency from image browser or me removing image browser - i really cannot deal with that anymore.

[AlUlkesh]

Ouch. Ok, I'll remove ImageReward, its dependency issues are too much of a headache.

[AlUlkesh]

It's removed: #220

[thejohnd0e]

I have exactly the same problem

[AlUlkesh]

The "arrow::fs::FinalizeS3 was not called even though S3 was initialized." error? I removed the cause in this extension in July.

Can you post a full log?

[thejohnd0e]

The "arrow::fs::FinalizeS3 was not called even though S3 was initialized." error? I removed the cause in this extension in July.

Can you post a full log?

Sure, here you go. This happens every time the extensions are updated and then reloaded WebUI

then I have to manually stop it and start it again.

[AlUlkesh]

I suspect it's some other extension that has run into the same issue I had back in July. You can search for pyarrow in all files under extensions\*.py to find the most likely candidate.

If you are under Windows you can simply run this from the main a1111 directory:

@echo off & (for /R "extensions" %G in (*.py) do findstr /I /M /C:"pyarrow" "%G") & echo on

[thejohnd0e]

I suspect it's some other extension that has run into the same issue I had back in July. You can search for pyarrow in all files under extensions\*.py to find the most likely candidate.

If you are under Windows you can simply run this from the main a1111 directory:

@echo off & (for /R "extensions" %G in (*.py) do findstr /I /M /C:"pyarrow" "%G") & echo on

I finally found the reason - sd-webui-EasyPhoto