Bot Crashes: Install lxml[html_clean] or lxml_html_clean directly.

[iTatla1]

Describe the bug Bot crashes when there is an apartment on WBM Site. It tries to restart itself then crash again

To Reproduce Just run the bot, one customisation i did was to set interval to 1 mionute

Expected behavior Bot should not crash and apply for the apartment

Screenshots

Environment

• MAC OS 14.4.1

Additional context [29.04.2024 - 15:54] [main.py:71] [INFO] Initializing Script (v1.1.4) (Headless? False) 🚀 [29.04.2024 - 15:54] [main.py:76] [INFO] Checking for internet connection 🔎 [29.04.2024 - 15:54] [main.py:84] [SUCCESS] Online 🟢 [29.04.2024 - 15:54] [io_operations.py:24] [INFO] Loading WBM config ⚙️ [29.04.2024 - 15:54] [main.py:107] [INFO] Connecting to 'https://www.wbm.de/wohnungen-berlin/angebote/' 🔗 [29.04.2024 - 15:54] [webDriverOperations.py:252] [SUCCESS] Cookies have been accepted 🍪 [29.04.2024 - 15:54] [webDriverOperations.py:380] [INFO] Looking for flats 👀 [29.04.2024 - 15:54] [webDriverOperations.py:387] [SUCCESS] Found 2 flat(s) in total 💡 [29.04.2024 - 15:54] [main.py:122] [ERROR] Bot has crashed... Attempting to restart it now! ❤️‍🩹 [29.04.2024 - 15:54] [main.py:125] [ERROR] Crash reason: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly. ^CTraceback (most recent call last): File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 111, in main webDriverOperations.process_flats( File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/helpers/webDriverOperations.py", line 391, in process_flats hpd.save_viewing_offline( File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/httpsWrapper/httpPageDownloader.py", line 30, in save_viewing_offline save_webpage( File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/init.py", line 107, in save_page page = config.create_page() ^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/configs.py", line 241, in create_page from .core import WebPage File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/core.py", line 12, in from .elements import HTMLResource File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/elements.py", line 24, in from .parsers import iterparse File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/pywebcopy/parsers.py", line 14, in from lxml.html.clean import Cleaner File "/Library/Frameworks/Python.framework/Versions/3.12/lib/python3.12/site-packages/lxml/html/clean.py", line 18, in raise ImportError( ImportError: lxml.html.clean module is now a separate project lxml_html_clean. Install lxml[html_clean] or lxml_html_clean directly.

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 134, in main() File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main main() File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main main() File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 129, in main main() File "/Users/UsmanTatla/Desktop/wbmbot_v2-main/wbmbot_v2/main.py", line 127, in main time.sleep(5) KeyboardInterrupt

[iTatla1]

Seems like a missing package on my environment installing package lxml_html_clean separately fixes the issue https://github.com/fedora-python/lxml_html_clean

[Vel-San]

Thanks @iTatla1! Glad you got it working.

I will be adding this as a note (or in the dependency file) for the bot!