datawhores / OF-Scraper

A completely revamped and redesigned fork, reimagined from scratch based on the original onlyfans-scraper
MIT License
559 stars 50 forks source link

Downloads with long filenames fail on Windows using latest versions of scraper# Clear Description of issue #389

Open r32bum opened 1 month ago

r32bum commented 1 month ago

Describe the bug

Longer filenames that need to be truncated will fail with the below listed errors. Setting text length and original flags (so as to not truncate) don't seem to correct

To Reproduce

attempt download of file (video or image), it will fail and give the listed error in the log

Expected behavior

File truncates or doesn't fail

Screenshots/Logs

2024-05-29 10:02:10:\[helpers.inner:11] Media:2758821526 Post:2957537474780 [attempt 5/3] 'WindowsPath' object has no attribute 'encode' 2024-05-29 10:02:10:\[downloadnormal.download:200] Media:2758821526 Post:2957537474780 exception 'WindowsPath' object has no attribute 'encode' 2024-05-29 10:02:10:\[downloadnormal.download:201] Media:2758821526 Post:2957537474780 exception Traceback (most recent call last): File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\downloadnormal.py", line 183, in download return await main_download( ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\main_download.py", line 56, in main_download result = await main_download_downloader( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\main_download.py", line 75, in main_download_downloader async for _ in download_retry(): File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\tenacity\_asyncio.py", line 123, in __anext__ do = await self.iter(retry_state=self._retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\tenacity\_asyncio.py", line 110, in iter result = await action(retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\tenacity\_asyncio.py", line 78, in inner return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\tenacity\__init__.py", line 410, in exc_check raise retry_exc.reraise() ^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\tenacity\__init__.py", line 183, in reraise raise self.last_attempt.result() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result raise self._exception File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\main_download.py", line 108, in main_download_downloader raise E File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\main_download.py", line 86, in main_download_downloader return await main_data_handler( ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\download\main_download.py", line 125, in main_data_handler placeholderObj = await placeholder.Placeholders(ele, content_type).init() ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\classes\placeholder.py", line 191, in init self._filepath = paths.truncate(pathlib.Path(dir, file)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\utils\paths\paths.py", line 66, in truncate return _windows_truncateHelper(path) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\your_username\AppData\Local\Programs\Python\Python311\Lib\site-packages\ofscraper\utils\paths\paths.py", line 107, in _windows_truncateHelper log.debug(f"path: {path} filepath bytesize: {len(path.encode(encode))}") ^^^^^^^^^^^ AttributeError: 'WindowsPath' object has no attribute 'encode'

Config

{
    "main_profile": "main_profile",
    "metadata": "{configpath}/{profile}/.data/{model_username}/{model_id}",
    "discord": "",
    "file_options": {
        "save_location": "D:\\OF",
        "dir_format": "{model_username}/{mediatype}/",
        "file_format": "{model_username}-{value}-{file}.{ext}",
        "textlength": 0,
        "space_replacer": " ",
        "date": "MM-DD-YYYY",
        "text_type_default": "word",
        "truncation_default": true
    },

System Info

datawhores commented 1 month ago

It should be fixed in https://github.com/datawhores/OF-Scraper/commit/6bec818d770411f788ef6f4756b189b67a3f07e5