FurryCoders / FALocalRepo

Pure Python program to download submissions, journals, and user folders from the FurAffinity forum in an easily handled database.
European Union Public License 1.2
21 stars 2 forks source link

[Bug]: Downloader doesn't work #7

Closed NSFWOrangeCat closed 1 year ago

NSFWOrangeCat commented 1 year ago

Version

falocalrepo 4.4.1 falocalrepo-database 5.4.3 falocalrepo-server 3.3.3 faapi 3.9.6

What happened?

On November 26th, FA introduced new site layout. FALocalRepo doesn't support it and throws an error when trying to download or update users.

Tested on Windows 10, Python version 3.11.0

How to reproduce the bug?

falocalrepo download users -u trioza -f gallery -f scraps

Relevant log output

Traceback (most recent call last):
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\__main__.py", line 59, in main
    exit(app.main(standalone_mode=False) or 0)
  File "D:\Program Files\Python\lib\site-packages\click\core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "D:\Program Files\Python\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:\Program Files\Python\lib\site-packages\click\core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "D:\Program Files\Python\lib\site-packages\click\core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "D:\Program Files\Python\lib\site-packages\click\core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "D:\Program Files\Python\lib\site-packages\click\decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\console\download.py", line 183, in download_users
    downloader.download_users(list(users), list(folders))
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\downloader.py", line 715, in download_users
    self._download_users([(u, folders) for u in users])
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\downloader.py", line 687, in _download_users
    err = self.download_user_submissions(user, Folder[folder.lower()], stop, stop == 1)
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\downloader.py", line 555, in download_user_submissions
    err = self.download_user_folder(
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\downloader.py", line 431, in download_user_folder
    result, err = download_catch(downloader_entries, user, page)
  File "D:\Program Files\Python\lib\site-packages\falocalrepo\downloader.py", line 108, in download_catch
    return func(*args, **kwargs), 0
  File "D:\Program Files\Python\lib\site-packages\faapi\base.py", line 223, in gallery
    info_parsed: dict[str, Any] = parse_user_submissions(page_parsed)
  File "D:\Program Files\Python\lib\site-packages\faapi\parse.py", line 791, in parse_user_submissions
    user_info: dict[str, str] = parse_user_folder(submissions_page)
  File "D:\Program Files\Python\lib\site-packages\faapi\parse.py", line 778, in parse_user_folder
    assert tag_username is not None, _raise_exception(ParsingError("Missing username tag"))
  File "D:\Program Files\Python\lib\site-packages\faapi\exceptions.py", line 56, in _raise_exception
    raise err
faapi.exceptions.ParsingError: Missing username tag
MatteoCampinoti94 commented 1 year ago

It was indeed caused by the new use page layout (it may have also affected comments but I need to test that too, they may have just changed the CSS style for those).

The error is actually in FAAPI, all the parsers are there. I am already working on a patch and should release by the end of the day with a new version of FALocalRepo :)

MatteoCampinoti94 commented 1 year ago

Updated the FAAPI dependency, gonna run a couple of tests then publish a patch :)

MatteoCampinoti94 commented 1 year ago

Published version 4.4.2. All download commands work again :)