fake-name / xA-Scraper

69 stars 8 forks source link

Ng user with zero submissions causes errors #45

Closed God-damnit-all closed 5 years ago

God-damnit-all commented 5 years ago

Because Newgrounds's newly offered art service is only slowly being migrated to, some of artists I follow have made an account and have even advertised it, but have yet to post anything yet.

I didn't know it would cause errors with xA-Scraper at the time, but it does. Seeing as there's currently no way to remove artists, only renaming them, I'd have to figure out who to follow on Newgrounds instead, and then, I'd have to keep checking back to see if any of the contributors that I had been following have started posting yet.

I have tried to detect We have yet to see how %s will contribute to the site. on the page but this code has been very difficult for me to figure out.

God-damnit-all commented 5 years ago

Still a problem after the latest commit. Throws an even bigger, scarier error now. Still retrieves the rest of the artists just fine, though.

herp-a-derp commented 5 years ago

You have an artist with zero items I can look at? It's kind of hard to find them, since they're not really discoverable.

Fwiw, to check for We have yet to see how %s will contribute to the site., you'd do it in _getTotalArtCount(). If there's no art found, you throw exceptions.NoArtException(). Realistically, you could throw exceptions.AccountDisabledException(), but that'd be confusing as hell, even if it did work.

God-damnit-all commented 5 years ago

https://github.com/herp-a-derp/xA-Scraper/commit/e329d1f690dc15d0d0fa428d7fcb29607a508b75 didn't fix it. Emailed you.

herp-a-derp commented 5 years ago

I didn't expect it to, it just provided the exception (and handled it appropriately). _getTotalArtCount() still needs to be patched.

herp-a-derp commented 5 years ago

Implemented.

God-damnit-all commented 5 years ago

Unfortunately the fix seems to be breaking other artists. Strange thing is they DO have art but the error is saying that they don't:

Main - CRITICAL - Uncaught exception!
Main - CRITICAL - Uncaught exception
Traceback (most recent call last):
  File "U:\xA-Scraper\xascraper\modules\scraper_base.py", line 535, in getArtist
    newArt = self._load_art(artist)
  File "U:\xA-Scraper\xascraper\modules\scraper_base.py", line 472, in _load_art
    totalArt = self._getTotalArtCount(artist)
  File "U:\xA-Scraper\xascraper\modules\ng\ngScrape.py", line 239, in _getTotalArtCount
    raise exceptions.NoArtException("No art for artist yet!")
xascraper.modules.exceptions.NoArtException: No art for artist yet!

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Python37\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Python37\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "U:\xA-Scraper\manage\__main__.py", line 101, in <module>
    go()
  File "U:\xA-Scraper\manage\__main__.py", line 95, in go
    two_arg_go(sys.argv[1], sys.argv[2])
  File "U:\xA-Scraper\manage\__main__.py", line 48, in two_arg_go
    do_fetch([param])
  File "U:\xA-Scraper\manage\scrape_manage.py", line 75, in do_fetch
    do_plugin(plgname)
  File "U:\xA-Scraper\manage\scrape_manage.py", line 51, in do_plugin
    plg.runScraper(namespace)
  File "U:\xA-Scraper\xascraper\modules\scraper_base.py", line 682, in runScraper
    instance.go(ctrlNamespace=managedNamespace)
  File "U:\xA-Scraper\xascraper\modules\scraper_base.py", line 666, in go
    errored  |= future.result()
  File "C:\Python37\lib\concurrent\futures\_base.py", line 428, in result
    return self.__get_result()
  File "C:\Python37\lib\concurrent\futures\_base.py", line 384, in __get_result
    raise self._exception
  File "C:\Python37\lib\concurrent\futures\thread.py", line 57, in run
    result = self.fn(*self.args, **self.kwargs)
  File "U:\xA-Scraper\xascraper\modules\scraper_base.py", line 610, in getArtist
    self.log.Warning("Artist has no art (yet)!")
AttributeError: 'Logger' object has no attribute 'Warning'
herp-a-derp commented 5 years ago

Goddammit, fucking autocomplete.

I'm playing with Deep TabNine, and while it's generally handy, it keeps doing dumb crap like the latter issue (capitalized warning, when it needs to be lower case).

I'm somewhat confused how it's triggering on artists incorrectly. The only way it can trigger is if the literal string We have yet to see how {artist-name} will contribute to the site. is found on the root page of the artist's content.