search

uvacw / inca

24 stars 6 forks source link

Bestpractices #362

Closed damian0604 closed 6 years ago

damian0604 commented 6 years ago

HERE IT IS! The major refactoring of the whole code. It will probably take some time to rest....

FeLoe commented 6 years ago

When trying to set the 'maxpages' argument for a non-rss scraper I get the error: 'get() got multiple values for argument 'maxpages' Seems to be an issue in scraper_class..

damian0604 commented 6 years ago

I think (after trying around a bit, it's maybe also too late and I might be too tired) that the way to go is to fix change the scraper itself (e.g., the scrapers/cda_scraper.py as follows: 

-    def get(self, maxpages=1):
+    def get(self, save, maxpages, *args, **kwargs):
FeLoe commented 6 years ago

Yes, that solves the problem, I now changed the scrapers. We do not have default values anymore but if the necessary arguments are not supplied (e.g. maxpages) the scraper does not run but gives an error asking to supply the arguments.

damian0604 commented 6 years ago

Thanks a lot! For future reference, here link to an example: https://stackoverflow.com/questions/21764770/typeerror-got-multiple-values-for-argument#21765286