search

mementoseeds / Memento

Memento is a cross-platform, offline, spaced-repetition learning application. It is made with the intent to be clean, simple to use and easily configurable to one's needs.
GNU General Public License v3.0
7 stars 0 forks source link

scraper fails with json error #1

Open cstrife1 opened 2 years ago

cstrife1 commented 2 years ago

Hi, not sure if this is still being worked on, but I just tried out the scraper to download a course, but it errored out early on.

Course being downloaded:

https://app.memrise.com/course/1614/1000-most-common-korean-words/

Here is the command line output (some stuff removed due to privacy):

Reading level 931 - 945 Reading level 946 - 960 Reading level 961 - 975 Reading level 976 - 987 mkdir \Memento_v0.6_Windows_10_64-bit\1000 Most Common Korean Words-1657763202.0941 Downloading icon Creating info.json Finding unique items across all levels Finding item information from Memrise API Scraping item 1 of 984 Caught exception Traceback (most recent call last): File "\Memento_v0.6_Windows_10_64-bit\scrape_memrise.py", line 101, in course.autoScrape(destination, minLevel, maxLevel, skipAudio, skipMnemonics) File "\Memento_v0.6_Windows_10_64-bit\MemriseCourse.py", line 411, in autoScrape self.buildSeedbox(skipAudio, skipMnemonics) File "\Memento_v0.6_Windows_10_64-bit\MemriseCourse.py", line 339, in buildSeedbox mnemonicsJson = requests.get(MemriseCourse.memriseApi + "mem/get_many_for_thing/?thing_id=" + key + "&learnable_id=" + self.itemLearnables[key]).json() File "C:\Users\Person\AppData\Local\Programs\Python\Python39\lib\site-packages\requests\models.py", line 910, in json return complexjson.loads(self.text, **kwargs) File "C:\Users\Person\AppData\Local\Programs\Python\Python39\lib\json\__init__.py", line 346, in loads return _default_decoder.decode(s) File "C:\Users\Person\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 337, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "C:\Users\Person\AppData\Local\Programs\Python\Python39\lib\json\decoder.py", line 355, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) Continuing to next course
mementoseeds commented 2 years ago

Hi, sorry for the late response, you're right I'm not as active as I used to be.

This error appears to be related to the API I use to scrape mems for every item. It doesn't work anymore, it just returns a 404 error. Maybe Memrise has changed it or hidden it or idk, because it doesn't work in any course anymore, not just your Korean one.

For now you can pass the -m option to skip scraping mems like this python3 scrape_memrise.py -m URL