search

C0D3D3V / Moodle-Downloader

A Moodle Crawler that downloads course content from Moodle (eg. lecture pdfs)
GNU General Public License v3.0
28 stars 4 forks source link

sre_constants.error: bad character range #20

Closed NullP0interEx closed 5 years ago

NullP0interEx commented 5 years ago

Hi,

I always get the following error:

13:17:22 Finished course: 'Informatik-für-Telematiker-(WS19-20)(Janine-Breßler)' 13:17:22 Check course: 'Ortung-und-Navigation-für-Telematik-Dienste-ONTD-WS19-20' ID: 15066 Traceback (most recent call last): File "/Users/romankobosil/Documents/Telematik/Moodle-Downloader/src/moodleCrawler.py", line 1706, in crawlMoodlePage(course[1], course[0], current_dir, mainpageURL + "my/") File "/Users/romankobosil/Documents/Telematik/Moodle-Downloader/src/moodleCrawler.py", line 1363, in crawlMoodlePage PageLinkContent, responsePageLink, pagelink) File "/Users/romankobosil/Documents/Telematik/Moodle-Downloader/src/moodleCrawler.py", line 440, in saveFile file_name, fileName, filetype, pathtoSearch) File "/Users/romankobosil/Documents/Telematik/Moodle-Downloader/src/moodleCrawler.py", line 704, in searchfordumpsSpecific fileName + '*' + filetype) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/fnmatch.py", line 56, in filter _cache[pat] = re_pat = re.compile(res) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/re.py", line 194, in compile return _compile(pattern, flags) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/re.py", line 251, in _compile raise error, v # invalid expression sre_constants.error: bad character range

C0D3D3V commented 5 years ago

Interesting. Are you running the script in Cygwin or something similar under windows?

NullP0interEx commented 5 years ago

No. MacOS 10.15

NullP0interEx commented 5 years ago

I suspect that has something to do with the name of our courses. Anyway, he can not compile the regular expression or even look for it

C0D3D3V commented 5 years ago

normaly the script suports fully utf8 ... under windows the python suport for utf8 struggle under bad conditions, but for macOS it should work.

Does it only struggle at this one course "Ortung und Navigation für Telematik"? So if you put the course number 15066 in the config variable " dontcrawlcourses" it works?

NullP0interEx commented 5 years ago

No, I have the problem with several courses. Maybe it's the square brackets?

NullP0interEx commented 5 years ago

When restarting the script, by the way, it works without error. But it annoys because he always gets stuck.

NullP0interEx commented 5 years ago
CleanShot 2019-10-14 at 14 58 40@2x CleanShot 2019-10-14 at 14 59 13@2x
NullP0interEx commented 5 years ago
CleanShot 2019-10-14 at 14 59 52@2x
C0D3D3V commented 5 years ago

ah yes the squere brackets is a good idea, that could it be. I will check it this evenig and fix it...

C0D3D3V commented 5 years ago

you can try if the last commit https://github.com/C0D3D3V/Moodle-Downloader/commit/4c22b2e652e5b6b1abf50dafd9b1893428794fd5 fixed the problem

C0D3D3V commented 5 years ago

mh I will refactor the code later, so that it dont use fnmatch, that should fix it. I think the implementation of this function differs in macos...

C0D3D3V commented 5 years ago

I made a new update https://github.com/C0D3D3V/Moodle-Downloader/commit/f9162ddb551fdd9368811af3799148d50a973bd6. Maybe it'll work now. If it doesn't work, I will remove the whole searchfordumpsSpecific function, which is unnecessary anyway. You could then simply remove line 438 " fileWasDeleted = searchfordumpsSpecific( file_name, fileName, filetype, pathtoSearch)"

Just delete completely, shouldn't make a big difference, because since some commits this is checked at two places. In the next weeks I will work on a new MoodleDownloader, which will be much more efficient, but until then this should help.

NullP0interEx commented 5 years ago

Great, it works! Thank you very much