C0D3D3V / Moodle-DL

Moodle-DL downloads course content fast from Moodle (eg. lecture pdfs)
GNU General Public License v3.0
433 stars 63 forks source link

Moodle rejected system request - invalid_response_exception #195

Closed davepivonka closed 7 months ago

davepivonka commented 1 year ago

Explanation

Hello, I have been trying to get the downloader to work for the past hour, but despite trying to change different settings (e.g. disabling or enabling the cookie files or ignoring ytdl errors) and tried both normal and SSO logins, I have not been able to get it to work and the error is always the same. Is there anything I can do about this? :)

Output without verbose

2023-06-29 18:45:04  DEBUG  {main}  Exception-Handling completed. Exiting...2023-06-29 19:03:39  INFO  {moodle_service}  Downloading account information
2023-06-29 19:03:40  INFO  {common}  Loaded all workshops
2023-06-29 19:03:40  INFO  {common}  Loaded all assignments
2023-06-29 19:03:41  INFO  {common}  Loaded all databases
2023-06-29 19:03:41  INFO  {common}  Loaded all quizzes
2023-06-29 19:03:41  INFO  {common}  Loaded all folders
2023-06-29 19:03:41  ERROR  {main}  Exception: The Moodle System rejected the Request. Details: invalid_response_exception (Error code: invalidresponse, Message: Invalid response value detected)

Verbose output:

2023-06-29 18:47:18  DEBUG  {main}  moodle-dl version: 2.3.2.0
2023-06-29 18:47:18  DEBUG  {main}  python version: 3.11.2
2023-06-29 18:47:18  DEBUG  {main}  Is ffmpeg available: True
2023-06-29 18:47:18  DEBUG  {main}  Checking for changes for the configured Moodle-Account....
2023-06-29 18:47:18  DEBUG  {database}  Database Version: 5
2023-06-29 18:47:18  DEBUG  {selector_events}  Using selector: KqueueSelector
2023-06-29 18:47:18  DEBUG  {moodle_service}  Fetching current Moodle state...
2023-06-29 18:47:18  INFO  {moodle_service}  Downloading account information
2023-06-29 18:47:19  DEBUG  {connectionpool}  Starting new HTTPS connection (1): moodle.essex.ac.uk:443
2023-06-29 18:47:19  DEBUG  {connectionpool}  https://moodle.essex.ac.uk:443 "POST /webservice/rest/server.php?moodlewsrestformat=json&wsfunction=core_webservice_get_site_info HTTP/1.1" 200 None
2023-06-29 18:47:19  DEBUG  {moodle_service}  Detected moodle version: 2022041905
2023-06-29 18:47:19  DEBUG  {connectionpool}  Starting new HTTPS connection (1): moodle.essex.ac.uk:443
2023-06-29 18:47:20  DEBUG  {connectionpool}  https://moodle.essex.ac.uk:443 "POST /webservice/rest/server.php?moodlewsrestformat=json&wsfunction=core_enrol_get_users_courses HTTP/1.1" 200 None
2023-06-29 18:47:20  INFO  {common}  Loaded all workshops
2023-06-29 18:47:20  INFO  {common}  Loaded all databases
2023-06-29 18:47:20  INFO  {common}  Loaded all assignments
2023-06-29 18:47:20  INFO  {common}  Loaded all folders
2023-06-29 18:47:20  INFO  {common}  Loaded all quizzes
2023-06-29 18:47:20  INFO  {common}  Loaded all forums
2023-06-29 18:47:20  INFO  {common}  Loaded all pages
2023-06-29 18:47:21  DEBUG  {request_helper}  Details about the failed request:
URL: https://moodle.essex.ac.uk/webservice/rest/server.php?moodlewsrestformat=json&wsfunction=mod_lesson_get_lessons_by_courses
Body: {'moodlewssettingfilter': 'true', 'moodlewssettingfileurl': 'true', 'courseids': {'0': 5844, '1': 7106, '2': 16435, '3': 14431, '4': 3886, '5': 6373, '6': 16888, '7': 5838, '8': 6260, '9': 5034, '10': 2454, '11': 3201, '12': 4554, '13': 2334, '14': 3908, '15': 14695, '16': 1847, '17': 13058, '18': 14491, '19': 5368}, 'wsfunction': 'mod_lesson_get_lessons_by_courses', 'wstoken': 'censored'}
2023-06-29 18:47:21  DEBUG  {mail_service}  Mail-Notifications not configured, skipping.
2023-06-29 18:47:21  DEBUG  {telegram_service}  Telegram-Notifications not configured, skipping.
2023-06-29 18:47:21  DEBUG  {xmpp_service}  XMPP-Notifications not configured, skipping.
2023-06-29 18:47:21  ERROR  {main}  Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/main.py", line 536, in main
    choose_task(config, opts)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/main.py", line 68, in choose_task
    run_main(config, opts)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/main.py", line 136, in run_main
    raise base_err
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/main.py", line 95, in run_main
    changed_courses = asyncio.run(moodle.fetch_state(database))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/moodle_service.py", line 116, in fetch_state
    fetched_mods_files = await fetch_mods_files(mods, courses)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/mods/__init__.py", line 40, in fetch_mods_files
    mods_results = await asyncio.gather(*[mod.fetch_mod_entries(courses_to_load) for mod in mods_to_fetch])
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/mods/common.py", line 57, in fetch_mod_entries
    result = await self.real_fetch_mod_entries(courses)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/mods/lesson.py", line 24, in real_fetch_mod_entries
    await self.client.async_post(
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/request_helper.py", line 141, in async_post
    self.check_json_for_moodle_error(resp_json, url, data)
  File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/moodle_dl/moodle/request_helper.py", line 325, in check_json_for_moodle_error
    raise RequestRejectedError(
moodle_dl.moodle.request_helper.RequestRejectedError: The Moodle System rejected the Request. Details: invalid_response_exception (Error code: invalidresponse, Message: Invalid response value detected)

2023-06-29 18:47:21  DEBUG  {main}  Exception-Handling completed. Exiting...
Aunib commented 1 year ago

I have the same (or a very similar) problem. It ran fine on version 2.1.2.6 last year. My school updated moodle for this school year and it didn't work. I updated the configuration (with moodle-dl -c) and tried again. I got the same error. Then I updated moodle-dl to version 2.3.2.0 and tried again. I got the same error (differently formatted, but the same message) this time. I updated the config file again and the error stayed.

Here the (imho) relevant part of the error message (with the -v flag)

2023-09-01 16:02:48  DEBUG  {request_helper}  Details about the failed request:
URL: https://censored.example.com/webservice/rest/server.php?moodlewsrestformat=json&wsfunction=core_course_get_contents
Body: {'moodlewssettingfilter': 'true', 'moodlewssettingfileurl': 'true', 'courseid': 2951, 'wsfunction': 'core_course_get_contents', 'wstoken': 'censored'}
The following error occurred during execution:
The Moodle System rejected the Request. Details: moodle_exception (Error code: cannotgetcoursecontents, Message: Kursinhalte können nicht geladen werden.)
2023-09-01 16:02:48  DEBUG  {mail_service}  Mail-Notifications not configured, skipping.
2023-09-01 16:02:48  INFO  {telegram_service}  Sending Notification via Telegram...
2023-09-01 16:02:48  DEBUG  {connectionpool}  Starting new HTTPS connection (1): api.telegram.org:443
2023-09-01 16:02:48  DEBUG  {connectionpool}  https://api.telegram.org:443 "POST /CENSORED/sendMessage HTTP/1.1" 200 467
2023-09-01 16:02:48  DEBUG  {xmpp_service}  XMPP-Notifications not configured, skipping.
2023-09-01 16:02:48  ERROR  {main}  Traceback (most recent call last):
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/main.py", line 536, in main
    choose_task(config, opts)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/main.py", line 68, in choose_task
    run_main(config, opts)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/main.py", line 136, in run_main
    raise base_err
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/main.py", line 95, in run_main
    changed_courses = asyncio.run(moodle.fetch_state(database))
  File "/usr/lib/python3.7/asyncio/runners.py", line 43, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.7/asyncio/base_events.py", line 584, in run_until_complete
    return future.result()
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/moodle/moodle_service.py", line 117, in fetch_state
    course_cores = await core_handler.async_load_course_cores(courses)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/moodle/core_handler.py", line 157, in async_load_course_cores
    cores = await asyncio.gather(*async_features)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/utils.py", line 64, in run_with_final_message
    result = await load_function(entry)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/moodle/core_handler.py", line 166, in async_load_course_core
    return await self.client.async_post('core_course_get_contents', data)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/moodle/request_helper.py", line 141, in async_post
    self.check_json_for_moodle_error(resp_json, url, data)
  File "/home/moodle-dl/.local/lib/python3.7/site-packages/moodle_dl/moodle/request_helper.py", line 328, in check_json_for_moodle_error
    + f" Message: {resp_json.get('message', '')})"
moodle_dl.moodle.request_helper.RequestRejectedError: The Moodle System rejected the Request. Details: moodle_exception (Error code: cannotgetcoursecontents, Message: Kursinhalte können nicht geladen werden.)

2023-09-01 16:02:48  DEBUG  {main}  Exception-Handling completed. Exiting...

The current moodle version according to /lib/upgrade.txt is 4.1.5, I don't know the previous version. If you need more information, I would be happy to provide it Thank you for the amazing project btw., it saved me a lot of hassle for the last three years.

C0D3D3V commented 1 year ago

@Aunib That is a completly different issue. Please move it into an own issue.

C0D3D3V commented 7 months ago

Looks like a misconfiguration of your moodle instance, see https://moodle.org/mod/forum/discuss.php?d=343122 or https://moodle.org/mod/forum/discuss.php?d=355334

My guess is that for one of your lessons there is an error on the server. Currentlly moodle-dl has no option to deactivate loading of top level entries like lesson names. We should implement it.