openedx / edx-platform

The Open edX LMS & Studio, powering education sites around the world!
https://openedx.org
GNU Affero General Public License v3.0
7.11k stars 3.8k forks source link

Too many console warnings #32888

Open jmbowman opened 11 months ago

jmbowman commented 11 months ago

Many warnings routinely get logged when doing console operations in edx-platform. This is especially visible when provisioning a devstack installation (although the problem isn't limited to devstack). These warnings often indicate latent problems with the code that will cause headaches when attempting future upgrades, they distract developers when looking for the root cause of a problem, and some cause a disproportionate amount of log clutter because they reappear every time a management command is called. We should clean up as many of these as practical to improve the developer experience when working with the repository.

A/C:

DanielVZ96 commented 9 months ago

@jmbowman I made some rough counts for all warnings from the provision run. There are plenty that are not repeated, almost all of them being deprecation warnings, with a big majority being deprecated js packages.

Given their frequencies, I'd summarize them in the following issues:

  1. Fix/Ignore: [Bourbon] [Deprecation] rem is deprecated and will be removed in 5.0.0.

  2. [From 2 external dependencies, one can be updated, the other needs to be ignored] Fix: Django now detects this configuration automatically. You can remove default_app_config.

  3. Fix: Flag '...' accessed without a request, which is likely in the context of a celery task.

  4. Replace Bourbon mixins with Autoprefixer

  5. Fix XBlock warnings

    • [xblock-google-drive] DeprecatedPackageWarning: Please use import xblock.utils.publish_event instead of xblockutils.publish_event because the 'xblock-utils' package has been deprecated and migrated to within 'xblock' package.
    • edX Studio doesn't support third-party handler urls for XBlock <class 'xblock.internal.VideoBlockWithMixins'>
    • [xblock, edx-platform] DeprecationWarning: XBlocks should not instantiate their own field_data store during parse_xml()
  6. [no idea what repo it's from] Fix: ModelBase.objects model manager will include soft-deleted objects in an upcoming release; please use ModelBase.available_objects to continue excluding soft-deleted objects.

  7. Fix: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0

  8. Fix Python deprecation warnings: collections.abc [this repo, need to update libsass to 0.15.0] , imp module [this repo, need replace boto with boto3 in a few lines], HTTPResponse.getheaders(), add_custom_parameter

  9. Fix: Flags not at the start of the expression

  10. Update deprecated packages in package.json

Note: I'll check if an issue is related to this repo before I create it, so the task list length may end up being less than the list above

Counts

count warning extract
165 rem is deprecated and will be removed
129 Django now detects this configuration automatically. You can remove default_app_config.
120 DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace'
35 accessed without a request, which is likely in the context of a celery task.
31 is deprecated and will be removed in 5.0.0. We suggest using an automated prefixing tool, like Autoprefixer.
31 DeprecationWarning: 'etree' is deprecated. Use 'xml.etree.ElementTree' instead.
30 XBlock <class 'xblock.internal.DiscussionXBlockWithMixins'> does not contain field url_name
24 ModelBase.objects model manager will include soft-deleted objects in an upcoming release; please use ModelBase.available_objects to continue excluding soft-deleted objects.
22 transition is deprecated and will be removed in 5.0.0. We suggest using an automated prefixing tool, like Autoprefixer.
18 because the 'xblock-utils' package has been deprecated and migrated to within 'xblock' package.
17 Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0
12 DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working
6 please upgrade to graceful-fs 4 for compatibility with current and future versions of Node.js
6 Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
6 deprecated browserslist@1.7.7: Browserslist 2 could fail on reading Browserslist >3.0 config used in other tools.
5 the imp module is deprecated in favour of importlib;
5 edX Studio doesn't support third-party handler urls for XBlock <class 'xblock.internal.VideoBlockWithMixins'>
4 Using / for division is deprecated and will be removed in Dart Sass 2.0.0.
4 deprecated uuid@3.4.0
3 DeprecationWarning: XBlocks should not instantiate their own field_data store during parse_xml()
2 skipping integrity check for git dependency
2 Mac users should run this from both the lms and studio shells
2 Falling back to fake video id due to Course Blocks API failure...
2 deprecated core-js@2.6.12: core-js@<3.4 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
2 Cannot inherit metadata to * block with KVS
1 Use of runtime.render_template is deprecated. Use MakoService.render_template or a JavaScript-based template instead.
1 runtime.user_id is deprecated. Use block.scope_ids.user_id or the user service instead.
1 ANTIALIAS is deprecated and will be removed in Pillow 10

Other warnings

Python:

  1. Flags not at the start of the expression.
  2. HTTPResponse.getheaders() is deprecated in urllib3.
  3. add_custom_parameter API in New Relic is deprecated.
  4. Transcript "subs_3_yD_cEKoCk.srt.sjson" for video "5e5d9190-9985-43ac-b859-0063968f88f2" is not found.

JavaScript:

DanielVZ96 commented 8 months ago

I can't create the tasklist myself, but here I'll keep track of the Issues I'm creating.

Issue(s) Warning
https://github.com/openedx/edx-platform/issues/33571 Fix/Ignore: [Bourbon] [Deprecation] rem is deprecated and will be removed in 5.0.0.
https://github.com/openedx/edx-platform/issues/33572 Fix: Django now detects this configuration automatically. You can remove default_app_config.
https://github.com/openedx/XBlock/issues/641 DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace'
https://github.com/openedx/edx-toggles/issues/320 Fix: Flag '...' accessed without a request, which is likely in the context of a celery task.
https://github.com/openedx/edx-platform/issues/33573 Fix/Ignore: Replace Bourbon mixins with Autoprefixer warnings
https://github.com/open-craft/xblock-poll/issues/114, - DeprecatedPackageWarning: Please use import xblock.utils.publish_event instead of xblockutils.publish_event because the 'xblock-utils' package has been deprecated and migrated to within 'xblock' package.
https://github.com/openedx/edx-platform/issues/33581 - edX Studio doesn't support third-party handler urls for XBlock <class 'xblock.internal.VideoBlockWithMixins'>
https://github.com/openedx/edx-platform/issues/33582 - DeprecationWarning: XBlocks should not instantiate their own field_data store during parse_xml()
https://github.com/openedx/edx-enterprise/issues/1927 Fix: ModelBase.objects model manager will include soft-deleted objects in an upcoming release; please use ModelBase.available_objects to continue excluding soft-deleted objects.
https://github.com/openedx/edx-platform/issues/33583 Fix: Using / for division outside of calc() is deprecated and will be removed in Dart Sass 2.0.0
https://github.com/openedx/edx-platform/issues/33584 Fix Python deprecation warnings: collections.abc , HTTPResponse.getheaders(), add_custom_parameter
https://github.com/openedx/edx-platform/issues/33585 Fix: Flags not at the start of the expression
https://github.com/openedx/edx-platform/issues/33586 Cannot inherit metadata to * block with KVS
https://github.com/openedx/edx-platform/issues/33587 runtime.user_id is deprecated. Use block.scope_ids.user_id or the user service instead.
https://github.com/openedx/edx-platform/issues/33588 XBlock <class 'xblock.internal.DiscussionXBlockWithMixins'> does not contain field url_name
https://github.com/openedx/edx-platform/issues/31175 the imp module is deprecated in favour of importlib;
https://github.com/openedx/edx-platform/issues/33589 Use of runtime.render_template is deprecated. Use MakoService.render_template or a JavaScript-based template instead.
https://github.com/openedx/edx-platform/issues/33621 DeprecationWarning: defusedxml.lxml is no longer supported and will be removed in a future release.
https://github.com/openedx/edx-platform/issues/33622 RemovedInDjango40Warning: The DEFAULT_HASHING_ALGORITHM transitional setting is deprecated. Support for it and tokens, cookies, sessions, and signatures that use SHA-1 hashing algorithm will be removed in Django 4.0.
todo Update deprecated packages in package.json
Agrendalath commented 8 months ago

@DanielVZ96,

Fix: ModelBase.objects model manager will include soft-deleted objects in an upcoming release; please use ModelBase.available_objects to continue excluding soft-deleted objects.

I suppose it comes from edx-enterprise. It can be triggered by running tox -e py38-django32-celery53 in this repo. Link to the CI run.

What about these?

the imp module is deprecated in favour of importlib;

skipping integrity check for git dependency

Cannot inherit metadata to * block with KVS

runtime.user_id is deprecated. Use block.scope_ids.user_id or the user service instead.

Use of runtime.render_template is deprecated. Use MakoService.render_template or a JavaScript-based template instead.

I believe the last one comes from here. Removing it properly will be a significant task.

DanielVZ96 commented 8 months ago

the imp module is deprecated in favour of importlib;

(i think i accidentally deleted it in the table) it comes from using boto, probably here, and would need to update that code so it uses boto3

I believe the last one comes from here. Removing it properly will be a significant task.

Then can i just create the issue for that one?

skipping integrity check for git dependency

This should be safe to ignore

Agrendalath commented 8 months ago

@DanielVZ96,

(i think i accidentally deleted it in the table) it comes from using boto, probably here, and would need to update that code so it uses boto3

We can link https://github.com/openedx/edx-platform/issues/31175. Looks like it should be handled by 2U.

Then can i just create the issue for that one?

Yep.

Agrendalath commented 8 months ago

@DanielVZ96, I'm seeing two more exceptions we didn't mention here:

2023-10-30 11:13:23,027 WARNING  /edx/app/edxapp/edx-platform/openedx/core/lib/safe_lxml/xmlparser.py:11: DeprecationWarning: defusedxml.lxml is no longer supported and will be removed in a future release.
  from defusedxml.lxml import DTDForbidden, EntitiesForbidden, NotSupportedError
2023-10-30 11:13:23,410 WARNING  /edx/app/edxapp/venvs/edxapp/lib/python3.8/site-packages/django/conf/__init__.py:198: RemovedInDjango40Warning: The DEFAULT_HASHING_ALGORITHM transitional setting is deprecated. Support for it and tokens, cookies, sessions, and signatures that use SHA-1 hashing algorithm will be removed in Django 4.0.
  warnings.warn(DEFAULT_HASHING_ALGORITHM_DEPRECATED_MSG, RemovedInDjango40Warning)

These are coming directly from the edx-platform, so it would be good to create issues for them.

jmbowman commented 7 months ago

I finally caught up on my GitHub notification emails...it's great to see all the progress on this! The DEFAULT_HASHING_ALGORITHM one should go away in a week or two once we wrap up the Django 4.2 upgrade. Anything else I can do to help move things along here?

devkhan commented 1 week ago

What's the current status of this? I can help if needed

Agrendalath commented 1 week ago

@devkhan, please feel free to pick any open task from the list in the first comment.