search

plone / Products.CMFPlone

The core of the Plone content management system
https://plone.org
GNU General Public License v2.0
256 stars 193 forks source link

Lots of deprecation warnings in Plone 5.2.2 #3130

Open vincentfretin opened 4 years ago

vincentfretin commented 4 years ago

Currently in buildout.coredev-5.2, there are lots of deprecation warnings. Can we fix those for next release Plone 5.2.3?

vincentfretin@tigron:~/workspace/buildout.coredev-5.2 (5.2) $ bin/instance fg
/home/vincentfretin/.buildout/eggs/Products.GenericSetup-2.0.2-py2.7.egg/Products/GenericSetup/registry.py:16: ImportWarning: Not importing directory '/home/vincentfretin/.buildout/eggs/Products.GenericSetup-2.0.2-py2.7.egg/Products/GenericSetup/xml': missing __init__.py
  from xml.sax import parseString
2020-06-28 09:47:20,015 WARNING [Init:89][MainThread] Class Products.CMFFormController.ControllerPythonScript.ControllerPythonScript has a security declaration for nonexistent method 'ZPythonScriptHTML_changePrefs'
2020-06-28 09:47:20,018 WARNING [Init:89][MainThread] Class Products.CMFFormController.ControllerValidator.ControllerValidator has a security declaration for nonexistent method 'ZPythonScriptHTML_changePrefs'
/home/vincentfretin/workspace/buildout.coredev-5.2/src/Products.CMFPlacefulWorkflow/Products/CMFPlacefulWorkflow/permissions.py:5: DeprecationWarning: setDefaultRoles is deprecated. Please use addPermission from AccessControl.Permission.
  from Products.CMFCore.permissions import setDefaultRoles
/home/vincentfretin/.buildout/eggs/Products.PortalTransforms-3.1.8-py2.7.egg/Products/PortalTransforms/transforms/text_to_html.py:2: DeprecationWarning: html_quote is deprecated. Please import from DocumentTemplate.html_quote. These shims will go away in DocumentTemplate 4.0.
  from DocumentTemplate.DT_Util import html_quote
2020-06-28 09:47:20,238 WARNING [TemporaryStorage:93][MainThread] DEPRECATED: Usage of the package tempstorage is deprecated, as it is known to randomly lose data.
Especially on Zope 4. For details see https://github.com/zopefoundation/tempstorage/issues/8
and https://github.com/zopefoundation/tempstorage

/home/vincentfretin/.buildout/eggs/tempstorage-5.1-py2.7.egg/tempstorage/TemporaryStorage.py:94: DeprecationWarning: DEPRECATED: Usage of the package tempstorage is deprecated, as it is known to randomly lose data.
Especially on Zope 4. For details see https://github.com/zopefoundation/tempstorage/issues/8
and https://github.com/zopefoundation/tempstorage

  warnings.warn(deprecation_warning, DeprecationWarning)
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.upgrade/plone/app/upgrade/__init__.py:167: DeprecationWarning: LockItem is deprecated. Please import from OFS.LockItem.
  from webdav.LockItem import LockItem
/home/vincentfretin/.buildout/eggs/plone.subrequest-1.9.2-py2.7.egg/plone/subrequest/__init__.py:18: DeprecationWarning: zope.site.hooks has moved to zope.component.hooks. Import of zope.site.hooks will become unsupported in Version 5.0
  from zope.site.hooks import getSite
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: IObjectEvent is deprecated. Import from zope.interface.interfaces
  obj = getattr(mod, oname)
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.textfield/plone/app/textfield/utils.py:10: DeprecationWarning: ComponentLookupError is deprecated. Import from zope.interface.interfaces
  from zope.component.interfaces import ComponentLookupError
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.content/plone/app/content/browser/vocabulary.py:10: DeprecationWarning: IFieldPermissionChecker is deprecated. Import IFieldPermissionChecker from plone.app.z3cform.interfaces instead
  from plone.app.widgets.interfaces import IFieldPermissionChecker
2020-06-28 09:47:20,924 INFO    [chameleon.config:38][MainThread] directory cache: /home/vincentfretin/workspace/buildout.coredev-5.2/var/cache.
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: IComponentRegistry is deprecated. Import from zope.interface.interfaces
  obj = getattr(mod, oname)
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LockableItem is deprecated. Please import from OFS.Lockable.
  obj = getattr(mod, oname)
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.schemaeditor/plone/schemaeditor/utils.py:6: DeprecationWarning: ObjectEvent is deprecated. Import from zope.interface.interfaces
  from zope.component.interfaces import ObjectEvent
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LazyMap is deprecated. Please import from ZTUtils.Lazy.
  obj = getattr(mod, oname)
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: LazyCat is deprecated. Please import from ZTUtils.Lazy.
  obj = getattr(mod, oname)
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.upgrade/plone/app/upgrade/v50/betas.py:8: DeprecationWarning: ILanguageSchema is deprecated. It has been moved to plone.i18n.interfaces, import from there instead.
  from Products.CMFPlone.interfaces import ILanguageSchema
/home/vincentfretin/.buildout/eggs/Products.PluggableAuthService-2.4-py2.7.egg/Products/PluggableAuthService/plugins/exportimport.py:19: ImportWarning: Not importing directory '/home/vincentfretin/.buildout/eggs/Products.PluggableAuthService-2.4-py2.7.egg/Products/PluggableAuthService/plugins/xml': missing __init__.py
  from xml.dom.minidom import parseString
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:225: ImportWarning: Not importing directory '/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.iterate/plone/app/iterate/profiles': missing __init__.py
  __import__(mname)
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.restapi/src/plone/restapi/controlpanels/__init__.py:6: DeprecationWarning: ILanguageSchema is deprecated. It has been moved to plone.i18n.interfaces, import from there instead.
  from Products.CMFPlone.interfaces.controlpanel import ILanguageSchema
/home/vincentfretin/workspace/buildout.coredev-5.2/src/plone.app.theming/src/plone/app/theming/plugins/utils.py:90: DeprecationWarning: This method will be removed in future versions.  Use 'parser.read_file()' instead.
  parser.readfp(fp)
2020-06-28 09:47:23,183 INFO    [Zope:45][MainThread] Ready to handle requests
Starting server in PID 18376.
/home/vincentfretin/.buildout/eggs/waitress-1.4.4-py2.7.egg/waitress/adjustments.py:445: DeprecationWarning: In future versions of Waitress clear_untrusted_proxy_headers will be set to True by default. You may opt-out by setting this value to False, or opt-in explicitly by setting this to True.
  DeprecationWarning,
Serving on http://0.0.0.0:8080
/home/vincentfretin/.buildout/eggs/zope.deprecation-4.4.0-py2.7.egg/zope/deprecation/deprecation.py:88: DeprecationWarning: isDefaultPage is deprecated. Import from Products.CMFPlone.defaultpage instead
  name)
/home/vincentfretin/.buildout/eggs/Zope-4.4.4-py2.7.egg/Products/PageTemplates/engine.py:72: DeprecationWarning: Using len() is deprecated. Use the `length` attribute for the size of the current page, which is what we return now. Use the `sequence_length` attribute for the size of the entire sequence. 
  iterable = list(iterable) if iterable is not None else ()

and when you press ctrl+c : 

 /home/vincentfretin/.buildout/eggs/waitress-1.4.4-py2.7.egg/waitress/wasyncore.py:649: UserWarning: unclosed file <waitress.wasyncore.file_wrapper instance at 0x7fe9597ceaa0>
  warnings.warn("unclosed file %r" % self, compat.ResourceWarning)
tisto commented 4 years ago

That would be really helpful. I already fixed a few and I will look into the RESTAPI ones. Maybe a checklist with the packages and fixes we need? Then we can divide the work.

vincentfretin commented 4 years ago

Products.GenericSetup 2.0.2 :

Products.CMFFormController :

Products.CMFPlacefulWorkflow :

Products.PortalTransforms 3.1.8 :

tempstorage / plone.recipe.zope2instance :

plone.app.upgrade :

plone.subrequest 1.9.2 :

plone.app.textfield :

plone.app.content :

plone.schemaeditor :

Products.PluggableAuthService 2.4 :

plone.restapi :

plone.app.theming :

waitress 1.4.4 :

Products.PageTemplates :

vincentfretin commented 4 years ago

For the deprecation coming from zope/configuration/config.py:240, this means it's a zcml file. To find out which one, you can add put a import pdb; pdb.set_trace() in DeferredAndDeprecated:get in parts/packages/zope/deferredimport/deferredmodule.py line 53 Then do this:

$ bin/instance fg
/home/vincentfretin/workspace/buildout.coredev-5.2/src/Products.CMFPlacefulWorkflow/Products/CMFPlacefulWorkflow/permissions.py:5: DeprecationWarning: setDefaultRoles is deprecated. Please use addPermission from AccessControl.Permission.
  from Products.CMFCore.permissions import setDefaultRoles
[24] > /home/vincentfretin/.buildout/eggs/zope.deferredimport-4.3.1-py2.7.egg/zope/deferredimport/deferredmodule.py(55)get()
-> return super(DeferredAndDeprecated, self).get()
(Pdb++) c
/home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py:240: DeprecationWarning: IObjectEvent is deprecated. Import from zope.interface.interfaces
  obj = getattr(mod, oname)
[66] > /home/vincentfretin/.buildout/eggs/zope.deferredimport-4.3.1-py2.7.egg/zope/deferredimport/deferredmodule.py(55)get()
-> return super(DeferredAndDeprecated, self).get()
(Pdb++) u
[65] > /home/vincentfretin/.buildout/eggs/zope.deferredimport-4.3.1-py2.7.egg/zope/deferredimport/deferredmodule.py(71)__getattr__()
-> v = get.get()
(Pdb++) u
[64] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py(240)resolve()
-> obj = getattr(mod, oname)
(Pdb++) u
[63] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/fields.py(171)fromUnicode()
-> value = self.context.resolve(name)
(Pdb++) name
'zope.component.interfaces.IObjectEvent'
(Pdb++) u
[62] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/fields.py(268)fromUnicode()
-> v = vt.fromUnicode(s)
(Pdb++) u
[61] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py(1686)toargs()
-> args[str(name)] = field.fromUnicode(s)
(Pdb++) u
[60] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py(867)finish()
-> args = toargs(context, *self.argdata)
(Pdb++) u
[59] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/config.py(703)end()
-> self.stack.pop().finish()
(Pdb++) u
[58] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/xmlconfig.py(391)endElementNS()
-> self.context.end()
(Pdb++) u
[57] > /usr/lib/python2.7/xml/sax/expatreader.py(384)end_element_ns()
-> self._cont_handler.endElementNS(pair, None)
(Pdb++) u
[56] > /build/python2.7-UKCoZ3/python2.7-2.7.17/Modules/pyexpat.c(623)EndElement()
(Pdb++) u
[55] > /usr/lib/python2.7/xml/sax/expatreader.py(220)feed()
-> self._parser.Parse(data, isFinal)
(Pdb++) u
[54] > /usr/lib/python2.7/xml/sax/xmlreader.py(123)parse()
-> self.feed(buffer)
(Pdb++) u
[53] > /usr/lib/python2.7/xml/sax/expatreader.py(111)parse()
-> xmlreader.IncrementalParser.parse(self, source)
(Pdb++) u
[52] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/xmlconfig.py(407)processxmlfile()
-> parser.parse(src)
(Pdb++) u
[51] > /home/vincentfretin/.buildout/eggs/zope.configuration-4.3.1-py2.7.egg/zope/configuration/xmlconfig.py(557)include()
-> processxmlfile(f, context)
(Pdb++) f
<open file '/home/vincentfretin/.buildout/eggs/Products.CMFUid-3.0.2-py2.7.egg/Products/CMFUid/event.zcml', mode 'r' at 0x7ffb64a875d0>
vincentfretin commented 4 years ago

So I got those ones:

vincentfretin commented 4 years ago

If you fixed one of those warnings, please add a comment to this issue or add a reference to another issue or PR and check the checkboxes here. Thanks.

vincentfretin commented 4 years ago

@mauritsvanrees I see you're fixing some of the warnings, don't forget to check them here. Thanks.

gforcada commented 4 years ago

I did a bunch of them, but as I did them directly on github web UI interface I did not the extra mile of creating a news snippet, my guess was that for a such small/trivial change it would not be needed... @mauritsvanrees as a release manager, you can enforce/allow this :angel: :slightly_smiling_face:

mauritsvanrees commented 4 years ago

@vincentfretin Ah, thanks for the link to this issue.

@gforcada Seems okay to skip the news snippet, although I do add them myself. You should be sure though that the change is fine in all supported Plone versions that use the master branch. But usually it should be fine if the Jenkins PR jobs pass.

mauritsvanrees commented 4 years ago

I have been fixing a few, and usually just merging them myself, unless it looks like it needs a second look. Note that the list above is not complete, and it cannot be: a deprecation warning is shown only once. So when you fix a deprecation warning for ObjectEvent in package A, then it may show up for package B.

mauritsvanrees commented 4 years ago

Also, I have been editing the list, and adding remarks. Especially the warnings in plone.app.upgrade probably should not be fixed, because they are there to deal with older code, and still have to work in Plone 5.0, or at least when upgrading from 5.0. Maybe we can filter those warnings though, I have not tried yet.

mauritsvanrees commented 4 years ago

There are two deprecation warnings in plone.app.upgrade. In old upgrade code it is good to use the old import. So in https://github.com/plone/plone.app.upgrade/pull/237 I have used a temporary warnings filter to catch them.

For Products.CMFPlone.interfaces.ILanguageSchema there are a few more packages where the new import (plone.i18n.interfaces) should be used, probably with backwards compatibility import:

(Also in Products.ATContentTypes and Products.Archetypes, but we don't really need to touch those, as they won't work on Plone 6.)

mauritsvanrees commented 4 years ago
mauritsvanrees commented 4 years ago
mauritsvanrees commented 4 years ago
mauritsvanrees commented 4 years ago

In coredev 5.2 with Python 2.7, bin/instance-archetypes fg still gives lots of warnings, but I don't care about those. In the normal bin/instance 2.7 I get:

$ bin/instance fg
2020-09-04 13:42:32,823 WARNING [TemporaryStorage:93][MainThread] DEPRECATED: Usage of the package tempstorage is deprecated, as it is known to randomly lose data.
Especially on Zope 4. For details see https://github.com/zopefoundation/tempstorage/issues/8
and https://github.com/zopefoundation/tempstorage

/Users/maurits/shared-eggs/cp27m/tempstorage-5.1-py2.7.egg/tempstorage/TemporaryStorage.py:94: DeprecationWarning: DEPRECATED: Usage of the package tempstorage is deprecated, as it is known to randomly lose data.
Especially on Zope 4. For details see https://github.com/zopefoundation/tempstorage/issues/8
and https://github.com/zopefoundation/tempstorage

  warnings.warn(deprecation_warning, DeprecationWarning)
2020-09-04 13:42:33,808 INFO    [chameleon.config:38][MainThread] directory cache: /Users/maurits/community/plone-coredev/5.2/var/cache.
2020-09-04 13:42:36,521 INFO    [Zope:45][MainThread] Ready to handle requests
Starting server in PID 37293.
/Users/maurits/shared-eggs/cp27m/waitress-1.4.4-py2.7.egg/waitress/adjustments.py:445: DeprecationWarning: In future versions of Waitress clear_untrusted_proxy_headers will be set to True by default. You may opt-out by setting this value to False, or opt-in explicitly by setting this to True.
  DeprecationWarning,
Serving on http://0.0.0.0:8080
^C/Users/maurits/shared-eggs/cp27m/waitress-1.4.4-py2.7.egg/waitress/wasyncore.py:649: UserWarning: unclosed file <waitress.wasyncore.file_wrapper instance at 0x107fe2aa0>
  warnings.warn("unclosed file %r" % self, compat.ResourceWarning)

So we have the tempstorage stuff, which can only be (possibly) fixed when we upgrade to Products.TemporaryFolder 6.0, which might be something to save until Plone 6. See https://github.com/plone/Products.CMFPlone/issues/2957

And we have waitress warnings. Copying some text from a comment above:

waitress 1.4.4 :

There may be more warnings when you actually click around in Plone or when you run tests. But for startup, this is much better than it was.

mauritsvanrees commented 4 years ago

Coredev 5.2 on Python 3 has a few more warnings. These are the extra ones:

/Users/maurits/community/plone-coredev/py3/src/Products.CMFFormController/Products/CMFFormController/ControllerPythonScript.py:41: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp

/Users/maurits/shared-eggs/cp37m/calmjs.parse-1.2.4-py3.7.egg/calmjs/parse/io.py:7: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working
  from collections import Iterable
/Users/maurits/shared-eggs/cp37m/pyScss-1.3.7-py3.7-macosx-10.15-x86_64.egg/scss/namespace.py:172: DeprecationWarning: inspect.getargspec() is deprecated since Python 3.0, use inspect.signature() or inspect.getfullargspec()
  argspec = inspect.getargspec(function)
/Users/maurits/shared-eggs/cp37m/pyScss-1.3.7-py3.7-macosx-10.15-x86_64.egg/scss/selector.py:54: FutureWarning: Possible nested set at position 329
  ''', re.VERBOSE | re.MULTILINE)

Strange, when I started it half minute before, I saw several invalid escape sequences. Maybe they are only reported when starting it for the first time? Confirmed: when I remove the pyc files, the warnings are back:

/Users/maurits/community/plone-coredev/py3/src/Products.CMFFormController/Products/CMFFormController/ControllerPythonScript.py:61: DeprecationWarning: invalid escape sequence \S
  _nonempty_line = re.compile('(?m)^(.*\S.*)$')

And several more of those. It could be that there are even more, because I only removed the pyc files in the src directory, not in parts/packages.

So specific for Python 3:

Later found more invalid escape sequences:

Note: in the initial comment in https://github.com/plone/plone.app.caching/pull/62 I have put some hints on how to fix or review these invalid escape sequences.