search

zopefoundation / Zope

Zope is an open-source web application server.
https://zope.readthedocs.io
Other
357 stars 101 forks source link

IPersistentExtra missing in zodbverify/zodbupdate #792

Closed mauritsvanrees closed 4 years ago

mauritsvanrees commented 4 years ago

This is on Plone 5.2 with Python 2.7 when using zodbverify:

Done! Scanned 327067 records. 
Found 4516 records that could not be loaded. 
Exceptions and how often they happened: 
ImportError: No module named ResourceRegistries.interfaces.settings: 8
AttributeError: 'module' object has no attribute 'IPersistentExtra': 4508

The ResourceRegistries error is Plone-specific.

But I would expect that the IPersistentExtra gives a problem also for Zope projects when trying zodbverify or zodbupdate. In Zope 2 this was in App.interfaces and persistent was patched with this for bobobase_modification_time.

In Zope 4 it is no longer needed. But I think this interface is on a lot of persistent objects in Zope 2. So I would expect a problem for everyone who migrates their ZODB from Python 2 to Python 3. It seems a good candidate for putting in a zodbupdate dict, like here for OFS.

I could make a PR for this. But this leaves me wondering: hasn't somebody else run into this?

See also post 1 and post 2 on community.plone.org a few months ago.

dataflake commented 4 years ago

I have not run into this at all and we had a large migration coming from Zope 2.13.26 on Python 2.7. Tthe content is exclusively standard Zope content items like DTML Methods, Page Templates, Python Scripts, ZSQL Methods and some customized Folder subclasses. So at least for "standard" Zope content that interface makes no difference at all.

icemac commented 4 years ago

I've not run into this either migrating multiple large Data.fs in union.cms.

As the interface is put onto the class in
https://github.com/zopefoundation/Zope/blob/170258611d03f7fe71ac2f0a445b8f0562a29d30/src/App/PersistentExtra.py#L50

the interface should not be serialized with the instance into ZODB. Is there code in Plone or your code which uses alsoProvides or one of its cousins to set this interface on the instance?

jugmac00 commented 4 years ago

FWIW - I just converted my app`s Data.fs two days ago with current versions of both mentioned tools - no errors.

mauritsvanrees commented 4 years ago

the interface should not be serialized with the instance into ZODB. Is there code in Plone or your code which uses alsoProvides or one of its cousins to set this interface on the instance?

No. Or at least not in an explicit way where I find it.

I just created a fresh database and Plone site with default content in Plone 5.1, then upgraded to 5.2, and zodbverify did not complain about IPersistentExtra. So it does not happen in all Plone Sites.

The unpickler.load line from zodbverify prints this:

  File "/Users/maurits/shared-eggs/cp27m/zodbverify-1.0.2-py2.7.egg/zodbverify/verify.py", line 60, in verify_record
    unpickler.load()
AttributeError: 'module' object has no attribute 'IPersistentExtra'

    0: \x80 PROTO      2
    2: (    MARK
    3: c        GLOBAL     'webdav.EtagSupport EtagBaseInterface'
   41: q        BINPUT     2
   43: c        GLOBAL     'Acquisition.interfaces IAcquirer'
   77: q        BINPUT     3
   79: c        GLOBAL     'zope.annotation.interfaces IAnnotatable'
  120: q        BINPUT     4
  122: c        GLOBAL     'zope.annotation.interfaces IAttributeAnnotatable'
  172: q        BINPUT     5
  174: c        GLOBAL     'plone.uuid.interfaces IAttributeUUID'
  212: q        BINPUT     6
  214: c        GLOBAL     'Products.CMFDynamicViewFTI.interfaces IBrowserDefault'
  269: q        BINPUT     7
  271: c        GLOBAL     'Products.CMFCore.interfaces._content ICatalogAware'
  323: q        BINPUT     8
  325: c        GLOBAL     'Products.CMFCore.interfaces._content ICatalogableDublinCore'
  386: q        BINPUT     9
  388: c        GLOBAL     'zope.location.interfaces IContained'
  425: q        BINPUT     10
  427: c        GLOBAL     'Products.CMFCore.interfaces._content IContentish'
  477: q        BINPUT     11
  479: c        GLOBAL     'OFS.interfaces ICopySource'
  507: q        BINPUT     12
  509: c        GLOBAL     'webdav.interfaces IDAVResource'
  541: q        BINPUT     13
  543: c        GLOBAL     'plone.dexterity.interfaces IDexterityContent'
  589: q        BINPUT     14
  591: c        GLOBAL     'plone.app.relationfield.interfaces IDexterityHasRelations'
  650: q        BINPUT     15
  652: c        GLOBAL     'plone.dexterity.interfaces IDexterityItem'
  695: q        BINPUT     16
  697: c        GLOBAL     'plone.app.iterate.dexterity.interfaces IDexterityIterateAware'
  760: q        BINPUT     17
  762: c        GLOBAL     'Products.CMFCore.interfaces._content IDublinCore'
  812: q        BINPUT     18
  814: c        GLOBAL     'Products.CMFCore.interfaces._content IDynamicType'
  865: q        BINPUT     19
  867: c        GLOBAL     'OFS.interfaces IFTPAccess'
  894: q        BINPUT     20
  896: c        GLOBAL     'z3c.relationfield.interfaces IHasIncomingRelations'
  948: q        BINPUT     21
  950: c        GLOBAL     'z3c.relationfield.interfaces IHasOutgoingRelations'
 1002: q        BINPUT     22
 1004: c        GLOBAL     'z3c.relationfield.interfaces IHasRelations'
 1048: q        BINPUT     23
 1050: c        GLOBAL     'plone.namedfile.interfaces IImageScaleTraversable'
 1101: q        BINPUT     24
 1103: c        GLOBAL     'OFS.interfaces IItem'
 1125: q        BINPUT     25
 1127: c        GLOBAL     'plone.app.iterate.interfaces IIterateAware'
 1171: q        BINPUT     26
 1173: c        GLOBAL     'plone.portlets.interfaces ILocalPortletAssignable'
 1224: q        BINPUT     27
 1226: c        GLOBAL     'zope.location.interfaces ILocation'
 1262: q        BINPUT     28
 1264: c        GLOBAL     'OFS.interfaces IManageable'
 1292: q        BINPUT     29
 1294: c        GLOBAL     'Products.CMFCore.interfaces._content IMinimalDublinCore'
 1351: q        BINPUT     30
 1353: c        GLOBAL     'Products.CMFCore.interfaces._content IMutableDublinCore'
 1410: q        BINPUT     31
 1412: c        GLOBAL     'Products.CMFCore.interfaces._content IMutableMinimalDublinCore'
 1476: q        BINPUT     32
 1478: c        GLOBAL     'plone.app.content.interfaces INameFromTitle'
 1523: q        BINPUT     33
 1525: c        GLOBAL     'Products.CMFCore.interfaces._content IOpaqueItemManager'
 1582: q        BINPUT     34
 1584: c        GLOBAL     'AccessControl.interfaces IOwned'
 1617: q        BINPUT     35
 1619: c        GLOBAL     'AccessControl.interfaces IPermissionMappingSupport'
 1671: q        BINPUT     36
 1673: c        GLOBAL     'persistent.interfaces IPersistent'
 1708: q        BINPUT     37
 1710: c        GLOBAL     'App.interfaces IPersistentExtra'
 1743: q        BINPUT     38
 1745: c        GLOBAL     'OFS.interfaces IPropertyManager'
 1778: q        BINPUT     39
 1780: c        GLOBAL     'zeelandia.basicrecipes.recipe IRecipe'
 1819: q        BINPUT     40
 1821: c        GLOBAL     'plone.app.relationfield.behavior IRelatedItems'
 1869: q        BINPUT     41
 1871: c        GLOBAL     'AccessControl.interfaces IRoleManager'
 1910: q        BINPUT     42
 1912: c        GLOBAL     'plone.contentrules.engine.interfaces IRuleAssignable'
 1966: q        BINPUT     43
 1968: c        GLOBAL     'collective.behavior.seo.interfaces ISEOFieldsMarker'
 2021: q        BINPUT     44
 2023: c        GLOBAL     'Products.CMFDynamicViewFTI.interfaces ISelectableBrowserDefault'
 2088: q        BINPUT     45
 2090: c        GLOBAL     'OFS.interfaces ISimpleItem'
 2118: q        BINPUT     46
 2120: c        GLOBAL     'OFS.interfaces ITraversable'
 2149: q        BINPUT     47
 2151: c        GLOBAL     'plone.uuid.interfaces IUUIDAware'
 2185: q        BINPUT     48
 2187: c        GLOBAL     'App.interfaces IUndoSupport'
 2216: q        BINPUT     49
 2218: c        GLOBAL     'Products.CMFCore.interfaces._content IWorkflowAware'
 2271: q        BINPUT     50
 2273: c        GLOBAL     'webdav.interfaces IWriteLock'
 2303: q        BINPUT     51
 2305: c        GLOBAL     'OFS.interfaces IZopeObject'
 2333: q        BINPUT     52
 2335: c        GLOBAL     'zope.interface Interface'
 2361: q        BINPUT     53
 2363: c        GLOBAL     'plone.supermodel.model Schema'
 2394: q        BINPUT     54
 2396: t        TUPLE      (MARK at 2)
 2397: \x85 TUPLE1
 2398: \x85 TUPLE1
 2399: \x85 TUPLE1
 2400: q    BINPUT     55
 2402: .    STOP
highest protocol among opcodes = 2

That suggests it is a dexterity content item, or at least something related to it. The same is true for the default contents of the Plone 5.1 site though, so I don't get why it is not always a problem.

Anyway, I will close this, since it is a Plone problem.