Closed mauritsvanrees closed 3 years ago
In coredev 5.2 with bin/zopepy
:
>>> import pkg_resources
>>> from pprint import pprint as pp
>>> pp([(ep.dist.project_name, ep.dist.key) for ep in list(pkg_resources.iter_entry_points(group="z3c.autoinclude.plugin"))])
[('plone.app.upgrade', 'plone.app.upgrade'),
('plone.restapi', 'plone.restapi'),
('mockup', 'mockup'),
('plone.app.caching', 'plone.app.caching'),
('plone.app.contentlisting', 'plone.app.contentlisting'),
('plone.app.contenttypes', 'plone.app.contenttypes'),
('plone.app.dexterity', 'plone.app.dexterity'),
('plone.app.discussion', 'plone.app.discussion'),
('plone.app.event', 'plone.app.event'),
('plone.app.intid', 'plone.app.intid'),
('plone.app.iterate', 'plone.app.iterate'),
('plone.app.linkintegrity', 'plone.app.linkintegrity'),
('plone.app.lockingbehavior', 'plone.app.lockingbehavior'),
('plone.app.multilingual', 'plone.app.multilingual'),
('plone.app.querystring', 'plone.app.querystring'),
('plone.app.theming', 'plone.app.theming'),
('plone.app.users', 'plone.app.users'),
('plone.app.versioningbehavior', 'plone.app.versioningbehavior'),
('plone.app.widgets', 'plone.app.widgets'),
('plone.batching', 'plone.batching'),
('plone.formwidget.namedfile', 'plone.formwidget.namedfile'),
('plone.outputfilters', 'plone.outputfilters'),
('plone.portlet.static', 'plone.portlet.static'),
('plone.resource', 'plone.resource'),
('plone.rest', 'plone.rest'),
('plone.staticresources', 'plone.staticresources'),
('plone.stringinterp', 'plone.stringinterp'),
('plone.subrequest', 'plone.subrequest'),
('plonetheme.barceloneta', 'plonetheme.barceloneta'),
('Products.CMFDiffTool', 'products.cmfdifftool'),
('Products.CMFEditions', 'products.cmfeditions'),
('archetypes.multilingual', 'archetypes.multilingual'),
('archetypes.schemaextender', 'archetypes.schemaextender'),
('plone.app.blob', 'plone.app.blob'),
('plone.app.collection', 'plone.app.collection'),
('plone.app.imaging', 'plone.app.imaging'),
('plone.app.referenceablebehavior', 'plone.app.referenceablebehavior'),
('plone.formwidget.recurrence', 'plone.formwidget.recurrence'),
('collective.MockMailHost', 'collective.mockmailhost'),
('z3c.formwidget.query', 'z3c.formwidget.query')]
For fun, this is the list of all packages of which the configure.zcml
is loaded, some explicitly by the configure.zcml
in Products.CMFPlone
(explicit.txt
below), others by z3c.autoinclude
(auto.txt
).
So:
includePlugins
setup.py
. Most of those are packages that started life outside of the core, but are now in.$ colordiff -U 30 explicit.txt auto.txt
--- explicit.txt 2020-02-24 21:59:46.000000000 +0100
+++ auto.txt 2020-02-24 21:59:02.000000000 +0100
@@ -1,36 +1,40 @@
-Products.CMFCore
-Products.GenericSetup
-borg.localrole
+Products.CMFDiffTool
+Products.CMFEditions
+archetypes.multilingual
+archetypes.schemaextender
+collective.MockMailHost
mockup
-plone.app.content
-plone.app.contentmenu
-plone.app.contentrules
+plone.app.blob
+plone.app.caching
+plone.app.collection
+plone.app.contentlisting
plone.app.contenttypes
-plone.app.customerize
+plone.app.dexterity
plone.app.discussion
-plone.app.i18n
-plone.app.layout
+plone.app.event
+plone.app.imaging
+plone.app.intid
+plone.app.iterate
plone.app.linkintegrity
-plone.app.locales
+plone.app.lockingbehavior
plone.app.multilingual
-plone.app.portlets
-plone.app.redirector
-plone.app.registry
+plone.app.querystring
+plone.app.referenceablebehavior
plone.app.theming
+plone.app.upgrade
plone.app.users
-plone.app.uuid
-plone.app.viewletmanager
-plone.app.vocabularies
-plone.app.workflow
+plone.app.versioningbehavior
+plone.app.widgets
plone.batching
-plone.browserlayer
-plone.indexer
-plone.memoize
+plone.formwidget.namedfile
+plone.formwidget.recurrence
plone.outputfilters
-plone.portlet.collection
plone.portlet.static
-plone.protect
-plone.session
+plone.resource
+plone.rest
+plone.restapi
plone.staticresources
-plone.theme
-zope.app.locales
+plone.stringinterp
+plone.subrequest
+plonetheme.barceloneta
+z3c.formwidget.query
See this thread on Twitter started by @jensens with an analysis of Plone startup time:
Plone Startup time (lots addons)
16.29s
(Zope2/Startup/serve.py) with biggest part 11.09s includePluginsDirective (z3c/autoinclude/zcml.py) with 8.53s find_packages (z3c/autoinclude/utils.py) mainly file system-operations on a NVMe SSD Lets get rid of z3c.autoinclude
pp([(ep.dist.project_name, ep.dist.key) for ep in list(pkg_resources.iter_entry_points(group="z3c.autoinclude.plugin"))])
You get here dotted paths because in lots o cases projectname == dottedpath. But this is not mandatory, the dist name can be different from the dottedpath or it may contain several paths (look at the Zope
package or Pillow
)
I was looking for a way to print which packages actually get auto included. I decided to put this in z3c.autoinclude
itself. Set an environment variable Z3C_AUTOINCLUDE_DEBUG
, startup Zope, and it prints the info in a way that you can copy-paste into zcml. That helps for migrating to explicitly including the zcml.
See https://github.com/zopefoundation/z3c.autoinclude/pull/13
I wanted to try something, to see what kind of information we can get with pkg_resources
, and if that would be enough for our use cases.
$ python3.7 -mvenv plonepip
$ cd plonepip
$ . bin/activate
$ pip install -c https://dist.plone.org/release/5.2.2-pending/constraints.txt Plone
...
$ python
>>> import pkg_resources
>>> pkg_resources.working_set
<pkg_resources.WorkingSet object at 0x10bed5090>
>>> len(list(pkg_resources.working_set))
244
>>> list(pkg_resources.working_set)[0]
Zope2 4.0 (/Users/maurits/tmp/plonepip/lib/python3.7/site-packages)
>>> list(pkg_resources.working_set)[0].__class__
<class 'pkg_resources.DistInfoDistribution'>
>>> for dist in pkg_resources.working_set:
... entry_map = pkg_resources.get_entry_map(dist)
... if "z3c.autoinclude.plugin" in entry_map:
... print("{}: {}".format(dist, entry_map["z3c.autoinclude.plugin"]))
...
z3c.formwidget.query 0.17: {'target': EntryPoint.parse('target = plone')}
Products.CMFEditions 3.3.4: {'target': EntryPoint.parse('target = plone')}
Products.CMFDiffTool 3.3.1: {'target': EntryPoint.parse('target = plone')}
plonetheme.barceloneta 2.1.8: {'target': EntryPoint.parse('target = plone')}
plone.subrequest 1.9.2: {'target': EntryPoint.parse('target = plone')}
plone.stringinterp 1.3.2: {'target': EntryPoint.parse('target = plone')}
plone.staticresources 1.3.1: {'target': EntryPoint.parse('target = plone')}
plone.restapi 6.13.7: {'target': EntryPoint.parse('target = plone')}
plone.rest 1.6.1: {'target': EntryPoint.parse('target = plone')}
plone.resource 2.1.2: {'target': EntryPoint.parse('target = plone')}
plone.portlet.static 3.1.4: {'target': EntryPoint.parse('target = plone')}
plone.outputfilters 4.0.1: {'target': EntryPoint.parse('target = plone')}
plone.formwidget.recurrence 2.1.4: {'target': EntryPoint.parse('target = plone')}
plone.formwidget.namedfile 2.1.0: {'target': EntryPoint.parse('target = plone')}
plone.batching 1.1.6: {'target': EntryPoint.parse('target = plone')}
plone.app.widgets 3.0.4: {'target': EntryPoint.parse('target = plone')}
plone.app.versioningbehavior 1.4.0: {'target': EntryPoint.parse('target = plone')}
plone.app.users 2.6.5: {'target': EntryPoint.parse('target = plone')}
plone.app.upgrade 2.0.33: {'target': EntryPoint.parse('target = plone')}
plone.app.theming 4.1.2: {'target': EntryPoint.parse('target = plone')}
plone.app.querystring 1.4.14: {'target': EntryPoint.parse('target = plone')}
plone.app.multilingual 5.6.1: {'target': EntryPoint.parse('target = plone')}
plone.app.lockingbehavior 1.0.7: {'target': EntryPoint.parse('target = plone')}
plone.app.linkintegrity 3.3.13: {'target': EntryPoint.parse('target = plone')}
plone.app.iterate 3.3.14: {'target': EntryPoint.parse('target = plone')}
plone.app.intid 1.1.4: {'target': EntryPoint.parse('target = plone')}
plone.app.event 3.2.7: {'target': EntryPoint.parse('target = plone')}
plone.app.discussion 3.4.2: {'target': EntryPoint.parse('target = plone')}
plone.app.dexterity 2.6.5: {'target': EntryPoint.parse('target = plone')}
plone.app.contenttypes 2.1.9: {'target': EntryPoint.parse('target = plone')}
plone.app.contentlisting 2.0.2: {'target': EntryPoint.parse('target = plone')}
plone.app.caching 2.0.6: {'target': EntryPoint.parse('target = plone')}
mockup 3.2.1: {'target': EntryPoint.parse('target = mockup')}
Get the interesting ones:
>>> dists = [dist for dist in pkg_resources.working_set if "z3c.autoinclude.plugin" in pkg_resources.get_entry_map(dist)]
>>> len(dists)
33
>>> dist = dists[0]
>>> dist
z3c.formwidget.query 0.17 (/Users/maurits/tmp/plonepip/lib/python3.7/site-packages)
Show attributes and simple callables of a distribution:
>>> def dist_info(dist):
... for attr in dir(dist):
... if not attr.startswith("_"):
... attrib = getattr(dist, attr)
... if callable(attrib):
... try:
... attrib = attrib()
... except:
... continue
... print("{}: {}".format(attr, attrib))
>>> dist_info(dist)
EQEQ: re.compile('([\\(,])\\s*(\\d.*?)\\s*([,\\)])')
PKG_INFO: METADATA
activate: None
as_requirement: z3c.formwidget.query==0.17
check_version_conflict: None
clone: z3c.formwidget.query 0.17
egg_info: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages/z3c.formwidget.query-0.17.dist-info
egg_name: z3c.formwidget.query-0.17-py3.7
extras: ['test']
get_entry_map: {'z3c.autoinclude.plugin': {'target': EntryPoint.parse('target = plone')}}
has_version: True
hashcmp: (<Version('0.17')>, -1, 'z3c.formwidget.query', '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages', '', '')
key: z3c.formwidget.query
loader: None
location: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages
module_path: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages
parsed_version: 0.17
platform: None
precedence: -1
project_name: z3c.formwidget.query
py_version: None
requires: [Requirement.parse('zope.schema'), Requirement.parse('setuptools'), Requirement.parse('zope.i18nmessageid'), Requirement.parse('z3c.form>=3.2.10'), Requirement.parse('zope.component'), Requirement.parse('zope.interface')]
version: 0.17
Can we import these by project_name?
>>> import importlib
>>> for dist in dists:
... importlib.import_module(dist.project_name)
...
<module 'z3c.formwidget.query' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/z3c/formwidget/query/__init__.py'>
<module 'Products.CMFEditions' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/Products/CMFEditions/__init__.py'>
<module 'Products.CMFDiffTool' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/Products/CMFDiffTool/__init__.py'>
<module 'plonetheme.barceloneta' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plonetheme/barceloneta/__init__.py'>
<module 'plone.subrequest' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/subrequest/__init__.py'>
<module 'plone.stringinterp' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/stringinterp/__init__.py'>
<module 'plone.staticresources' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/staticresources/__init__.py'>
<module 'plone.restapi' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/restapi/__init__.py'>
<module 'plone.rest' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/rest/__init__.py'>
<module 'plone.resource' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/resource/__init__.py'>
<module 'plone.portlet.static' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/portlet/static/__init__.py'>
<module 'plone.outputfilters' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/outputfilters/__init__.py'>
<module 'plone.formwidget.recurrence' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/formwidget/recurrence/__init__.py'>
<module 'plone.formwidget.namedfile' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/formwidget/namedfile/__init__.py'>
<module 'plone.batching' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/batching/__init__.py'>
<module 'plone.app.widgets' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/widgets/__init__.py'>
<module 'plone.app.versioningbehavior' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/versioningbehavior/__init__.py'>
<module 'plone.app.users' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/users/__init__.py'>
<module 'plone.app.upgrade' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/upgrade/__init__.py'>
<module 'plone.app.theming' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/theming/__init__.py'>
<module 'plone.app.querystring' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/querystring/__init__.py'>
<module 'plone.app.multilingual' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/multilingual/__init__.py'>
<module 'plone.app.lockingbehavior' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/lockingbehavior/__init__.py'>
<module 'plone.app.linkintegrity' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/linkintegrity/__init__.py'>
<module 'plone.app.iterate' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/iterate/__init__.py'>
<module 'plone.app.intid' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/intid/__init__.py'>
<module 'plone.app.event' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/event/__init__.py'>
<module 'plone.app.discussion' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/discussion/__init__.py'>
<module 'plone.app.dexterity' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/dexterity/__init__.py'>
<module 'plone.app.contenttypes' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/contenttypes/__init__.py'>
<module 'plone.app.contentlisting' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/contentlisting/__init__.py'>
<module 'plone.app.caching' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/plone/app/caching/__init__.py'>
<module 'mockup' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/mockup/__init__.py'>
So project_name
works, at least for these.
And key
?
>>> for dist in dists:
... importlib.import_module(dist.key)
...
<module 'z3c.formwidget.query' from '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages/z3c/formwidget/query/__init__.py'>
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/Users/maurits/.pyenv/versions/3.7.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'products'
>>> dist
Products.CMFEditions 3.3.4 (/Users/maurits/tmp/plonepip/lib/python3.7/site-packages)
>>> dist.key
'products.cmfeditions'
>>> dist.project_name
'Products.CMFEditions'
So importing dist.key
does not work, but importing dist.project_name
does. I don't know if that is true in all cases.
As Jens says above:
You get here dotted paths because in lots o cases projectname == dottedpath. But this is not mandatory, the dist name can be different from the dottedpath or it may contain several paths (look at the Zope package or Pillow)
Let's look at the Zope dist:
>>> dist = [dist for dist in pkg_resources.working_set if dist.project_name == 'Zope'][0]
>>> dist_info(dist)
EQEQ: re.compile('([\\(,])\\s*(\\d.*?)\\s*([,\\)])')
PKG_INFO: METADATA
activate: None
as_requirement: Zope==4.5
check_version_conflict: None
clone: Zope 4.5
egg_info: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages/Zope-4.5.dist-info
egg_name: Zope-4.5-py3.7
extras: ['docs', 'wsgi']
get_entry_map: {'console_scripts': {'addzope2user': EntryPoint.parse('addzope2user = Zope2.utilities.adduser:main'), 'mkwsgiinstance': EntryPoint.parse('mkwsgiinstance = Zope2.utilities.mkwsgiinstance:main'), 'runwsgi': EntryPoint.parse('runwsgi = Zope2.Startup.serve:main'), 'zconsole': EntryPoint.parse('zconsole = Zope2.utilities.zconsole:main')}, 'paste.app_factory': {'main': EntryPoint.parse('main = Zope2.Startup.run:make_wsgi_app')}, 'paste.filter_app_factory': {'httpexceptions': EntryPoint.parse('httpexceptions = ZPublisher.httpexceptions:main')}, 'zodbupdate': {'renames': EntryPoint.parse('renames = OFS:zodbupdate_rename_dict')}, 'zodbupdate.decode': {'decodes': EntryPoint.parse('decodes = OFS:zodbupdate_decode_dict')}}
has_version: True
hashcmp: (<Version('4.5')>, -1, 'zope', '/Users/maurits/tmp/plonepip/lib/python3.7/site-packages', '', '')
key: zope
loader: None
location: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages
module_path: /Users/maurits/tmp/plonepip/lib/python3.7/site-packages
parsed_version: 4.5
platform: None
precedence: -1
project_name: Zope
py_version: None
requires: [Requirement.parse('transaction>=2.4'), Requirement.parse('zope.pagetemplate>=4.0.2'), Requirement.parse('zope.traversing'), Requirement.parse('ZConfig>=2.9.2'), Requirement.parse('zope.contenttype'), Requirement.parse('zope.container'), Requirement.parse('zope.event'), Requirement.parse('ExtensionClass'), Requirement.parse('PasteDeploy'), Requirement.parse('z3c.pt'), Requirement.parse('six'), Requirement.parse('zope.configuration'), Requirement.parse('zope.security'), Requirement.parse('zope.interface>=3.8'), Requirement.parse('AccessControl>=4.2'), Requirement.parse('waitress'), Requirement.parse('zope.deferredimport'), Requirement.parse('zope.browserpage>=4.4.0.dev0'), Requirement.parse('zope.contentprovider'), Requirement.parse('zope.tales>=5.0.2'), Requirement.parse('zope.publisher'), Requirement.parse('zope.testing'), Requirement.parse('setuptools>=36.2'), Requirement.parse('zope.size'), Requirement.parse('zope.testbrowser'), Requirement.parse('RestrictedPython'), Requirement.parse('Chameleon>=3.7.0'), Requirement.parse('zope.ptresource'), Requirement.parse('zope.location'), Requirement.parse('zope.tal'), Requirement.parse('Persistence'), Requirement.parse('zope.lifecycleevent'), Requirement.parse('zope.schema'), Requirement.parse('zope.site'), Requirement.parse('zope.exceptions'), Requirement.parse('DocumentTemplate>=3.0b9'), Requirement.parse('Acquisition'), Requirement.parse('DateTime'), Requirement.parse('zope.component'), Requirement.parse('zope.browser'), Requirement.parse('zope.viewlet'), Requirement.parse('zope.proxy'), Requirement.parse('zope.sequencesort'), Requirement.parse('zope.processlifetime'), Requirement.parse('zExceptions>=3.4'), Requirement.parse('zope.i18n[zcml]'), Requirement.parse('zope.i18nmessageid'), Requirement.parse('MultiMapping'), Requirement.parse('zope.browserresource>=3.11'), Requirement.parse('BTrees'), Requirement.parse('ZODB'), Requirement.parse('zope.globalrequest'), Requirement.parse('zope.browsermenu')]
version: 4.5
>>> importlib.import_module(dist.project_name)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/maurits/.pyenv/versions/3.7.7/lib/python3.7/importlib/__init__.py", line 127, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
File "<frozen importlib._bootstrap>", line 983, in _find_and_load
File "<frozen importlib._bootstrap>", line 965, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'Zope'
So importing by project name indeed does not work for all distributions.
>>> for dist in pkg_resources.working_set:
... try:
... ignored = importlib.import_module(dist.project_name)
... except ImportError:
... print("Failed importing by project name: {}".format(dist.project_name))
...
Failed importing by project name: Zope
Failed importing by project name: ZODB3
Failed importing by project name: WSGIProxy2
Failed importing by project name: WebTest
Failed importing by project name: WebOb
Failed importing by project name: Unidecode
Failed importing by project name: python-gettext
Failed importing by project name: python-dateutil
Failed importing by project name: pyScss
Failed importing by project name: PyJWT
Failed importing by project name: Plone
Failed importing by project name: Pillow
Failed importing by project name: PasteDeploy
Failed importing by project name: Markdown
Failed importing by project name: importlib-metadata
Failed importing by project name: Chameleon
Failed importing by project name: beautifulsoup4
Failed importing by project name: attrs
That is 18 packages, most of which do not have any zcml.
So in a z3c.autoinclude
alternative it might be an option to require that the project name is importable.
I just keep hoping that, given a distribution, pkg_resources
or some other tool will give us what we need...
BTW, it would also be useful if an entry point can signal that the Python code of a package needs to be loaded at startup, even when it has no zcml.
Currently, if you add a package in the instance eggs, its code does not get loaded. You can force load by putting it in the zcml instance option, or it happens when the package declares a z3c.autoinclude
entrypoint. I am not sure from the top of my head if you get an error then when it has no actual zcml.
The only way to make this happen when no zcml is involved, is to put the package in the special Products
namespace. This is what we do for Plone hotfixes: latest hotfix is called Products.PloneHotfix20200121
.
So what I am hoping we can do with an entry point, using a variant of the notation from the initial comment:
# Import the package by name, using its dist.project_name:
[zope.autoinclude]
# Import the package by the given package name:
[zope.autoinclude]
package = collective.mypackage
# For importing multiple packages, like in Zope:
[zope.autoinclude]
package = package1,package2,package3
# Import the package and load zcml:
[zope.autoinclude]
package = collective.mypackage
zcml = configure.zcml,overrides.zcml
# When loading multiple packages, maybe we could support importing a different zcml per package,
# but importing multiple zcmls in one package then gets difficult to express:
[zope.autoinclude]
package = package1:configure.zcml,package2:overrides.zcml
Note that currently you need to add a target:
[z3c.autoinclude.plugin]
target = plone
There is code in Plone to only load the zcml of entry points that have target plone
. So we may need to add a target as well in the new situation. Theoretically Zope could load zcml targeted at zope
, or a plain CMF site could load zcml targeted at cmf
. But the only target that I see other than plone
, is mockup
, which is in mockup
itself so I wonder if that is a mistake.
Also note that, as far as I know, only Plone uses z3c.autoinclude
, so maybe the target is overkill. And we could make a function that looks for entry points that either explicitly have target = plone
or have no target.
Enough brain dump for today. :-)
Okay, I have some initial code for a possible plone.autoinclude
, very much Work In Progress:
I improved some more. If anyone wants to try it out (please not on production...) I have updated the README. Especially see the section on installing with pip.
Status:
z3c.autoinclude
entry point (most should not have this, but that is beside the point). Needs some changes in CMFPlone or additional zcml in the buildout so z3c.autoinclude
is not used but only our code is used.bin/mkwsgiinstance
for the core Plone packages that have the z3c.autoinclude
entry point.TODO
items.I am also experimenting with tox
, not yet for testing, but for QA (lint), isort/black, release. See tox.ini
. I am a bit inspired by https://github.com/zopefoundation/meta.
Note that there is no master
branch, but only a main
branch. GitHub recommends it and has it in the instructions when you create a new repository. Let's see.
See CMPlone
branch plone-autoinclude
for the changes that would be needed to make the switch from z3c.autoinclude
to plone.autoinclude
.
A lot of progress was made by @tschorr and me during the 'not an Alpine City Sprint' sprint. Mostly tests. With tox
you can run them all locally. Takes less than 30 seconds for me when run in parallel (tox -p auto
). GitHub Actions is setup for this as well. We have several test packages in the repository. Some tox tests install them with pip
, others in a buildout. Both work. Tested on Python 3.6-3.9 plus PyPy3.
Biggest change to the actual code in plone.autoinclude
is that we now support an own EntryPoint plone.autoinclude.plugin
. I had some ideas for that in a comment above. But entry points are less flexible than I had hoped, so those ideas are for the most part not possible. See comments in the load_own_packages
function. The summary is that entry points can only have one option: one key-value pair.
Let me give some examples of entry points in setup.py
.
An empty EntryPoint is completely ignored, you cannot find it with pkg_resources.iter_entry_points
.
# ignored
[plone.autoinclude.plugin]
So you must pass an option. And it must have a value, otherwise you get an error when pip or buildout installs it:
# error
[plone.autoinclude.plugin]
target =
You can pass a target, like z3c.autoinclude
supports:
[plone.autoinclude.plugin]
target = plone
Or when your package is called A and it has module B, you can specify a module name:
[plone.autoinclude.plugin]
module = B
In this case you do not specify a target. In our code this is no problem: when we look for all entry points with a specific target (plone
) and no target is set, we still include it.
In fact you cannot specify both a target and a module if you wanted to.
One of the two is ignored by pkg_resources
and is invisible to us:
[plone.autoinclude.plugin]
target = plone
# ignored:
module = B
TODO:
Hmmm. What about making module = ..
mandatory? Otherwise it is very confusing IMO.
In fact you cannot specify both a target and a module if you wanted to. One of the two is ignored by
pkg_resources
and is invisible to us:[plone.autoinclude.plugin] target = plone # ignored: module = B
I wondered about this because you can e.g. specify multiple console script entry points. I've played with a different way of reading the entry maps in https://github.com/plone/plone.autoinclude/pull/1 and I can read both target and module with my variant. I tried to keep the current semantics to keep the tests green.
On second thought, instead of using pkg_resources
for iterating over entry points, we could use importlib.metadata.entry_points()
. That would remove setuptools
from the list of requirements.
... but that API is only available since Python 3.8.7 :-(
The way @tschorr reads the entry maps works, so we could indeed use both a target and module. I have merged it. We need a test package using that, but can be done later.
I have released 1.0.0a1 of plone.autoinclude
.
I think it is ready for inclusion in core Plone 6, and I will make a proper PLIP for it.
We may want to have this battle-tested more before including it, but there is 100 percent test coverage.
I have released version 1.0.0a2, which I tested in a customer project.
I will close this 'pre-PLIP' in favour of a proper PLIP: #3339.
Copied from the Alpine City Strategic Sprint 2020 doc:
Plone on pip
pip install Plone
will work with Plone 5.2.1. But z3c.autoinclude can fail when add-ons useincludeDependencies
in zcml. And buildout should get an option/extension to not install any Python packages anymore. Or we switch away from buildout to something else (Ansible?), but there are still all kinds of nice buildout recipes. Integrating and finishing https://github.com/datakurre/plonectl Some initial work by Six Feet Up https://github.com/sixfeetup/dietplonedockerGood example is the warehouse project how to deploy. Ansible role should be updated too, to be able to run without buildout. Maybe we could create an even lighter version of ansible.plone_server role, which does just the Plone setup, nothing more.
Who:
Prediscussion for PLIP:
Discourage use of
z3c.autoinclude
: Problem here is:We want pip installed packages automatically picked up at Zope startup Idea:
pkg_resources.iter_entry_points(group="z3c.autoinclude.plugin")
and then for each you can getentrypoint.dist
, which seems to give enough info about the package.Maybe try this out in a branch of CMFPlone, because that is where
z3c.autoinclude
is currently loaded, forconfigure.zcml
andoverrides.zcml
.