search

aws / aws-cdk

The AWS Cloud Development Kit is a framework for defining cloud infrastructure in code
https://aws.amazon.com/cdk
Apache License 2.0
11.58k stars 3.88k forks source link

Fix the doc build in the `cdk-v2-main` pipeline #15845

Closed skinny85 closed 3 years ago

skinny85 commented 3 years ago

After #15722 was merged, the cdk-v2-main pipeline failed in the doc build:

Successfully downloaded aws-cdk.cx-api aws-cdk.cloud-assembly-schema aws-cdk.region-info aws-cdk-lib constructs jsii attrs cattrs publication typing-extensions python-dateutil six
Processing /codebuild/output/src550519960/src/python/aws_cdk.cx_api-2.0.0rc15-py3-none-any.whl
Processing /codebuild/output/src550519960/src/python/aws_cdk.region_info-2.0.0rc15-py3-none-any.whl
Processing /codebuild/output/src550519960/src/python/aws_cdk.cloud_assembly_schema-2.0.0rc15-py3-none-any.whl
Processing /codebuild/output/src550519960/src/python/aws_cdk_lib-2.0.0rc15-py3-none-any.whl
Installing collected packages: aws-cdk.region-info, aws-cdk-lib, aws-cdk.cx-api, aws-cdk.cloud-assembly-schema
Successfully installed aws-cdk-lib-2.0.0rc15 aws-cdk.cloud-assembly-schema-2.0.0rc15 aws-cdk.cx-api-2.0.0rc15 aws-cdk.region-info-2.0.0rc15
Processing ./deps/attrs-20.3.0-py2.py3-none-any.whl
Processing ./deps/aws_cdk_lib-2.0.0rc15-py3-none-any.whl
Processing ./deps/cattrs-1.6.0-py3-none-any.whl
Processing ./deps/constructs-10.0.5-py3-none-any.whl
Processing ./deps/jsii-1.32.0-py3-none-any.whl
Processing ./deps/publication-0.0.3-py2.py3-none-any.whl
Processing ./deps/python_dateutil-2.8.2-py2.py3-none-any.whl
Processing ./deps/six-1.16.0-py2.py3-none-any.whl
Processing ./deps/typing_extensions-3.10.0.0-py3-none-any.whl
aws-cdk-lib is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
typing-extensions is already installed with the same version as the provided wheel. Use --force-reinstall to force an installation of the wheel.
Installing collected packages: six, python-dateutil, publication, jsii, constructs, cattrs, attrs
Successfully installed attrs-20.3.0 cattrs-1.6.0 constructs-10.0.5 jsii-1.32.0 publication-0.0.3 python-dateutil-2.8.2 six-1.16.0
Documenting aws_cdk
Documenting aws_cdk.alexa_ask
Documenting aws_cdk.alpha
WARNING: The candidate selected for download or install is a yanked version: 'cattrs' candidate (version 1.6.0 at https://files.pythonhosted.org/packages/06/40/9a3f0e80af334bd808549c0b09fafe44269ddf261933f97723fd9b3f0deb/cattrs-1.6.0-py3-none-any.whl#sha256=c8de53900e3acad94ca83750eb12bb38aa85ce9114be47177c943e2f0eca63b0 (from https://pypi.org/simple/cattrs/) (requires-python:>=3.7,<4.0))
Reason for being yanked: Wrong dependency metadata
[30] Preparing doc index...
Traceback (most recent call last):
  File "/tmp/scriptdir/python/gen_doc_index.py", line 618, in <module>
    main()
  File "/tmp/scriptdir/python/gen_doc_index.py", line 64, in main
    packages_and_members = list(documentable_elements(all_jsii_modules()))
  File "/tmp/scriptdir/python/gen_doc_index.py", line 207, in documentable_elements
    for mod in modules:
  File "/tmp/scriptdir/python/gen_doc_index.py", line 257, in all_jsii_modules
    module = importlib.import_module(name)
  File "/usr/lib64/python3.7/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
  File "<frozen importlib._bootstrap>", line 983, in _find_and_load
  File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 728, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/tmp/python/venv/lib64/python3.7/site-packages/aws_cdk/alpha/aws_amplify/__init__.py", line 11, in <module>
    from .._jsii import *
ModuleNotFoundError: No module named 'aws_cdk.alpha._jsii'
---------- python FAILED after 34 seconds ----------

Looks like the @aws-cdk-alpha/* packages are somehow being picked up by the Python doc build, despite not producing an artifact from pack.

Perhaps it's something in the gen_doc_index.py file in the doc build for Python?

skinny85 commented 3 years ago

I have a suspicion this might be related to this comment from #15722, which resulted in changes to the jsii keys of the unstable packages on the v2-main branch, like these.

njlynch commented 3 years ago

All of the service module packages (both stable and experimental) include a from .._jsii import * import line within their __init__.py file.

This works for all of the stable services, because they are all in a flat directory structure alongside the _jsii module:

% ls -1 ./lib/python3.9/site-packages/aws_cdk | head
__init__.py
__pycache__
_jsii
alexa_ask
alpha
aws_accessanalyzer
aws_acmpca
aws_amazonmq
aws_apigateway
aws_appconfig

However, the alpha packages are all one level deeper (under the alpha directory), meaning the correct import statement would be from ..._jsii import * (note the extra .). This is indeed related to the changes to add a nested module name (e.g., "aws-cdk.alpha.aws-amplify") to the jsii packages.

Still tracking down where this import is being generated and if there is a way to customize it.

github-actions[bot] commented 3 years ago

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.