search

litex-hub / conda-build-prepare

Apache License 2.0
2 stars 1 forks source link

subpackages should be supported #7

Open proppy opened 2 years ago

proppy commented 2 years ago

It seems that the current implementation strips the outputs section when rendering meta.yaml thus preventing the tool to "prepare" recipe with subpackages.

Example:

# Rendered by conda-build-prepare
# Original meta.yaml can be found at the end of this file

package:
  name: open_pdks.sky130a
  version: 1.0.329_0_gf245a7a
source:
- git_rev: master
  git_url: /tmp/conda-gf180/git-repos/open_pdks
  patches:
  - disable-link-targets.patch
  - use-sources-from-conda.patch
- folder: skywater-pdk
  url: https://github.com/google/skywater-pdk/archive/refs/heads/main.zip
- folder: skywater-pdk/libraries/sky130_fd_sc_hd/latest
  url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hd/archive/refs/heads/branch-0.0.2.zip
- folder: skywater-pdk/libraries/sky130_fd_sc_hvl/latest
  url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hvl/archive/refs/heads/branch-0.0.3.zip
- folder: skywater-pdk/libraries/sky130_fd_io/latest
  url: https://github.com/google/skywater-pdk-libs-sky130_fd_io/archive/refs/heads/branch-0.2.1.zip
- folder: skywater-pdk/libraries/sky130_fd_pr/latest
  url: https://github.com/google/skywater-pdk-libs-sky130_fd_pr/archive/refs/heads/branch-0.20.1.zip
- sha256: 1d7f3a284a49d350ddbabde0e7d0c5ffa34a144aaf1bcb5b9f2c87673ff0c76e
  url: https://files.pythonhosted.org/packages/07/1f/3d9ae865addc9ef6cb7b102d7d93e227c46b6e5e94db345cae2a30944efa/dataclasses_json-0.5.6-py3-none-any.whl
- folder: gf180mcu-pdk/gf180mcu_fd_pr
  git_depth: '1'
  git_rev: main
  git_url: /tmp/conda-gf180/git-repos/globalfoundries-pdk-libs-gf180mcu_fd_pr
- folder: gf180mcu-pdk/gf180mcu_fd_io
  git_depth: '1'
  git_rev: main
  git_url: /tmp/conda-gf180/git-repos/globalfoundries-pdk-libs-gf180mcu_fd_io
- folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu7t5v0
  git_depth: '1'
  git_rev: main
  git_url: /tmp/conda-gf180/git-repos/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu7t5v0
- folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu9t5v0
  git_depth: '1'
  git_rev: main
  git_url: /tmp/conda-gf180/git-repos/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu9t5v0
- folder: gf180mcu-pdk/gf180mcu_fd_ip_sram
  git_depth: '1'
  git_rev: main
  git_url: /tmp/conda-gf180/git-repos/globalfoundries-pdk-ip-gf180mcu_fd_ip_sram
build:
  noarch: false
  noarch_python: false
  number: '20220908014309'
  script: build_sky130.sh
  script_env: []
  string: '20220908_014309'
requirements:
  build:
  - autoconf
  - automake
  - make
  host:
  - python 3.7.*
  - pip
  - typing_inspect
  - marshmallow
  - marshmallow-enum
  - magic
  run_constrained:
  - magic
test:
  commands:
  - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_io
  - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_pr
  - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hd
  - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hvl
about:
  home: http://opencircuitdesign.com/open_pdks/
  license: Apache-2.0
  summary: PDK installer for open-source EDA tools and toolchains. Distributed with
    a setup for the Google/SkyWater 130nm process.
extra:
  build_type: local
  final: true
  parent_recipe:
    name: open_pdks
    path: /tmp/conda-gf180/recipe
    version: 1.0.329_0_gf245a7a
  recipe-maintainers:
  - proppy
  recipe_source:
    branch: gf180
    commit: 004d21b47706e4d198fef4d977bf4d54cc5966c0
    date: '20220908_090957'
    describe: v0.0-1329-g004d21b
    repo: https://github.com/hdl/conda-eda

# Original meta.yaml:
#
# # Use `conda-build-prepare` before building for a better version string.
# {% set version = '%s_%04i_%s'| format(GIT_DESCRIBE_TAG | default('0.X'), GIT_DESCRIBE_NUMBER | default(0) | int, GIT_DESCRIBE_HASH | default('gUNKNOWN')) %}
# 
# package:
#   name: open_pdks
#   version: {{ version }}
# 
# source:
#   - git_url: https://github.com/RTimothyEdwards/open_pdks.git
#     git_rev: master
#     patches:
#       - disable-link-targets.patch
#       - use-sources-from-conda.patch
#   - url: https://github.com/google/skywater-pdk/archive/refs/heads/main.zip
#     folder: skywater-pdk
#   - url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hd/archive/refs/heads/branch-0.0.2.zip
#     folder: skywater-pdk/libraries/sky130_fd_sc_hd/latest
#   - url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hvl/archive/refs/heads/branch-0.0.3.zip
#     folder: skywater-pdk/libraries/sky130_fd_sc_hvl/latest
#   - url: https://github.com/google/skywater-pdk-libs-sky130_fd_io/archive/refs/heads/branch-0.2.1.zip
#     folder: skywater-pdk/libraries/sky130_fd_io/latest
#   - url: https://github.com/google/skywater-pdk-libs-sky130_fd_pr/archive/refs/heads/branch-0.20.1.zip
#     folder: skywater-pdk/libraries/sky130_fd_pr/latest
#   - url: https://files.pythonhosted.org/packages/07/1f/3d9ae865addc9ef6cb7b102d7d93e227c46b6e5e94db345cae2a30944efa/dataclasses_json-0.5.6-py3-none-any.whl
#     sha256: 1d7f3a284a49d350ddbabde0e7d0c5ffa34a144aaf1bcb5b9f2c87673ff0c76e
#   - git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr.git
#     git_rev: main
#     folder: gf180mcu-pdk/gf180mcu_fd_pr
#     git_depth: 1
#   - git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_io.git
#     git_rev: main
#     git_depth: 1
#     folder: gf180mcu-pdk/gf180mcu_fd_io
#   - git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu7t5v0.git
#     git_rev: main
#     git_depth: 1
#     folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu7t5v0
#   - git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu9t5v0.git
#     git_rev: main
#     git_depth: 1
#     folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu9t5v0
#   - git_url: https://github.com/google/globalfoundries-pdk-ip-gf180mcu_fd_ip_sram.git
#     git_rev: main
#     folder: gf180mcu-pdk/gf180mcu_fd_ip_sram
#     git_depth: 1
# 
# build:
#   noarch: generic
#   # number: 202202031935
#   number: {{ environ.get('DATE_NUM') }}
#   # string: 20220203_1935
#   string: {{ environ.get('DATE_STR') }}
#   script_env:
#     - CI
# 
# outputs:
#   - name: open_pdks.sky130a
#     requirements:
#       build:
#         - autoconf
#         - automake
#         - make
#       host:
#         - python
#         - pip
#         - typing_inspect
#         - marshmallow
#         - marshmallow-enum
#         - magic
#       run_constrained:
#         - {{ pin_compatible('magic', min_pin='x.x.x', max_pin='x.x.x') }}
#     script: build_sky130.sh
#     files:
#       - $PREFIX/share/pdk/sky130A
#     test:
#       commands:
#         - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hd
#         - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hvl
#         - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_io
#         - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_pr
#   - name: open_pdks.gf180mcuc
#     requirements:
#       build:
#         - autoconf
#         - automake
#         - make
#       host:
#         - python
#         - pip
#         - magic
#       run_constrained:
#         - {{ pin_compatible('magic', min_pin='x.x.x', max_pin='x.x.x') }}
#     script: build_gf180.sh
#     files:
#       - $PREFIX/share/pdk/gf180mcu
#     test:
#       commands:
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_io
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_sc_mcu7t5v0
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_sc_mcu9t5v0
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_ip_sram
#         # from gf180mcu_fd_pr
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/ngspice
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/xyce
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/klayout
#         - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/xschem
# 
# about:
#   home: http://opencircuitdesign.com/open_pdks/
#   license: Apache-2.0
#   summary: 'PDK installer for open-source EDA tools and toolchains. Distributed with a setup for the Google/SkyWater 130nm process.'
# 
# extra:
#   recipe-maintainers:
#     - proppy

For some reason the name of the package also get "promoted" to the one of the first of the outputs.

proppy commented 2 years ago

ah this seems to comes from the interface with conda render as it creates one recipe for each output:

(base) proppy@proppy1:~/src/github.com/hdl/conda-eda$ DATE_NUM=0 DATE_STR=0 conda render misc/open_pdks/meta.yaml 
DATE_NUM=0 DATE_STR=0 conda render misc/open_pdks/meta.yaml 
Patch level ambiguous, selecting least deep
Patch level ambiguous, selecting least deep
Patch analysis gives:
[[ RA-MD1-OVE ]] - [[                                              disable-link-targets.patch ]]
[[ RA-MD1-OVE ]] - [[                                            use-sources-from-conda.patch ]]

Key:

R :: Reversible                       A :: Applicable
Y :: Build-prefix patch in use        M :: Minimal, non-amalgamated
D :: Dry-runnable                     N :: Patch level (1 is preferred)
L :: Patch level not-ambiguous        O :: Patch applies without offsets
V :: Patch applies without fuzz       E :: Patch applies without emitting to stderr

No hash (md5, sha1, sha256) provided for main.zip.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided for main.zip.  Source download forced.  Add hash to recipe to use source cache.
No hash (md5, sha1, sha256) provided for branch-0.0.2.zip.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided for branch-0.0.2.zip.  Source download forced.  Add hash to recipe to use source cache.
No hash (md5, sha1, sha256) provided for branch-0.0.3.zip.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided for branch-0.0.3.zip.  Source download forced.  Add hash to recipe to use source cache.
No hash (md5, sha1, sha256) provided for branch-0.2.1.zip.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided for branch-0.2.1.zip.  Source download forced.  Add hash to recipe to use source cache.
No hash (md5, sha1, sha256) provided for branch-0.20.1.zip.  Source download forced.  Add hash to recipe to use source cache.
WARNING:conda_build.source:No hash (md5, sha1, sha256) provided for branch-0.20.1.zip.  Source download forced.  Add hash to recipe to use source cache.
/home/proppy/src/github.com/hdl/conda-eda/env/lib/python3.9/site-packages/conda_build/environ.py:438: UserWarning: The environment variable 'CI' is undefined.
  warnings.warn(
Could not finalize metadata due to missing dependencies: ['magic']
WARNING:conda_build.metadata:Could not finalize metadata due to missing dependencies: ['magic']
--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
  name: open_pdks.gf180mcuc
  version: 1.0.329_0000_gf245a7a
source:
  - git_rev: master
    git_url: https://github.com/RTimothyEdwards/open_pdks.git
    patches:
      - disable-link-targets.patch
      - use-sources-from-conda.patch
  - folder: skywater-pdk
    url: https://github.com/google/skywater-pdk/archive/refs/heads/main.zip
  - folder: skywater-pdk/libraries/sky130_fd_sc_hd/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hd/archive/refs/heads/branch-0.0.2.zip
  - folder: skywater-pdk/libraries/sky130_fd_sc_hvl/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hvl/archive/refs/heads/branch-0.0.3.zip
  - folder: skywater-pdk/libraries/sky130_fd_io/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_io/archive/refs/heads/branch-0.2.1.zip
  - folder: skywater-pdk/libraries/sky130_fd_pr/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_pr/archive/refs/heads/branch-0.20.1.zip
  - sha256: 1d7f3a284a49d350ddbabde0e7d0c5ffa34a144aaf1bcb5b9f2c87673ff0c76e
    url: https://files.pythonhosted.org/packages/07/1f/3d9ae865addc9ef6cb7b102d7d93e227c46b6e5e94db345cae2a30944efa/dataclasses_json-0.5.6-py3-none-any.whl
  - folder: gf180mcu-pdk/gf180mcu_fd_pr
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr.git
  - folder: gf180mcu-pdk/gf180mcu_fd_io
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_io.git
  - folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu7t5v0
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu7t5v0.git
  - folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu9t5v0
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu9t5v0.git
  - folder: gf180mcu-pdk/gf180mcu_fd_ip_sram
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-ip-gf180mcu_fd_ip_sram.git
build:
  noarch: false
  noarch_python: false
  number: '0'
  script: build_gf180.sh
  script_env:
    - CI
  string: '0'
requirements:
  build:
    - autoconf
    - automake
    - make
  host:
    - python 3.9.*
    - pip
    - magic
  run_constrained:
    - magic
test:
  commands:
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_io
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_ip_sram
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_sc_mcu7t5v0
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.ref/gf180mcu_fd_sc_mcu9t5v0
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/klayout
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/ngspice
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/xschem
    - test -d $PREFIX/share/pdk/gf180mcuC/libs.tech/xyce
about:
  home: http://opencircuitdesign.com/open_pdks/
  license: Apache-2.0
  summary: PDK installer for open-source EDA tools and toolchains. Distributed with
    a setup for the Google/SkyWater 130nm process.
extra:
  final: true
  parent_recipe:
    name: open_pdks
    path: /home/proppy/src/github.com/hdl/conda-eda/misc/open_pdks
    version: 1.0.329_0000_gf245a7a
  recipe-maintainers:
    - proppy

--------------
Hash contents:
--------------
{}
----------
meta.yaml:
----------
package:
  name: open_pdks.sky130a
  version: 1.0.329_0000_gf245a7a
source:
  - git_rev: master
    git_url: https://github.com/RTimothyEdwards/open_pdks.git
    patches:
      - disable-link-targets.patch
      - use-sources-from-conda.patch
  - folder: skywater-pdk
    url: https://github.com/google/skywater-pdk/archive/refs/heads/main.zip
  - folder: skywater-pdk/libraries/sky130_fd_sc_hd/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hd/archive/refs/heads/branch-0.0.2.zip
  - folder: skywater-pdk/libraries/sky130_fd_sc_hvl/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_sc_hvl/archive/refs/heads/branch-0.0.3.zip
  - folder: skywater-pdk/libraries/sky130_fd_io/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_io/archive/refs/heads/branch-0.2.1.zip
  - folder: skywater-pdk/libraries/sky130_fd_pr/latest
    url: https://github.com/google/skywater-pdk-libs-sky130_fd_pr/archive/refs/heads/branch-0.20.1.zip
  - sha256: 1d7f3a284a49d350ddbabde0e7d0c5ffa34a144aaf1bcb5b9f2c87673ff0c76e
    url: https://files.pythonhosted.org/packages/07/1f/3d9ae865addc9ef6cb7b102d7d93e227c46b6e5e94db345cae2a30944efa/dataclasses_json-0.5.6-py3-none-any.whl
  - folder: gf180mcu-pdk/gf180mcu_fd_pr
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_pr.git
  - folder: gf180mcu-pdk/gf180mcu_fd_io
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_io.git
  - folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu7t5v0
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu7t5v0.git
  - folder: gf180mcu-pdk/gf180mcu_fd_sc_mcu9t5v0
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-libs-gf180mcu_fd_sc_mcu9t5v0.git
  - folder: gf180mcu-pdk/gf180mcu_fd_ip_sram
    git_depth: '1'
    git_rev: main
    git_url: https://github.com/google/globalfoundries-pdk-ip-gf180mcu_fd_ip_sram.git
build:
  noarch: false
  noarch_python: false
  number: '0'
  script: build_sky130.sh
  script_env:
    - CI
  string: '0'
requirements:
  build:
    - autoconf
    - automake
    - make
  host:
    - python 3.9.*
    - pip
    - typing_inspect
    - marshmallow
    - marshmallow-enum
    - magic
  run_constrained:
    - magic
test:
  commands:
    - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_io
    - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_pr
    - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hd
    - test -d $PREFIX/share/pdk/sky130A/libs.ref/sky130_fd_sc_hvl
about:
  home: http://opencircuitdesign.com/open_pdks/
  license: Apache-2.0
  summary: PDK installer for open-source EDA tools and toolchains. Distributed with
    a setup for the Google/SkyWater 130nm process.
extra:
  final: true
  parent_recipe:
    name: open_pdks
    path: /home/proppy/src/github.com/hdl/conda-eda/misc/open_pdks
    version: 1.0.329_0000_gf245a7a
  recipe-maintainers:
    - proppy
ajelinski commented 2 years ago

I can see it's a problem indeed. Currently, conda render -f is used to save the rendered recipe and it chooses to save just a single subpackage's recipe in the file:

WARNING:conda_build.cli.main_render:Multiple variants rendered. Only one will be written to the file you specified (meta.yaml).

I need to think more on how to properly handle subpackages because multiple meta.yaml files can't be supplied to conda build...