prefix-dev / rattler-build

rattler-build is a universal package builder for Windows, macOS and Linux
https://prefix-dev.github.io/rattler-build
BSD 3-Clause "New" or "Revised" License
185 stars 38 forks source link

rattler_build creates conda packages that are ~10^6 times bigger than they should be #981

Open sg-s opened 1 month ago

sg-s commented 1 month ago

clearly i'm doing something wrong here, but i can't see what it is.

this is the content of my recipe.yaml file:

context:
  version: "1.1.0"
  name: "humanize"

package:
  name: ${{ name }}
  version: ${{ version }}

source:
  - url: https://files.pythonhosted.org/packages/91/f4/916fc73fc4c7a94ea5bc1ab993936d9b1220b8e6299e5878a22e092b7af9/humanize-1.1.0-py3-none-any.whl
    sha256: 3959c76f1f6c9b43258bb0b0f6b6161d6c07b8717b306c0b713ea36b7999331e

build:
  noarch: python
  script:
    - python -m pip install humanize-1.1.0-py3-none-any.whl -vv --no-deps --ignore-installed

requirements:
  host:
    - pip
    - setuptools
    - python 3.12
  run:
    - python

tests:
  - python:
      imports:
        - humanize
      pip_check: true

the size of humanize-1.1.0-py3-none-any.whl is around 38kB (see https://pypi.org/project/humanize/1.1.0/#files)

however, the size of the conda file that rattler-build makes is 103MB!!!!

looking into the conda file, i find two .zst files:

drwx------@ 5 srinivas  staff   160B Jul 22 15:39 .
drwxr-xr-x  6 srinivas  staff   192B Jul 22 15:39 ..
-rw-r--r--  1 srinivas  staff    98M Jul 22  2024 info-humanize-1.1.0-pyh4616a5c_0.tar.zst
-rw-r--r--  1 srinivas  staff    30B Jul 22  2024 metadata.json
-rw-r--r--  1 srinivas  staff    24K Jul 22  2024 pkg-humanize-1.1.0-pyh4616a5c_0.tar.zst

the info folder is 200MB (!!!) unzipped

to get the behavior i desire (which is not to make giant conda packages), i need to do:

rattler-build build --no-include-recipe

i find it strange that this is not the default, and furthermore do not understand why anyone would ever want this behaviour

wolfv commented 1 month ago

I suspect that you are running the build from inside the recipe folder which means that the ./output folder is entirely included in package.

This has come up a number of times and we need to find a fix for it. You can easily work around this by building outside the recipe folder or by setting a different output directory explicitly.

sg-s commented 1 month ago

i am! thank you for the diagnosis.