conda / constructor

tool for creating installers from conda packages
https://conda.github.io/constructor/
Other
457 stars 168 forks source link

`info.json` only contains `pkg` information for `installer_type: all` #814

Open marcoesters opened 3 months ago

marcoesters commented 3 months ago

Checklist

What happened?

The output files have generic names like info.json. When creating both pkg and sh installers, constructor does not distinguish between the two when creating output files so that info.json is overwritten output information on the sh installer.

It is of course possible to just call them info.sh.json and info.pkg.json, but I propose a different naming scheme: <installer name>_info.json. While this will make file names longer, it makes file names unique and directly tied to the installer and simplifies the work of users: if multiple installers are built into one directory or uploaded to a GitHub release, the user will not have to do any sort of renaming to make sure files are not overwritten.

An alternative is to output the information of both installer types into the info.json file.

Conda Info

No response

Conda Config

No response

Conda list

# packages in environment at /Users/mesters/opt/miniconda3/envs/constructor:
#
# Name                    Version                   Build  Channel
archspec                  0.2.3              pyhd3eb1b0_0    defaults
boltons                   23.0.0          py312hca03da5_0    defaults
brotli-python             1.0.9           py312h313beb8_8    defaults
bzip2                     1.0.8                h80987f9_6    defaults
c-ares                    1.19.1               h80987f9_0    defaults
ca-certificates           2024.3.11            hca03da5_0    defaults
certifi                   2024.6.2        py312hca03da5_0    defaults
cffi                      1.16.0          py312h80987f9_1    defaults
charset-normalizer        2.0.4              pyhd3eb1b0_0    defaults
conda                     24.5.0          py312hca03da5_0    defaults
conda-libmamba-solver     24.1.0             pyhd3eb1b0_0    defaults
conda-package-handling    2.3.0           py312hca03da5_0    defaults
conda-package-streaming   0.10.0          py312hca03da5_0    defaults
conda-standalone          24.5.0               hca03da5_0    defaults
constructor               3.8.0           py312hca03da5_0    defaults
distro                    1.9.0           py312hca03da5_0    defaults
expat                     2.6.2                h313beb8_0    defaults
fmt                       9.1.0                h48ca7d4_1    defaults
freetype                  2.12.1               h1192e45_0    defaults
frozendict                2.4.2           py312hca03da5_0    defaults
icu                       73.1                 h313beb8_0    defaults
idna                      3.7             py312hca03da5_0    defaults
jinja2                    3.1.4           py312hca03da5_0    defaults
jpeg                      9e                   h80987f9_1    defaults
jsonpatch                 1.33            py312hca03da5_1    defaults
jsonpointer               2.1                pyhd3eb1b0_0    defaults
krb5                      1.20.1               hf3e1bf2_1    defaults
lcms2                     2.12                 hba8e193_0    defaults
lerc                      3.0                  hc377ac9_0    defaults
libarchive                3.6.2                h62fee54_3    defaults
libcurl                   8.7.1                h3e2b118_0    defaults
libcxx                    14.0.6               h848a8c0_0    defaults
libdeflate                1.17                 h80987f9_1    defaults
libedit                   3.1.20230828         h80987f9_0    defaults
libev                     4.33                 h1a28f6b_1    defaults
libffi                    3.4.4                hca03da5_1    defaults
libiconv                  1.16                 h80987f9_3    defaults
libmamba                  1.5.8                haeffa04_2    defaults
libmambapy                1.5.8           py312h1c5506f_2    defaults
libnghttp2                1.57.0               h62f6fdd_0    defaults
libpng                    1.6.39               h80987f9_0    defaults
libsolv                   0.7.24               h514c7bf_1    defaults
libssh2                   1.11.0               h3e2b118_0    defaults
libtiff                   4.5.1                h313beb8_0    defaults
libwebp-base              1.3.2                h80987f9_0    defaults
libxml2                   2.10.4               h0b34f26_2    defaults
lz4-c                     1.9.4                h313beb8_1    defaults
markupsafe                2.1.3           py312h80987f9_0    defaults
menuinst                  2.1.1           py312hca03da5_0    defaults
ncurses                   6.4                  h313beb8_0    defaults
openjpeg                  2.3.0                h7a6adac_2    defaults
openssl                   3.0.14               h80987f9_0    defaults
packaging                 23.2            py312hca03da5_0    defaults
pcre2                     10.42                hb066dcc_1    defaults
pillow                    10.3.0          py312h80987f9_0    defaults
pip                       24.0            py312hca03da5_0    defaults
platformdirs              3.10.0          py312hca03da5_0    defaults
pluggy                    1.0.0           py312hca03da5_1    defaults
pybind11-abi              5                    hd3eb1b0_0    defaults
pycosat                   0.6.6           py312h80987f9_1    defaults
pycparser                 2.21               pyhd3eb1b0_0    defaults
pysocks                   1.7.1           py312hca03da5_0    defaults
python                    3.12.4               h99e199e_1    defaults
readline                  8.2                  h1a28f6b_0    defaults
reproc                    14.2.4               h313beb8_2    defaults
reproc-cpp                14.2.4               h313beb8_2    defaults
requests                  2.32.2          py312hca03da5_0    defaults
ruamel.yaml               0.17.21         py312h80987f9_0    defaults
setuptools                69.5.1          py312hca03da5_0    defaults
sqlite                    3.45.3               h80987f9_0    defaults
tk                        8.6.14               h6ba3021_0    defaults
tqdm                      4.66.4          py312h989b03a_0    defaults
truststore                0.8.0           py312hca03da5_0    defaults
tzdata                    2024a                h04d1e81_0    defaults
urllib3                   2.2.2           py312hca03da5_0    defaults
wheel                     0.43.0          py312hca03da5_0    defaults
xz                        5.4.6                h80987f9_1    defaults
yaml-cpp                  0.8.0                h313beb8_1    defaults
zlib                      1.2.13               h18a0788_1    defaults
zstandard                 0.22.0          py312h1a4646a_0    defaults
zstd                      1.5.5                hd90d995_2    defaults

Additional Context

No response

marcoesters commented 3 months ago

Looking more into the code, it looks like only one output is generated. However, the output only contains the information of the last installer that was created. So, I'm wondering if this should be changed to create one output per installer file or if the info.json object needs to be improved to contain the information of all installers that were created.

I updated the initial post to reflect that.

jaimergp commented 3 months ago

Even if we have two installers generated, so far the info.json contents would have been the same (both installers bundle the same packages), but if we are going to add more output types, maybe we do need to disambiguate. I'd stick to something predictable like installer-version.ext.json?

marcoesters commented 3 months ago

Most information is the same, but not all. I am playing with merging the information right now to something like this:

    # Merge info files for each installer type
    if len(itypes) > 1:
        keys = set()
        for info_dict in info_dicts:
            keys.update(info_dict.keys())
        for key in keys:
            if any(info_dict.get(key) != info.get(key) for info_dict in info_dicts):
                info[key] = [info_dict.get(key, "") for info_dict in info_dicts]

For now, this could be more feasible than creating two largely identical files.