conda / conda-build

Commands and tools for building conda packages
https://docs.conda.io/projects/conda-build/
Other
381 stars 423 forks source link

conda-forge conda-build built a bad version into a package #2670

Closed kalefranz closed 1 year ago

kalefranz commented 6 years ago

Putting this here for reference. https://github.com/conda-forge/pyside2-feedstock/issues/14#issuecomment-362395773

Is this something we should handle with conda-verify? I'm honestly not sure how that ended up sneaking through conda-build. And it might not have snuck through with conda-build 3.

jakirkham commented 6 years ago

Have to say, am a bit surprised this isn't caught by trying to install the package in the test environment after the build. Is this some consequence of how the package is installed for testing?

msarahan commented 6 years ago

conda-verify might catch it, but conda-verify is still a dud. The 2.0 packages had an issue in setup.py that led to empty packages. Conda-verify has been a no-op for its entire lifetime. It is now fixed, but we need a new release, and we also need to flip conda-verify from being default-on to default-off, so that people's normally-working recipes don't suddenly all fail.

I am also surprised that installing the package for tests works at all. Do we have build logs for this? I don't see such a version in the commit history. https://github.com/conda-forge/pyside2-feedstock/commits/master/recipe/meta.yaml

jakirkham commented 6 years ago

The branch was never merged. However here is the PR ( https://github.com/conda-forge/pyside2-feedstock/pull/12 ). Should be able to find logs from there. If not, please let us know.

msarahan commented 6 years ago

Thanks for the link. Conda-build is quite happy to install this for testing:

TEST START: /feedstock_root/build_artefacts/linux-64/pyside2-2.0.0~alpha0-py36_0.tar.bz2
Deleting work directory, /feedstock_root/build_artefacts/pyside2_1517498193484/work

The following NEW packages will be INSTALLED:

    ca-certificates:  2017.11.5-0         conda-forge
    dbus:             1.10.22-0           conda-forge
    expat:            2.2.5-0             conda-forge
    fontconfig:       2.12.6-0            conda-forge
    freetype:         2.8.1-0             conda-forge
    gettext:          0.19.7-1            conda-forge
    glib:             2.55.0-0            conda-forge
    gst-plugins-base: 1.8.0-0             conda-forge
    gstreamer:        1.8.0-1             conda-forge
    icu:              58.2-0              conda-forge
    jpeg:             9b-2                conda-forge
    libffi:           3.2.1-3             conda-forge
    libgcc:           7.2.0-h69d50b8_2    defaults   
    libgcc-ng:        7.2.0-h7cc24e2_2    defaults   
    libiconv:         1.15-0              conda-forge
    libpng:           1.6.34-0            conda-forge
    libstdcxx-ng:     7.2.0-h7a57d05_2    defaults   
    libxcb:           1.12-1              conda-forge
    libxml2:          2.9.7-0             conda-forge
    libxslt:          1.1.32-0            conda-forge
    ncurses:          5.9-10              conda-forge
    openssl:          1.0.2n-0            conda-forge
    pcre:             8.39-0              conda-forge
    pyside2:          2.0.0~alpha0-py36_0 local      
    python:           3.6.4-0             conda-forge
    qt:               5.6.2-7             conda-forge
    readline:         7.0-0               conda-forge
    sqlite:           3.20.1-2            conda-forge
    tk:               8.6.7-0             conda-forge
    xorg-libxau:      1.0.8-3             conda-forge
    xorg-libxdmcp:    1.1.2-3             conda-forge
    xz:               5.2.3-0             conda-forge
    zlib:             1.2.11-0            conda-forge

dbus post-link :: /etc/machine-id not found ..
dbus post-link :: .. using /proc/sys/kernel/random/boot_id

WARNING: conda-build appears to be out of date. You have version 2.1.18 but the
latest version is 3.4.0. Run

conda update -n root conda-build

to get the latest version.

+ source /opt/conda/bin/activate /feedstock_root/build_artefacts/pyside2_1517498193484/_t_env
+ /feedstock_root/build_artefacts/pyside2_1517498193484/_t_env/bin/python -s /feedstock_root/build_artefacts/pyside2_1517498193484/test_tmp/run_test.py
import: 'PySide2.QtConcurrent'
import: 'PySide2.QtCore'
import: 'PySide2.QtGui'
import: 'PySide2.QtHelp'
import: 'PySide2.QtMultimedia'
import: 'PySide2.QtMultimediaWidgets'
import: 'PySide2.QtNetwork'
import: 'PySide2.QtOpenGL'
import: 'PySide2.QtPrintSupport'
import: 'PySide2.QtQml'
import: 'PySide2.QtQuick'
import: 'PySide2.QtQuickWidgets'
import: 'PySide2.QtScript'
import: 'PySide2.QtScriptTools'
import: 'PySide2.QtSql'
import: 'PySide2.QtSvg'
import: 'PySide2.QtTest'
import: 'PySide2.QtUiTools'
import: 'PySide2.QtWebChannel'
import: 'PySide2.QtWebSockets'
import: 'PySide2.QtWidgets'
import: 'PySide2.QtXml'
import: 'PySide2.QtXmlPatterns'
import: 'PySide2.QtConcurrent'
import: 'PySide2.QtCore'
import: 'PySide2.QtGui'
import: 'PySide2.QtHelp'
import: 'PySide2.QtMultimedia'
import: 'PySide2.QtMultimediaWidgets'
import: 'PySide2.QtNetwork'
import: 'PySide2.QtOpenGL'
import: 'PySide2.QtPrintSupport'
import: 'PySide2.QtQml'
import: 'PySide2.QtQuick'
import: 'PySide2.QtQuickWidgets'
import: 'PySide2.QtScript'
import: 'PySide2.QtScriptTools'
import: 'PySide2.QtSql'
import: 'PySide2.QtSvg'
import: 'PySide2.QtTest'
import: 'PySide2.QtUiTools'
import: 'PySide2.QtWebChannel'
import: 'PySide2.QtWebSockets'
import: 'PySide2.QtWidgets'
import: 'PySide2.QtXml'
import: 'PySide2.QtXmlPatterns'
+ /bin/bash -x -e /feedstock_root/build_artefacts/pyside2_1517498193484/test_tmp/run_test.sh
+ shiboken2 --help
Usage:
  shiboken [options] header-file typesystem-file

General options:
  --api-version=<"package mask">,<"version">Specify the supported api version used to generate the bindings
  --debug-level=[sparse|medium|full]      Set the debug level                   
  --documentation-only                    Do not generates any code, just the documentation
  --drop-type-entries="<TypeEntry0>[;TypeEntry1;...]"Semicolon separated list of type system entries (classes, namespaces, global functions and enums) to be dropped from generation.
  --generator-set=<"generator module">    generator-set to be used. e.g. qtdoc  
  --help                                  Display this help and exit            
  --include-paths=<path>[:<path>:...]     Include paths used by the C++ parser  
  --license-file=<license-file>           File used for copyright headers of generated files
  --no-suppress-warnings                  Show all warnings                     
  --output-directory=<path>               The directory where the generated files will be written
  --project-file=<file>                   text file containing a description of the binding project. Replaces and overrides command line arguments
  --silent                                Avoid printing any message            
  --typesystem-paths=<path>[:<path>:...]  Paths used when searching for typesystems
  --version                               Output version information and exit   

Shiboken options:
  --avoid-protected-hack                  Avoid the use of the '#define protected public' hack.
  --disable-verbose-error-messages        Disable verbose error messages. Turn the python code hard to debug but safe few kB on the generated bindings.
  --enable-parent-ctor-heuristic          Enable heuristics to detect parent relationship on constructors.
  --enable-pyside-extensions              Enable PySide extensions, such as support for signal/slots, use this if you are creating a binding for a Qt-based library.
  --enable-return-value-heuristic         Enable heuristics to detect parent relationship on return values (USE WITH CAUTION!)
  --use-isnull-as-nb_nonzero              If a class have an isNull()const method, it will be used to compute the value of boolean casts

QtDocGenerator options:
  --doc-parser                            The documentation parser used to interpret the documentation input files (qdoc3|doxygen)
  --documentation-code-snippets-dir       Directory used to search code snippets used by the documentation
  --documentation-data-dir                Directory with XML files generated by documentation tool (qdoc3 or Doxygen)
  --documentation-extra-sections-dir      Directory used to search for extra documentation sections
  --library-source-dir                    Directory where library source code is located
+ pyside2-lupdate -help
Usage:
    pyside2-lupdate [options] project-file
    pyside2-lupdate [options] source-files -ts ts-files
Options:
    -help  Display this information and exit
    -noobsolete
           Drop all obsolete strings
    -verbose
           Explain what is being done
    -version
           Display the version of pyside2-lupdate and exit
+ pyside2-rcc -help
PySide2 resource compiler
Usage: pyside2-rcc  [options] <inputs>

Options:
    -o file           Write output to file rather than stdout
    -py2              Generate code for any Python v2.x version
    -py3              Generate code for any Python v3.x version (default)
    -name name        Create an external initialization function with name
    -threshold level  Threshold to consider compressing files
    -compress level   Compress input files by level
    -root path        Prefix resource access path with root path
    -no-compress      Disable all compression
    -version          Display version
    -help             Display this information
+ pyside2-uic -help
Usage: pyside2-uic [options] <ui-file>

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -p, --preview         show a preview of the UI instead of generating code
  -o FILE, --output=FILE
                        write generated code to FILE instead of stdout
  -x, --execute         generate extra code to test and display the class
  -d, --debug           show debug output
  -i N, --indent=N      set indent width to N spaces, tab if N is 0 (default:
                        4)

  Code generation options:
    --from-imports      generate imports relative to '.'
+ shiboken2 --help
Usage:
  shiboken [options] header-file typesystem-file

General options:
  --api-version=<"package mask">,<"version">Specify the supported api version used to generate the bindings
  --debug-level=[sparse|medium|full]      Set the debug level                   
  --documentation-only                    Do not generates any code, just the documentation
  --drop-type-entries="<TypeEntry0>[;TypeEntry1;...]"Semicolon separated list of type system entries (classes, namespaces, global functions and enums) to be dropped from generation.
  --generator-set=<"generator module">    generator-set to be used. e.g. qtdoc  
  --help                                  Display this help and exit            
  --include-paths=<path>[:<path>:...]     Include paths used by the C++ parser  
  --license-file=<license-file>           File used for copyright headers of generated files
  --no-suppress-warnings                  Show all warnings                     
  --output-directory=<path>               The directory where the generated files will be written
  --project-file=<file>                   text file containing a description of the binding project. Replaces and overrides command line arguments
  --silent                                Avoid printing any message            
  --typesystem-paths=<path>[:<path>:...]  Paths used when searching for typesystems
  --version                               Output version information and exit   

Shiboken options:
  --avoid-protected-hack                  Avoid the use of the '#define protected public' hack.
  --disable-verbose-error-messages        Disable verbose error messages. Turn the python code hard to debug but safe few kB on the generated bindings.
  --enable-parent-ctor-heuristic          Enable heuristics to detect parent relationship on constructors.
  --enable-pyside-extensions              Enable PySide extensions, such as support for signal/slots, use this if you are creating a binding for a Qt-based library.
  --enable-return-value-heuristic         Enable heuristics to detect parent relationship on return values (USE WITH CAUTION!)
  --use-isnull-as-nb_nonzero              If a class have an isNull()const method, it will be used to compute the value of boolean casts

QtDocGenerator options:
  --doc-parser                            The documentation parser used to interpret the documentation input files (qdoc3|doxygen)
  --documentation-code-snippets-dir       Directory used to search code snippets used by the documentation
  --documentation-data-dir                Directory with XML files generated by documentation tool (qdoc3 or Doxygen)
  --documentation-extra-sections-dir      Directory used to search for extra documentation sections
  --library-source-dir                    Directory where library source code is located
+ pyside2-lupdate -help
Usage:
    pyside2-lupdate [options] project-file
    pyside2-lupdate [options] source-files -ts ts-files
Options:
    -help  Display this information and exit
    -noobsolete
           Drop all obsolete strings
    -verbose
           Explain what is being done
    -version
           Display the version of pyside2-lupdate and exit
+ pyside2-rcc -help
PySide2 resource compiler
Usage: pyside2-rcc  [options] <inputs>

Options:
    -o file           Write output to file rather than stdout
    -py2              Generate code for any Python v2.x version
    -py3              Generate code for any Python v3.x version (default)
    -name name        Create an external initialization function with name
    -threshold level  Threshold to consider compressing files
    -compress level   Compress input files by level
    -root path        Prefix resource access path with root path
    -no-compress      Disable all compression
    -version          Display version
    -help             Display this information
+ pyside2-uic -help
Usage: pyside2-uic [options] <ui-file>

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -p, --preview         show a preview of the UI instead of generating code
  -o FILE, --output=FILE
                        write generated code to FILE instead of stdout
  -x, --execute         generate extra code to test and display the class
  -d, --debug           show debug output
  -i N, --indent=N      set indent width to N spaces, tab if N is 0 (default:
                        4)

  Code generation options:
    --from-imports      generate imports relative to '.'
TEST END: /feedstock_root/build_artefacts/linux-64/pyside2-2.0.0~alpha0-py36_0.tar.bz2
INFO:conda_build.config:--dirty flag and --keep-old-work not specified.Removing build/test folder after successful build/test.

IMHO, no easy answer here. We need to get conda-verify operational.

isuruf commented 6 years ago

Is there a python function in conda or conda-build that we can use to verify that a version string is conda compliant?

kalefranz commented 6 years ago

Looks like MatchSpec is in conda.exports, right now, and that's probably what I'm most comfortable with using here. So if something like ms = MatchSpec(version='1.2.3') succeeds without error, you should be good to go.

isuruf commented 6 years ago

Thanks @kalefranz, is there an equivalent in conda 4.3?

msarahan commented 6 years ago

MatchSpec has been there quite a while, I think we're OK to use it irrespective of conda version

isuruf commented 6 years ago
In [10]: conda.__version__
Out[10]: '4.3.34'

In [11]: ms = MatchSpec(version='1.2.3')
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-11-893362789212> in <module>()
----> 1 ms = MatchSpec(version='1.2.3')

TypeError: __new__() got an unexpected keyword argument 'version'

In [12]: ?MatchSpec
Init signature: MatchSpec(spec, target=Ellipsis, optional=Ellipsis, normalize=False)
Docstring:      <no docstring>
File:           ~/miniconda3/lib/python3.6/site-packages/conda/resolve.py
Type:           type
kalefranz commented 6 years ago

To be compatible with conda 4.3, you should be able to use something like

MatchSpec('%s=%s' % (name, version))
kalefranz commented 6 years ago

I'm wrong. That succeeds when it shouldn't.

>>> MatchSpec("conda=4.3~~post1")
MatchSpec('conda=4.3~~post1')

Maybe that's actually the root cause of the issue here...

Edit: I should clarify that it succeeds in 4.3. It correctly fails in 4.4.

kalefranz commented 6 years ago

Use conda.export.VersionOrder

kfranz@0283:~/continuum/conda 4.3.x ❯ python
Python 3.5.2 (default, Sep 15 2016, 07:38:42)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from conda.exports import VersionOrder
>>> VersionOrder('4.3~~post1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kfranz/continuum/conda/conda/version.py", line 161, in __new__
    raise CondaValueError(message + "invalid character(s).")
conda.exceptions.CondaValueError: Malformed version string '4.3~~post1': invalid character(s).
>>> ^D
kfranz@0283:~/continuum/conda 4.3.x ❯ git checkout 4.4.x
warning: unable to rmdir tests/conda_env/support/advanced-pip/src/argh: Directory not empty
Switched to branch '4.4.x'
Your branch is up-to-date with 'origin/4.4.x'.
kfranz@0283:~/continuum/conda 4.4.x ❯ python
Python 3.5.2 (default, Sep 15 2016, 07:38:42)
[GCC 4.2.1 Compatible Apple LLVM 7.3.0 (clang-703.0.31)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from conda.exports import VersionOrder
>>> VersionOrder('4.3~~post1')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/kfranz/continuum/conda/conda/models/version.py", line 163, in __new__
    raise CondaValueError(message + "invalid character(s).")
conda.exceptions.CondaValueError: Malformed version string '4.3~~post1': invalid character(s).
isuruf commented 6 years ago

Looks like conda now allows -s in the version string as long as there is no _. conda-build chokes on this though. See https://github.com/conda-forge/conda-smithy/issues/779

github-actions[bot] commented 2 years ago

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

  1. Verify that you can still reproduce the issue at hand
  2. Comment that the issue is still reproducible and include:
    • What OS and version you reproduced the issue on
    • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!