Closed wblumberg closed 4 years ago
Bug fix:
Status update for today:
Attempted to deploy v1.4.0a4 last night with @tsupinie . Seems the entire release pipeline went to hell:
py26_64_appveyor.txt py36_64_appveyor.txt
Ws-MacBook-Pro:conda-recipe blumberg$ conda build .
No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
WARNING:conda_build.metadata:No numpy version specified in conda_build_config.yaml. Falling back to default numpy value of 1.11
Cloning into '/Users/blumberg/conda-bld/sharppy_1547684816468/work'...
done.
Checking out files: 100% (2249/2249), done.
checkout: 'andover'
Checking out files: 100% (837/837), done.
Branch 'andover' set up to track remote branch 'andover' from 'origin'.
Switched to a new branch 'andover'
==> git log -n1 <==
commit 5b1a3ecea303e7e3e6bc97c38a45e676d187feee
Author: Greg Blumberg <wblumberg@ou.edu>
Date: Wed Jan 16 02:01:47 2019 -0500
trying to fix deployment pipeline
==> git describe --tags --dirty <==
v1.4.0a4
==> git status <==
On branch andover
Your branch is up to date with 'origin/andover'.
nothing to commit, working tree clean
Adding in variants from internal_defaults
INFO:conda_build.variants:Adding in variants from internal_defaults
/Users/blumberg/anaconda3/lib/python3.6/site-packages/conda_build/environ.py:422: UserWarning: The environment variable 'CONDA_BLD_PATH' is being passed through with value /Users/blumberg/conda-bld. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
UserWarning
/Users/blumberg/anaconda3/lib/python3.6/site-packages/conda_build/environ.py:422: UserWarning: The environment variable 'CONDA_BLD_PATH' is being passed through with value /Users/blumberg/conda-bld. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
UserWarning
/Users/blumberg/anaconda3/lib/python3.6/site-packages/conda_build/environ.py:422: UserWarning: The environment variable 'CONDA_BLD_PATH' is being passed through with value /Users/blumberg/conda-bld. If you are splitting build and test phases with --no-test, please ensure that this value is also set similarly at test time.
UserWarning
Received dictionary as spec. Note that pip requirements are not supported in conda-build meta.yaml.
Ws-MacBook-Pro:conda-recipe blumberg$
The reason for this issue is unclear. This link: https://github.com/bioconda/bioconda-utils/issues/157, seems to suggest that it's due to a pip requirement being used in the meta.yaml file, but there's no reference to pip in conda-recipes/meta.yaml. The deployment worked previously, so I'm not sure what's going on. Might need to look at the versions of conda that are being used on the CI services.
Possible Solutions:
Tried this last night in: 5b1a3ecea303e7e3e6bc97c38a45e676d187feee
I tried to remove some of the packages shown in the install_requires list in setup.py. This didn't fix the Conda-build issue.
Moved the deployment to PyPI in .travis.yml to be the last step. This way if it the deployment fails in any previous step (e.g. conda), we don't deploy to PyPI.
"Turned off" deployment to PyPI by forcing it to only deploy if it's on the "andover-off" branch. This branch doesn't exist, so deployment to PyPI will not occur anymore. This should help with testing.
Tried to load sphinx using conda install
. It was slow. The problem may have been that my command wasn't correct. I changed ci/build_docs.sh to reflect the new command from conda install -q -c sphinx sphinx-gallery
to conda install -q -c anaconda sphinx sphinx_rtd_theme
and conda install -q -c conda-forge sphinx-gallery
. Haven't tested this yet though. It may solve the docs deployment step - see 29f070038d8265efde20219eef318470b2050980 for the commit in which I did this.
Tried to add in requests and dateutil.parser into the .spec file for macOS. The binary built locally on my machine, but it will not load - it looks like it's having problems with the requests/python-dateutil dependencies. See sharppy-out.txt for the results from the binary. Here is the commit: 1b33954f247341e3b793731c3a31ba71091e78bb.
Got the conda-build working again on my machine...see d142a7e. The error was because the NUMPY_VERSION environmental variable was not being read in or parsed correctly when the meta.yaml was not being read in.
ci/build_docs.sh is slow when the sphinx-gallery extension is downloaded from conda, but it successfully installed. See: 42e63db4b9bdf5bfb5aef34c2e9e028922b1cece
Regarding AppVeyor and the Windows binary: this doesn't seem to be working anymore. I've started building binaries to Azure-Pipelines. The deployment of these binaries works a lot more smoothly than the one with AppVeyor - binaries actually get built and published to Github Releases as a draft along with an automatically build CHANGELOG. I can also output the artifacts from PyInstaller from both the OSX and windows build for trouble shooting in addition to the screenshots of the GUI from Azure. Results from pytests also get published nicely in the browser. With some more modifications I may be able to enable to build tagged releases.
A problem has arisen with the PyInstaller that I've used the artifacts to better understand. On the VMs, none of the relevant conda packages (PySide, requests, python-dateutil) get packaged into the binaries. The sharppy packages appear to get included though. I am not sure how to get around this yet.
Now doing pyinstaller from within the runsharp directory. It successfully finds the full_gui import, but does not load in any other packages (e.g., Numpy). It does load in the PySide hooks like it should though. A problem is arising with the .nib file PySide needs to handle the Mac menu bars. For seem reason the directory gets ripped out on Azure.
Got Azure to build the macOS version almost correctly: an error associated with writing the correct version in the .spec file using versioneer keeps preventing the binary from being built correctly. Not sure what's going on with the windows version.
Info on signing .app:
https://docs.microsoft.com/en-us/azure/devops/pipelines/library/secure-files?view=vsts https://docs.microsoft.com/en-us/azure/devops/pipelines/tasks/utility/install-apple-certificate?view=vsts https://github.com/pyinstaller/pyinstaller/wiki/Recipe-OSX-Code-Signing https://docs.microsoft.com/en-us/appcenter/build/macos/code-signing
Potential fix for docs not building on Travis CI:
https://cloudblogs.microsoft.com/opensource/2019/04/05/publishing-github-pages-from-azure-pipelines/
Known issues right now:
Still need to test:
Todo before release:
Update on binaries:
Tried some things to solve the Windows problem:
Putting the platforms/ folder next to the SHARPpy.exe per the recommendation in the last comment of (https://stackoverflow.com/questions/47468705/pyinstaller-could-not-find-or-load-the-qt-platform-plugin-windows) successfully allowed the SHARPpy.exe to run. It did NOT have the problem with the available.py seen in the macOS binary.
Found out I can debug Pyinstaller binaries using pyi-archive_viewer and by using the debug and console switches in the EXE (https://stackoverflow.com/questions/13765801/how-do-i-debug-a-non-functioning-pyinstaller-build).
Replies to my post on https://github.com/pyinstaller/pyinstaller/issues/2857 oriented me to try to test making an Anaconda environment using the Pip version of PySide2. This is commit bf7dc12 and was successful at building a binary that could be launched on Windows 8. However, this environment.yml caused the Linux CIs to fail. Maybe I can try using PySide2 5.12 from Conda-forge.
Still need to put print statements everywhere in the Mac version to figure out what's going on there.
Primary Goals:
Low Priority Goals: