Open krosenfeld-IDM opened 1 year ago
Hi! Thanks for trying to build the article locally and reporting an issue.
First, I noticed you are using a much more recent version of showyourwork
compared to what was available at the time of writing this. I used the version from the commit cadb4086dd2a46cb68e6acffa73c2f7976e6d64d
of the main
branch of showyourwork
(see version
in showyourwork.yml
).
There is no guarantee that newer showyourwork
versions will work in the same way or at all, and I am indeed able to reproduce your error with a recent showyourwork
. You can install the specific version of showyourwork
needed by cloning that repo, checking out the commit above, and installing (after activating the conda environment you will use to build the paper) it with python setup.py install
from within the newly cloned folder.
After doing this, I get:
showyourwork --version
0.3.1.dev35+gcadb408
and typing showyourwork
in the CE_accretors
folder I can produce the manuscript locally.
Hi @mathren - thanks for the detailed instructions. I tried installing the corresponding version of showyourwork using pip:
$ pip install git+https://github.com/showyourwork/showyourwork.git@cadb4086dd2a46cb68e6acffa73c2f7976e6d64d
which looks good:
$ showyourwork --version
0.3.1.dev35+gcadb408
but the build still fails :
$ showyourwork
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Traceback (most recent call last):
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/__init__.py", line 701, in snakemake
success = workflow.execute(
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/workflow.py", line 1060, in execute
logger.run_info("\n".join(dag.stats()))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/dag.py", line 2191, in stats
yield tabulate(rows, headers="keys")
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 2048, in tabulate
list_of_lists, headers = _normalize_tabular_data(
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 1471, in _normalize_tabular_data
rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 1471, in <lambda>
rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 107, in _is_separating_line
(len(row) >= 1 and row[0] == SEPARATING_LINE)
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/rules.py", line 1138, in __eq__
return self.name == other.name and self.output == other.output
AttributeError: 'str' object has no attribute 'name'
Here is the environment:
# Name Version Build Channel
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 2_gnu conda-forge
appdirs 1.4.4 pypi_0 pypi
arrow 1.2.3 pypi_0 pypi
beautifulsoup4 4.12.2 pypi_0 pypi
binaryornot 0.4.4 pypi_0 pypi
bzip2 1.0.8 h7f98852_4 conda-forge
ca-certificates 2023.7.22 hbcca054_0 conda-forge
cachecontrol 0.12.11 pypi_0 pypi
certifi 2023.7.22 pypi_0 pypi
chardet 5.2.0 pypi_0 pypi
charset-normalizer 3.2.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
cookiecutter 2.3.1 pypi_0 pypi
distro 1.8.0 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
idna 3.4 pypi_0 pypi
jinja2 3.1.2 pypi_0 pypi
lastversion 3.3.2 pypi_0 pypi
ld_impl_linux-64 2.40 h41732ed_0 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 13.2.0 h807b86a_2 conda-forge
libgomp 13.2.0 h807b86a_2 conda-forge
libnsl 2.0.0 h7f98852_0 conda-forge
libsqlite 3.43.0 h2797004_0 conda-forge
libuuid 2.38.1 h0b41bf4_0 conda-forge
libzlib 1.2.13 hd590300_5 conda-forge
lockfile 0.12.2 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
msgpack 1.0.6 pypi_0 pypi
ncurses 6.4 hcb278e6_0 conda-forge
openssl 3.1.3 hd590300_0 conda-forge
packaging 23.1 pypi_0 pypi
pip 23.2.1 pyhd8ed1ab_0 conda-forge
pygments 2.16.1 pypi_0 pypi
python 3.9.18 h0755675_0_cpython conda-forge
python-dateutil 2.8.2 pypi_0 pypi
python-slugify 8.0.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h8228510_1 conda-forge
requests 2.31.0 pypi_0 pypi
rich 13.5.3 pypi_0 pypi
setuptools 68.2.2 pyhd8ed1ab_0 conda-forge
setuptools-scm 8.0.3 pypi_0 pypi
sgmllib3k 1.0.0 pypi_0 pypi
showyourwork 0.3.1.dev35+gcadb408 pypi_0 pypi
six 1.16.0 pypi_0 pypi
soupsieve 2.5 pypi_0 pypi
text-unidecode 1.3 pypi_0 pypi
tk 8.6.13 h2797004_0 conda-forge
tomli 2.0.1 pypi_0 pypi
tqdm 4.66.1 pypi_0 pypi
typing-extensions 4.8.0 pypi_0 pypi
tzdata 2023c h71feb2d_0 conda-forge
urllib3 1.26.16 pypi_0 pypi
wheel 0.41.2 pyhd8ed1ab_0 conda-forge
xz 5.2.6 h166bdaf_0 conda-forge
mmh, that looks like a snakemake
incompatibility error (see e.g., https://github.com/snakemake/snakemake/issues/1899). You may need to configure the conda priority to strict
.
On my workstation, where the build of the last commit works locally for me, conda list
outputs something different than your environment:
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
appdirs 1.4.4 pypi_0 pypi
arrow 1.2.3 pypi_0 pypi
beautifulsoup4 4.12.2 pypi_0 pypi
binaryornot 0.4.4 pypi_0 pypi
blas 1.0 mkl
ca-certificates 2023.05.30 h06a4308_0
cachecontrol 0.12.11 pypi_0 pypi
certifi 2023.7.22 pypi_0 pypi
chardet 5.2.0 pypi_0 pypi
charset-normalizer 3.2.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
compare-workdir 2.0 pypi_0 pypi
contourpy 1.1.0 pypi_0 pypi
cookiecutter 2.3.0 pypi_0 pypi
cycler 0.11.0 pypi_0 pypi
distro 1.8.0 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
fonttools 4.42.1 pypi_0 pypi
idna 3.4 pypi_0 pypi
importlib-resources 6.0.1 pypi_0 pypi
intel-openmp 2021.4.0 h06a4308_3561
jinja2 3.1.2 pypi_0 pypi
joblib 1.1.0 pyhd3eb1b0_0
kiwisolver 1.4.5 pypi_0 pypi
lastversion 3.0.1 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 7.5.0 ha8ba4b0_17
libgfortran4 7.5.0 ha8ba4b0_17
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
lockfile 0.12.2 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
matplotlib 3.7.2 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py39h7f8727e_0
mkl_fft 1.3.1 py39hd3c417c_0
mkl_random 1.2.2 py39h51133e4_0
msgpack 1.0.5 pypi_0 pypi
ncurses 6.4 h6a678d5_0
numpy 1.20.1 py39h93e21f0_0
numpy-base 1.20.1 py39h7d8b39e_0
openssl 3.0.10 h7f8727e_2
packaging 23.1 pypi_0 pypi
pillow 10.0.0 pypi_0 pypi
pip 23.2.1 py39h06a4308_0
pygments 2.16.1 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
python 3.9.17 h955ad1f_0
python-dateutil 2.8.2 pypi_0 pypi
python-slugify 8.0.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.31.0 pypi_0 pypi
rich 13.5.2 pypi_0 pypi
scipy 1.7.1 py39h292c36d_2
setuptools 68.0.0 py39h06a4308_0
showyourwork 0.3.1.dev35+gcadb408 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
soupsieve 2.5 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
termcolor 1.1.0 pypi_0 pypi
text-unidecode 1.3 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
tqdm 4.66.1 pypi_0 pypi
tzdata 2023c h04d1e81_0
urllib3 1.26.16 pypi_0 pypi
wheel 0.38.4 py39h06a4308_0
xz 5.4.2 h5eee18b_0
zipp 3.16.2 pypi_0 pypi
zlib 1.2.13 h5eee18b_0
Bizarre. Using the same version of python, setting channel priority to strict
with the defaults
channel my environment now looks similar to yours:
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
appdirs 1.4.4 pypi_0 pypi
arrow 1.2.3 pypi_0 pypi
beautifulsoup4 4.12.2 pypi_0 pypi
binaryornot 0.4.4 pypi_0 pypi
ca-certificates 2023.08.22 h06a4308_0
cachecontrol 0.12.11 pypi_0 pypi
certifi 2023.7.22 pypi_0 pypi
chardet 5.2.0 pypi_0 pypi
charset-normalizer 3.2.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
cookiecutter 2.3.1 pypi_0 pypi
distro 1.8.0 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
idna 3.4 pypi_0 pypi
jinja2 3.1.2 pypi_0 pypi
lastversion 3.3.2 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
lockfile 0.12.2 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
msgpack 1.0.6 pypi_0 pypi
ncurses 6.4 h6a678d5_0
openssl 3.0.11 h7f8727e_2
packaging 23.1 pypi_0 pypi
pip 23.2.1 py39h06a4308_0
pygments 2.16.1 pypi_0 pypi
python 3.9.17 h955ad1f_0
python-dateutil 2.8.2 pypi_0 pypi
python-slugify 8.0.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.31.0 pypi_0 pypi
rich 13.5.3 pypi_0 pypi
setuptools 68.0.0 py39h06a4308_0
setuptools-scm 8.0.3 pypi_0 pypi
sgmllib3k 1.0.0 pypi_0 pypi
showyourwork 0.3.1.dev35+gcadb408 pypi_0 pypi
six 1.16.0 pypi_0 pypi
soupsieve 2.5 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
text-unidecode 1.3 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
tomli 2.0.1 pypi_0 pypi
tqdm 4.66.1 pypi_0 pypi
typing-extensions 4.8.0 pypi_0 pypi
tzdata 2023c h04d1e81_0
urllib3 1.26.16 pypi_0 pypi
wheel 0.41.2 py39h06a4308_0
xz 5.4.2 h5eee18b_0
zlib 1.2.13 h5eee18b_0
Same error though:
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Traceback (most recent call last):
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/__init__.py", line 701, in snakemake
success = workflow.execute(
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/workflow.py", line 1060, in execute
logger.run_info("\n".join(dag.stats()))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/dag.py", line 2191, in stats
yield tabulate(rows, headers="keys")
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 2048, in tabulate
list_of_lists, headers = _normalize_tabular_data(
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 1471, in _normalize_tabular_data
rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 1471, in <lambda>
rows = list(map(lambda r: r if _is_separating_line(r) else list(r), rows))
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/tabulate/__init__.py", line 107, in _is_separating_line
(len(row) >= 1 and row[0] == SEPARATING_LINE)
File "/home/krosenfeld/.showyourwork/env/lib/python3.9/site-packages/snakemake/rules.py", line 1138, in __eq__
return self.name == other.name and self.output == other.output
AttributeError: 'str' object has no attribute 'name'
My ~/.condarc
:
channels:
- defaults
- conda-forge
channel_priority: strict
mmh..
I have successfully re-build locally a commit that github struggles with, apparently because of different conda
versions. What are you using?
with conda=22.9.0
, I can create the environment, activate it, pip install the correct shouwyourwork
version, and then successfully building (with showyourwork
downloading the datasets from zenodo as expected).
From a working environment, I created an updated environment using conda env export
, see https://github.com/mathren/CE_accretors/blob/debug/environment.yml.
Can you try building a conda environment from that file, pip-install showyourwork, and build?
The commit I can locally build (but github actions fails using a more recent conda version) is on the branch debug
,
https://github.com/mathren/CE_accretors/commit/34059fa5277a2de59664ccf2f52452fb6ecff731
The conda version seems likely. I have tried versions 23.5.2 and 23.3.1 and the updated environment you provided. All result in that same AttributeError
.
any chance you could try conda=22.9.0
?
That did not seem to work either. Here is what I am looking at: My version of CE_accretors:
git branch
* (HEAD detached at 34059fa)
my conda version:
conda --version
conda 22.9.0
and my environment:
# Name Version Build Channel
_libgcc_mutex 0.1 main
_openmp_mutex 5.1 1_gnu
appdirs 1.4.4 pypi_0 pypi
arrow 1.2.3 pypi_0 pypi
beautifulsoup4 4.12.2 pypi_0 pypi
binaryornot 0.4.4 pypi_0 pypi
blas 1.0 mkl
ca-certificates 2023.08.22 h06a4308_0
cachecontrol 0.12.11 pypi_0 pypi
certifi 2023.7.22 pypi_0 pypi
chardet 5.2.0 pypi_0 pypi
charset-normalizer 3.2.0 pypi_0 pypi
click 8.1.7 pypi_0 pypi
compare-workdir 2.0 pypi_0 pypi
contourpy 1.1.1 pypi_0 pypi
cookiecutter 2.4.0 pypi_0 pypi
cycler 0.12.0 pypi_0 pypi
distro 1.8.0 pypi_0 pypi
feedparser 6.0.10 pypi_0 pypi
fonttools 4.43.0 pypi_0 pypi
idna 3.4 pypi_0 pypi
importlib-resources 6.1.0 pypi_0 pypi
intel-openmp 2021.4.0 h06a4308_3561
jinja2 3.1.2 pypi_0 pypi
joblib 1.1.0 pyhd3eb1b0_0
kiwisolver 1.4.5 pypi_0 pypi
lastversion 3.3.2 pypi_0 pypi
ld_impl_linux-64 2.38 h1181459_1
libffi 3.4.4 h6a678d5_0
libgcc-ng 11.2.0 h1234567_1
libgfortran-ng 7.5.0 ha8ba4b0_17
libgfortran4 7.5.0 ha8ba4b0_17
libgomp 11.2.0 h1234567_1
libstdcxx-ng 11.2.0 h1234567_1
lockfile 0.12.2 pypi_0 pypi
markdown-it-py 3.0.0 pypi_0 pypi
markupsafe 2.1.3 pypi_0 pypi
matplotlib 3.8.0 pypi_0 pypi
mdurl 0.1.2 pypi_0 pypi
mkl 2021.4.0 h06a4308_640
mkl-service 2.4.0 py39h7f8727e_0
mkl_fft 1.3.1 py39hd3c417c_0
mkl_random 1.2.2 py39h51133e4_0
msgpack 1.0.7 pypi_0 pypi
ncurses 6.4 h6a678d5_0
numpy 1.26.0 pypi_0 pypi
openssl 3.0.11 h7f8727e_2
packaging 23.1 pypi_0 pypi
pillow 10.0.1 pypi_0 pypi
pip 23.2.1 py39h06a4308_0
pygments 2.16.1 pypi_0 pypi
pyparsing 3.1.1 pypi_0 pypi
python 3.9.18 h955ad1f_0
python-dateutil 2.8.2 pypi_0 pypi
python-slugify 8.0.1 pypi_0 pypi
pyyaml 6.0.1 pypi_0 pypi
readline 8.2 h5eee18b_0
requests 2.31.0 pypi_0 pypi
rich 13.5.3 pypi_0 pypi
scipy 1.7.1 py39h292c36d_2
setuptools 68.0.0 py39h06a4308_0
setuptools-scm 8.0.3 pypi_0 pypi
sgmllib3k 1.0.0 pypi_0 pypi
showyourwork 0.3.1.dev35+gcadb408 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
soupsieve 2.5 pypi_0 pypi
sqlite 3.41.2 h5eee18b_0
termcolor 1.1.0 pypi_0 pypi
text-unidecode 1.3 pypi_0 pypi
tk 8.6.12 h1ccaba5_0
tomli 2.0.1 pypi_0 pypi
tqdm 4.66.1 pypi_0 pypi
typing-extensions 4.8.0 pypi_0 pypi
tzdata 2023c h04d1e81_0
urllib3 1.26.16 pypi_0 pypi
wheel 0.41.2 py39h06a4308_0
xz 5.4.2 h5eee18b_0
zipp 3.17.0 pypi_0 pypi
zlib 1.2.13 h5eee18b_0
I appreciate your looking into this. It is not-critical from my perspective. Is this useful for you?
I wouldn't say "useful", since this is a wrapped up project on my side. But it is frustrating it is not working for you, and a good test of showyourwork. The error you find is always the same AttributeError
?
Yes - same error.
When I try to build the article locally following the instructions in the README I get the following output and error:
I am on a Linux machine (Ubuntu 22.04.2) using showyourwork
0.4.4.dev4+gacd9a28
.