Closed jpivarski closed 3 years ago
chrisburr
commented
3 years ago Thanks for the bug report. I'll try asking the ROOT team to see if they can guess why the pcm's are being rebuilt.
In the meantime I have a couple of questions which might explain why the rebuild is failing:
export PATH=/home/jpivarski/miniconda3/bin:$PATH or are you using eval/source with one of the activation scripts?
amadio
commented
3 years ago What is the output of root-config --config | tr ' ' '\n' | grep -i tbb? Do you have LD_LIBRARY_PATH and/or CPATH set by any means, like by loading ICC from somewhere?
jpivarski
commented
3 years ago
- What shell are you using?
% echo $SHELL
/bin/bash
- How do you activate the environment? Are you doing
export PATH=/home/jpivarski/miniconda3/bin:$PATHor are you usingeval/sourcewith one of the activation scripts?
I have this at the end of my ~/.bashrc:
# >>> conda initialize >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$('/home/jpivarski/miniconda3/bin/conda' 'shell.bash' 'hook' 2> /dev/null)"
if [ $? -eq 0 ]; then
eval "$__conda_setup"
else
if [ -f "/home/jpivarski/miniconda3/etc/profile.d/conda.sh" ]; then
. "/home/jpivarski/miniconda3/etc/profile.d/conda.sh"
else
export PATH="/home/jpivarski/miniconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda initialize <<<What is the output of root-config --config | tr ' ' '\n' | grep -i tbb?
For that, I'll need to switch back into the latest ROOT. Right now, I have 6.22/08 and I don't want to rock the boat (even in a new conda environment) because I have a demo to do on Monday. (CMS LPC HATS)
For what it's worth, this is the output of root-config for my 6.22/08 version:
% root-config --config | tr ' ' '\n' | grep -i tbb
TBB_INCLUDE_DIR=/home/jpivarski/miniconda3/include
TBB_LIBRARY=/home/jpivarski/miniconda3/lib/libtbb.so
TBB_LIBRARY_RELEASE=/home/jpivarski/miniconda3/lib/libtbb.soI'll come back to this and can answer the question again when I un-pin the ROOT version.
Do you have LD_LIBRARY_PATH and/or CPATH set by any means, like by loading ICC from somewhere?
No:
% echo $LD_LIBRARY_PATH
% echo $CPATH
The first two points look fine.
If you see this again after Monday it would be interesting to try if installing tbb-devel fixes the issue completely. I can half reproduce this locally by deleting all of the lib/*.pcm and it seems to help but I then run into other issues which I think are related to me deleting the files.
jpivarski
commented
3 years ago Will do, thanks!
And thanks, @amadio, for pointing me to Mattermost/ROOT/build-system. That does look relevant, because TReentrantRWLock.hxx is in my error output. If that's what the problem is, then just waiting for 6.24/02 would fix it.
jpivarski
commented
3 years ago I've upgraded (conda uninstall root root_base && conda update --all && conda install root tbb-devel) and so far, so good: I can start root and import ROOT in Python. I'll know by the end of the week if the problem reappears, so I'll be testing it every day and will close this issue on Friday if tbb-devel fixes it.
Oh, and @amadio, with ROOT 6.24/00,
% root-config --config | tr ' ' '\n' | grep -i tbb
TBB_INCLUDE_DIR=/home/jpivarski/miniconda3/include
TBB_LIBRARY=/home/jpivarski/miniconda3/lib/libtbb.so
TBB_LIBRARY_RELEASE=/home/jpivarski/miniconda3/lib/libtbb.soalthough I have installed the tbb-devel explicitly. Without that, I didn't have any TBB from conda.
jpivarski
commented
3 years ago I didn't wait until Friday, but I've been checking it every day and haven't run into any problems. I think that adding tbb-devel to the conda install line did it.
jpivarski
commented
3 years ago I guess it took from June 17 until August 11 for it to spontaneously break again. I have more evidence this time that it was spontaneous: I was working with ROOT in the morning, then it stopped working in the afternoon.
tbb-devel is still explicitly included in my set of packages.conda update --all since June, so my packages could have been getting out of date.This is the output from attempting to launch root, just before I started a conda remove root ; conda config --set channel_priority strict ; conda update --all ; conda install root:
<<< cling interactive line includer >>>: remark: building module 'MathCore' as '/home/jpivarski/miniconda3/lib/MathCore.pcm' [-Rmodule-build]
While building module 'MathCore':
In file included from <module-includes>:3:
In file included from /home/jpivarski/miniconda3/include/Fit/Chi2FCN.h:19:
/home/jpivarski/miniconda3/include/Fit/FitUtil.h:404:30: error: no matching constructor for initialization of 'std::vector<double>'
std::vector<double> ones{1, 1, 1, 1};
^ ~~~~~~~~~~~~
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:650:2: note: candidate constructor template not viable: requires at most 3 arguments, but 4 were provided
vector(_InputIterator __first, _InputIterator __last,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:519:7: note: candidate constructor not viable: requires at most 3 arguments, but 4 were provided
vector(size_type __n, const value_type& __value,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:582:7: note: candidate constructor not viable: requires 3 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m, true_type) noexcept
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:586:7: note: candidate constructor not viable: requires 3 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m, false_type)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:507:7: note: candidate constructor not viable: requires at most 2 arguments, but 4 were provided
vector(size_type __n, const allocator_type& __a = allocator_type())
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:572:7: note: candidate constructor not viable: requires 2 arguments, but 4 were provided
vector(const vector& __x, const allocator_type& __a)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:604:7: note: candidate constructor not viable: requires 2 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:622:7: note: candidate constructor not viable: requires at most 2 arguments, but 4 were provided
vector(initializer_list<value_type> __l,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:494:7: note: candidate constructor not viable: requires single argument '__a', but 4 arguments were provided
vector(const allocator_type& __a) _GLIBCXX_NOEXCEPT
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:550:7: note: candidate constructor not viable: requires single argument '__x', but 4 arguments were provided
vector(const vector& __x)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:569:7: note: candidate constructor not viable: requires 1 argument, but 4 were provided
vector(vector&&) noexcept = default;
^
/home/jpivarski/miniconda3/bin/../lib/gcc/x86_64-conda-linux-gnu/9.3.0/../../../../x86_64-conda-linux-gnu/include/c++/9.3.0/bits/stl_vector.h:484:7: note: candidate constructor not viable: requires 0 arguments, but 4 were provided
vector() = default;
^
<<< cling interactive line includer >>>: remark: finished building module 'MathCore' [-Rmodule-build]
<<< cling interactive line includer >>>: fatal error: could not build module 'MathCore'
Failed to load module MathCore
While building module 'Hist':
In file included from <module-includes>:5:
/home/jpivarski/miniconda3/include/TBackCompFitter.h:14:10: fatal error: could not build module 'MathCore'
#include "Fit/BasicFCN.h"
~~~~~~~~^~~~~~~~~~~~~~~~
Failed to load module Hist
------------------------------------------------------------------
| Welcome to ROOT 6.24/02 https://root.cern |
| (c) 1995-2021, The ROOT Team; conception: R. Brun, F. Rademakers |
| Built for linuxx8664gcc on Jul 03 2021, 07:58:00 |
| From tag , 28 June 2021 |
| With |
| Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
------------------------------------------------------------------
Error in <TCling::LoadPCM>: ROOT PCM /home/jpivarski/miniconda3/lib/libNet_rdict.pcm file does not exist
Failed to load module MathCore
Info in <TInterpreter::TCling::RegisterModule>: Module MathCore in modulemap failed to load.
Error in <TCling::LoadPCM>: ROOT PCM /home/jpivarski/miniconda3/lib/libMathCore_rdict.pcm file does not exist
root [0] .qIt lasted 7 days this time. A few minutes ago (the last time I ran pytest), I had a working ROOT installation, now it does this:
% root
Warning in cling::IncrementalParser::CheckABICompatibility():
Possible C++ standard library mismatch, compiled with __GLIBCXX__ '20200312'
Extraction of runtime standard library version was: '20210601'
<<< cling interactive line includer >>>: remark: building module 'MathCore' as '/home/jpivarski/miniconda3/lib/MathCore.pcm' [-Rmodule-build]
While building module 'MathCore':
In file included from <module-includes>:3:
In file included from /home/jpivarski/miniconda3/include/Fit/Chi2FCN.h:19:
/home/jpivarski/miniconda3/include/Fit/FitUtil.h:404:30: error: no matching constructor for initialization of 'std::vector<double>'
std::vector<double> ones{1, 1, 1, 1};
^ ~~~~~~~~~~~~
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:650:2: note: candidate constructor template not viable: requires at most 3 arguments, but 4 were provided
vector(_InputIterator __first, _InputIterator __last,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:519:7: note: candidate constructor not viable: requires at most 3 arguments, but 4 were provided
vector(size_type __n, const value_type& __value,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:582:7: note: candidate constructor not viable: requires 3 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m, true_type) noexcept
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:586:7: note: candidate constructor not viable: requires 3 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m, false_type)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:507:7: note: candidate constructor not viable: requires at most 2 arguments, but 4 were provided
vector(size_type __n, const allocator_type& __a = allocator_type())
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:572:7: note: candidate constructor not viable: requires 2 arguments, but 4 were provided
vector(const vector& __x, const allocator_type& __a)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:604:7: note: candidate constructor not viable: requires 2 arguments, but 4 were provided
vector(vector&& __rv, const allocator_type& __m)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:622:7: note: candidate constructor not viable: requires at most 2 arguments, but 4 were provided
vector(initializer_list<value_type> __l,
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:494:7: note: candidate constructor not viable: requires single argument '__a', but 4 arguments were provided
vector(const allocator_type& __a) _GLIBCXX_NOEXCEPT
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:550:7: note: candidate constructor not viable: requires single argument '__x', but 4 arguments were provided
vector(const vector& __x)
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:569:7: note: candidate constructor not viable: requires 1 argument, but 4 were provided
vector(vector&&) noexcept = default;
^
/home/jpivarski/miniconda3/bin/../lib/gcc/../../x86_64-conda-linux-gnu/include/c++/9.4.0/bits/stl_vector.h:484:7: note: candidate constructor not viable: requires 0 arguments, but 4 were provided
vector() = default;
^
<<< cling interactive line includer >>>: remark: finished building module 'MathCore' [-Rmodule-build]
<<< cling interactive line includer >>>: fatal error: could not build module 'MathCore'
Failed to load module MathCore
While building module 'Hist':
In file included from <module-includes>:5:
/home/jpivarski/miniconda3/include/TBackCompFitter.h:14:10: fatal error: could not build module 'MathCore'
#include "Fit/BasicFCN.h"
~~~~~~~~^~~~~~~~~~~~~~~~
Failed to load module Hist
------------------------------------------------------------------
| Welcome to ROOT 6.24/02 https://root.cern |
| (c) 1995-2021, The ROOT Team; conception: R. Brun, F. Rademakers |
| Built for linuxx8664gcc on Jul 03 2021, 07:58:00 |
| From tag , 28 June 2021 |
| With |
| Try '.help', '.demo', '.license', '.credits', '.quit'/'.q' |
------------------------------------------------------------------
Error in <TCling::LoadPCM>: ROOT PCM /home/jpivarski/miniconda3/lib/libNet_rdict.pcm file does not exist
Failed to load module MathCore
Info in <TInterpreter::TCling::RegisterModule>: Module MathCore in modulemap failed to load.
Error in <TCling::LoadPCM>: ROOT PCM /home/jpivarski/miniconda3/lib/libMathCore_rdict.pcm file does not exist
root [0] .qI'm going to see if it can be fixed with a single conda update --all command. Doing so does not change my ROOT version:
The following packages will be UPDATED:
arrow-cpp 5.0.0-py38hf24f39c_2_cpu --> 5.0.0-py38h4026a5f_3_cpu
dask 2021.7.2-pyhd8ed1ab_0 --> 2021.8.0-pyhd8ed1ab_0
dask-core 2021.7.2-pyhd8ed1ab_0 --> 2021.8.0-pyhd8ed1ab_0
dataclasses 0.8-pyhc8e2a94_1 --> 0.8-pyhc8e2a94_3
distributed 2021.7.2-py38h578d9bd_0 --> 2021.8.0-py38h578d9bd_0
grpc-cpp 1.39.0-hf1f433d_2 --> 1.39.1-hf1f433d_0
harfbuzz 2.8.2-h83ec7ef_0 --> 2.9.0-h83ec7ef_0
importlib-metadata 4.6.3-py38h578d9bd_0 --> 4.6.4-py38h578d9bd_0
importlib_metadata 4.6.3-hd8ed1ab_0 --> 4.6.4-hd8ed1ab_0
ipykernel 6.0.3-py38hd0cf306_0 --> 6.2.0-py38he5a9106_0
jax 0.2.18-pyhd8ed1ab_0 --> 0.2.19-pyhd8ed1ab_0
jupyterlab 3.1.4-pyhd8ed1ab_0 --> 3.1.7-pyhd8ed1ab_0
jupyterlab_server 2.7.0-pyhd8ed1ab_0 --> 2.7.1-pyhd8ed1ab_0
keyring 23.0.1-py38h578d9bd_0 --> 23.1.0-py38h578d9bd_0
libwebp 1.2.0-h3452ae3_0 --> 1.2.1-h3452ae3_0
libwebp-base 1.2.0-h7f98852_2 --> 1.2.1-h7f98852_0
matplotlib 3.4.2-py38h578d9bd_0 --> 3.4.3-py38h578d9bd_0
matplotlib-base 3.4.2-py38hf4fb855_0 --> 3.4.3-py38hf4fb855_0
nbclient 0.5.3-pyhd8ed1ab_0 --> 0.5.4-pyhd8ed1ab_0
numpy 1.21.1-py38h9894fe3_0 --> 1.21.2-py38he2449b9_0
openssl 1.1.1k-h7f98852_0 --> 1.1.1k-h7f98852_1
orc 1.6.9-h58a87f1_0 --> 1.6.10-h58a87f1_0
pandas 1.3.1-py38h1abd341_0 --> 1.3.2-py38h43a58ef_0
pango 1.48.7-hb8ff022_0 --> 1.48.9-hb8ff022_0
pip 21.2.3-pyhd8ed1ab_0 --> 21.2.4-pyhd8ed1ab_0
pyarrow 5.0.0-py38h1bc9799_2_cpu --> 5.0.0-py38h1bc9799_3_cpu
pygments 2.9.0-pyhd8ed1ab_0 --> 2.10.0-pyhd8ed1ab_0
pyyaml 5.4.1-py38h497a2fe_0 --> 5.4.1-py38h497a2fe_1
seaborn 0.11.1-hd8ed1ab_1 --> 0.11.2-hd8ed1ab_0
seaborn-base 0.11.1-pyhd8ed1ab_1 --> 0.11.2-pyhd8ed1ab_0
setuptools 49.6.0-py38h578d9bd_3 --> 57.4.0-py38h578d9bd_0
tqdm 4.62.0-pyhd8ed1ab_0 --> 4.62.1-pyhd8ed1ab_0
wheel 0.37.0-pyhd8ed1ab_0 --> 0.37.0-pyhd8ed1ab_1Doing conda update --all does not fix it, but
conda remove root
conda update --all
conda install rootdoes.
jpivarski
commented
3 years ago I was just going to report back here that it's happened again, just to take more statistics on how many days have elapsed. It has been 7 days. Interestingly, the time before that was another 7 days. And before that, 56 days, though I hadn't been working on anything that used ROOT in that interval and had no reason to notice a disruption.
Each of these is divisible by 7!
Does something conda-related or something ROOT-related happen on Wednesdays? At least for this time and the time right before it, I know I wasn't doing anything but running ROOT interactively and in PyROOT, and at about mid-day Chicago-time (~18:00 UTC), it failed with the above error messages. My only known fix is to uninstall it and reinstall it.
I have an up-to-date conda environment with
conda-forgeas the only non-default channel. Recently, just starting uprootorimport ROOThas resulted in a core dump:Interestingly enough, removing ROOT,
conda update --all, and reinstalling ROOT (through conda) made it work for a few days, but then the segfault is back. Something I'm running in the interim (PyROOT? JupyROOT?) is changing its state and causing it to fail at startup.Environment (
conda list):Details about
condaand system (conda info):Removing ROOT (
conda remove root root_base) applies the following changes:After doing that, I've verified that I have no ROOT installation whatsoever (i.e. nothing interfering from another PATH). Same for PyROOT.
Also after doing that, I've verified that
conda update --alldoes nothing. It's already up to date.I'm going to try using 6.22.8 instead.