search

ChangLabSNU / python-linearfold

Unofficial CPython binding to LinearFold
Other
2 stars 1 forks source link

dependency issue #1

Open seqirus-jawon opened 1 month ago

seqirus-jawon commented 1 month ago

hi, I am getting error installing withpip install linearfold-unofficial on python3.12. Are there dependencies I need to install for this? Also, I tried with native linearfold pypi (pip install LinearFold) and it looks like it needs customized linearfold build? Thanks.

Cauch-BS commented 1 month ago

Hello, could you give details on the full error message?

It might also be helpful if you clarified the OS on which you are installing LinearFold. If you are running windows, it might be helpful to know the Visual Studio Version.

For example, installing linearfold on Windows using Visual Studio 2022 seems to result in an error (fatal error C1083: Cannot open include file: 'sys/time.h'). (In case you are encountering this error, you might want to use WSL or MinGW: see here for more) However, installing it on Ubuntu-22.04-LTS with build-essential might not produce this error.

seqirus-jawon commented 1 month ago

Hi, I am running on Mac OS. 

==> Initializing sequence with LinearDesign...
LinearDesign: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3795/3795 [12:05<00:00,  5.23nt/s]
Traceback (most recent call last):
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/sequence_evaluator.py", line 52, in initialize
    import linearfold
ModuleNotFoundError: No module named 'linearfold'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jawon/miniforge3-arm64/bin/vaxpress", line 8, in <module>
    sys.exit(run_vaxpress())
             ^^^^^^^^^^^^^^
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/__main__.py", line 354, in run_vaxpress
    evochamber = CDSEvolutionChamber(
                 ^^^^^^^^^^^^^^^^^^^^
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/evolution_chamber.py", line 76, in __init__
    self.initialize()
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/evolution_chamber.py", line 136, in initialize
    self.seqeval = SequenceEvaluator(self.scoringfuncs, self.scoreopts,
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/sequence_evaluator.py", line 126, in __init__
    self.initialize()
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/sequence_evaluator.py", line 129, in initialize
    self.foldeval = FoldEvaluator(self.execopts.folding_engine)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/sequence_evaluator.py", line 40, in __init__
    self.initialize()
  File "/Users/jawon/miniforge3-arm64/lib/python3.12/site-packages/vaxpress/sequence_evaluator.py", line 54, in initialize
    raise ImportError('LinearFold module is not available. Try "'
ImportError: LinearFold module is not available. Try "pip install linearfold-unofficial" to install.

This was the error for Vaxpress which uses linearfold, and when I tried to install linearfold-unofficial, 

Building wheels for collected packages: linearfold-unofficial
  Building wheel for linearfold-unofficial (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [115 lines of output]
      running bdist_wheel
      running build
      running build_ext
      building 'linearfold' extension
      creating build/temp.macosx-10.13-x86_64-cpython-312
      clang++ -fno-strict-overflow -Wsign-compare -Wunreachable-code -DNDEBUG -O2 -Wall -fPIC -O2 -isystem /Users/jawon/conda.x86/include -fPIC -O2 -isystem /Users/jawon/conda.x86/include -ILinearFold/src -I/Users/jawon/conda.x86/include/python3.12 -c linearfoldmodule.cc -o build/temp.macosx-10.13-x86_64-cpython-312/linearfoldmodule.o
      In file included from linearfoldmodule.cc:33:
      In file included from LinearFold/src/LinearFold.cpp:21:
      LinearFold/src/LinearFold.h:53:21: error: no template named 'tuple' in namespace 'std'
      bool cmp(const std::tuple<value_type, int, int>& a, const std::tuple<value_type, int, int>& b)
                     ~~~~~^
      LinearFold/src/LinearFold.h:53:64: error: no template named 'tuple' in namespace 'std'
      bool cmp(const std::tuple<value_type, int, int>& a, const std::tuple<value_type, int, int>& b)
                                                                ~~~~~^
      LinearFold/src/LinearFold.h:90:14: warning: field 'manner' will be initialized after field 'score' [-Wreorder-ctor]
          State(): manner(MANNER_NONE), score(VALUE_MIN) {};
                   ^~~~~~~~~~~~~~~~~~~  ~~~~~~~~~~~~~~~~
                   score(VALUE_MIN)     manner(MANNER_NONE)
      LinearFold/src/LinearFold.h:117:20: warning: default member initializer for non-static data member is a C++11 extension [-Wc++11-extensions]
          bool use_shape = false;
                         ^
      LinearFold/src/LinearFold.h:118:14: warning: default member initializer for non-static data member is a C++11 extension [-Wc++11-extensions]
          double m = 1.8;
                   ^
      LinearFold/src/LinearFold.h:119:14: warning: default member initializer for non-static data member is a C++11 extension [-Wc++11-extensions]
          double b = -0.6;
                   ^
      LinearFold/src/LinearFold.h:120:19: warning: default member initializer for non-static data member is a C++11 extension [-Wc++11-extensions]
          bool is_fasta = false;
                        ^
      LinearFold/src/LinearFold.h:146:127: error: no member named 'tuple' in namespace 'std'
          std::pair<std::string, std::string> get_parentheses_outside_real_backtrace(int i, int j, State& state_beta, std::map<std::tuple<BestTypes, int, int>, std::pair<std::string, std::string> >& global_visited_outside, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                                               ~~~~~^
      LinearFold/src/LinearFold.h:146:133: error: 'BestTypes' does not refer to a value
          std::pair<std::string, std::string> get_parentheses_outside_real_backtrace(int i, int j, State& state_beta, std::map<std::tuple<BestTypes, int, int>, std::pair<std::string, std::string> >& global_visited_outside, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                                                          ^
      LinearFold/src/LinearFold.h:44:6: note: declared here
      enum BestTypes {
           ^
      LinearFold/src/LinearFold.h:146:191: error: expected ')'
          std::pair<std::string, std::string> get_parentheses_outside_real_backtrace(int i, int j, State& state_beta, std::map<std::tuple<BestTypes, int, int>, std::pair<std::string, std::string> >& global_visited_outside, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                                                                                                                    ^
      LinearFold/src/LinearFold.h:146:79: note: to match this '('
          std::pair<std::string, std::string> get_parentheses_outside_real_backtrace(int i, int j, State& state_beta, std::map<std::tuple<BestTypes, int, int>, std::pair<std::string, std::string> >& global_visited_outside, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                    ^
      LinearFold/src/LinearFold.h:147:97: error: no member named 'tuple' in namespace 'std'
          std::string get_parentheses_inside_real_backtrace(int i, int j, State& state, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                 ~~~~~^
      LinearFold/src/LinearFold.h:147:103: error: 'BestTypes' does not refer to a value
          std::string get_parentheses_inside_real_backtrace(int i, int j, State& state, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                            ^
      LinearFold/src/LinearFold.h:44:6: note: declared here
      enum BestTypes {
           ^
      LinearFold/src/LinearFold.h:147:136: error: expected ')'
          std::string get_parentheses_inside_real_backtrace(int i, int j, State& state, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                                                                                                             ^
      LinearFold/src/LinearFold.h:147:54: note: to match this '('
          std::string get_parentheses_inside_real_backtrace(int i, int j, State& state, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);
                                                           ^
      LinearFold/src/LinearFold.h:151:46: error: a space is required between consecutive right angle brackets (use '> >')
          std::vector<std::unordered_map<int, State>> bestH, bestP, bestM2, bestMulti, bestM;
                                                   ^~
                                                   > >
      LinearFold/src/LinearFold.h:154:46: error: a space is required between consecutive right angle brackets (use '> >')
          std::vector<std::unordered_map<int, State>> bestH_beta, bestP_beta, bestM2_beta, bestMulti_beta, bestM_beta;
                                                   ^~
                                                   > >
      LinearFold/src/LinearFold.h:161:54: error: a space is required between consecutive right angle brackets (use '> >')
          std::vector<std::vector<std::pair<value_type, int>>> sorted_bestM;
                                                           ^~
                                                           > >
      LinearFold/src/LinearFold.h:164:37: error: a space is required between consecutive right angle brackets (use '> >')
          std::vector<std::pair<int, State>> keys;
                                          ^~
                                          > >
      LinearFold/src/LinearFold.h:167:88: error: a space is required between consecutive right angle brackets (use '> >')
          void sort_keys(std::unordered_map<int, State> &map, std::vector<std::pair<int,State>> &sorted_keys);
                                                                                             ^~
                                                                                             > >
      LinearFold/src/LinearFold.h:171:53: error: a space is required between consecutive right angle brackets (use '> >')
                     std::vector<std::pair<value_type, int>>& sorted_stepM);
                                                          ^~
                                                          > >
      LinearFold/src/LinearFold.h:208:42: error: a space is required between consecutive right angle brackets (use '> >')
          std::vector<std::pair<value_type, int>> scores;
                                               ^~
                                               > >
      In file included from linearfoldmodule.cc:33:
      In file included from LinearFold/src/LinearFold.cpp:23:
      In file included from LinearFold/src/LinearFoldEval.h:18:
      LinearFold/src/Utils/utility_v.h:201:9: warning: unused variable 'p' [-Wunused-variable]
          int p = i;
              ^
      LinearFold/src/Utils/utility_v.h:202:9: warning: unused variable 'q' [-Wunused-variable]
          int q = k;
              ^
      In file included from linearfoldmodule.cc:33:
      In file included from LinearFold/src/LinearFold.cpp:23:
      LinearFold/src/LinearFoldEval.h:45:24: error: a space is required between consecutive right angle brackets (use '> >')
          stack<pair<int, int>> stk; // tuple of (index, page)
                             ^~
                             > >
      LinearFold/src/LinearFoldEval.h:46:5: error: no template named 'tuple'
          tuple<int, int> inner_loop;
          ^
      LinearFold/src/LinearFoldEval.h:66:13: error: no template named 'tuple'
                  tuple<int, int> top = stk.top();
                  ^
      LinearFold/src/LinearFoldEval.h:115:26: error: use of undeclared identifier 'make_tuple'
                  inner_loop = make_tuple(i, j);
                               ^
      fatal error: too many errors emitted, stopping now [-ferror-limit=]
      7 warnings and 20 errors generated.
      error: command '/usr/bin/clang++' failed with exit code 1
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for linearfold-unofficial
  Running setup.py clean for linearfold-unofficial
Failed to build linearfold-unofficial
ERROR: ERROR: Failed to build installable wheels for some pyproject.toml based projects (linearfold-unofficial)

this was the error message.

Cauch-BS commented 1 month ago

Hello, thank you for clarifying the situation and providing the detailed error context. This was very helpful.

The most important warning in the error log seems to be this line, 

 LinearFold/src/LinearFold.h:146:127: error: no member named 'tuple' in namespace 'std'
          std::pair<std::string, std::string> get_parentheses_outside_real_backtrace(int i, int j, State& state_beta, std::map<std::tuple<BestTypes, int, int>, std::pair<std::string, std::string> >& global_visited_outside, std::map<std::tuple<BestTypes, int, int>, std::string>& global_visited_inside, std::set<std::pair<int,int> >& window_visited);

which seems to indicate the clang compiler is compiling with a standard version before C++11. This is not expected, and the original linearfold was written with C++11 in mind. For the relevant StackOverflow discussion, checkout this.

To fix this, it might be recommended to add a -std=c++11 flag in the setup.py. That is you might want to change the definition of lfoldext in setup.py. 

lfoldext = Extension(
    'linearfold',
    ['linearfoldmodule.cc'],
    include_dirs=['LinearFold/src'],
    extra_compile_args=['-std=c++11']  # Add the C++11 flag here
)

To download linearfold-unofficial with this option, you can download from my fork of linear fold at branch patched_setup. To do this, do

git clone https://github.com/Cauch-BS/python-linearfold
cd python-linearfold
git checkout patched_setup
pip install .

If you encounter any further issues, it might be due to a lower version of clang. If this is the case, please print the version of clang, which can be done through clang -v. To update it, you can do command xcode-select–install. For more information on clang checkout here. Be warned that updating clang without checking dependencies could damage your MacOS system. I urge you to look more into this option if you are considering this. (I do not use MacOS, so I'm not sure about this)

If you any further questions, or if the install works fine after this, please let us know!