search

scrapinghub / frontera

A scalable frontier for web crawlers
BSD 3-Clause "New" or "Revised" License
1.29k stars 216 forks source link

Failed install frontera in a cluster mode on Python 3.7 #362

Open phongtnit opened 5 years ago

phongtnit commented 5 years ago

Hello,

I install frontera with the following command: pip3 install frontera[distributed,kafka,hbase]

There is an error as shown in the below image

How should I do to fix the error and setup frontera in a cluster? Thanks for your help,

My environment: MacOS: 10.14.2 Python: 3.7.2 Scrapy: 1.5.1 Frontera: 0.8.0.1

Detail error:

`Building wheels for collected packages: happybase, thriftpy Running setup.py bdist_wheel for happybase ... done Stored in directory: /Users/phongnguyen/Library/Caches/pip/wheels/7a/d7/f7/447e781495238e394f04fe169f14549d31c63993e90152693b Running setup.py bdist_wheel for thriftpy ... error Complete output from command /usr/local/opt/python/bin/python3.7 -u -c "import setuptools, tokenize;file='/private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-install-hy9tc71e/thriftpy/setup.py';f=getattr(tokenize, 'open', open)(file);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, file, 'exec'))" bdist_wheel -d /private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-wheel-0zueewe3 --python-tag cp37: running bdist_wheel The [wheel] section is deprecated. Use [bdist_wheel] instead. running build running build_py creating build creating build/lib.macosx-10.14-x86_64-3.7 creating build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/server.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/hook.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/thrift.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/rpc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/utils.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/tornado.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy copying thriftpy/_compat.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport copying thriftpy/transport/_ssl.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport copying thriftpy/transport/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport copying thriftpy/transport/sslsocket.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport copying thriftpy/transport/socket.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/binary.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/compact.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/exc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/multiplex.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol copying thriftpy/protocol/json.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser copying thriftpy/parser/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser copying thriftpy/parser/parser.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser copying thriftpy/parser/exc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser copying thriftpy/parser/lexer.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib copying thriftpy/contrib/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/memory copying thriftpy/transport/memory/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/memory creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/buffered copying thriftpy/transport/buffered/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/buffered creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/framed copying thriftpy/transport/framed/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/framed creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking copying thriftpy/contrib/tracking/init.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking copying thriftpy/contrib/tracking/tracker.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking copying thriftpy/contrib/tracking/tracking.thrift -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking running build_ext building 'thriftpy.transport.cybase' extension creating build/temp.macosx-10.14-x86_64-3.7 creating build/temp.macosx-10.14-x86_64-3.7/thriftpy creating build/temp.macosx-10.14-x86_64-3.7/thriftpy/transport clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c thriftpy/transport/cybase.c -o build/temp.macosx-10.14-x86_64-3.7/thriftpy/transport/cybase.o thriftpy/transport/cybase.c:3127:24: error: no member named 'exc_type' in 'struct _ts' tmp_type = tstate->exc_type;


  thriftpy/transport/cybase.c:3128:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      tmp_value = tstate->exc_value;
                          ^~~~~~~~~
                          curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3129:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      tmp_tb = tstate->exc_traceback;
                       ^~~~~~~~~~~~~
                       curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3130:13: error: no member named 'exc_type' in 'struct _ts'
      tstate->exc_type = local_type;
      ~~~~~~  ^
  thriftpy/transport/cybase.c:3131:13: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      tstate->exc_value = local_value;
              ^~~~~~~~~
              curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3132:13: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      tstate->exc_traceback = local_tb;
              ^~~~~~~~~~~~~
              curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3154:24: error: no member named 'exc_type' in 'struct _ts'
      tmp_type = tstate->exc_type;
                 ~~~~~~  ^
  thriftpy/transport/cybase.c:3155:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      tmp_value = tstate->exc_value;
                          ^~~~~~~~~
                          curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3156:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      tmp_tb = tstate->exc_traceback;
                       ^~~~~~~~~~~~~
                       curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3157:13: error: no member named 'exc_type' in 'struct _ts'
      tstate->exc_type = *type;
      ~~~~~~  ^
  thriftpy/transport/cybase.c:3158:13: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      tstate->exc_value = *value;
              ^~~~~~~~~
              curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3159:13: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      tstate->exc_traceback = *tb;
              ^~~~~~~~~~~~~
              curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3178:21: error: no member named 'exc_type' in 'struct _ts'
      *type = tstate->exc_type;
              ~~~~~~  ^
  thriftpy/transport/cybase.c:3179:22: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      *value = tstate->exc_value;
                       ^~~~~~~~~
                       curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3180:19: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      *tb = tstate->exc_traceback;
                    ^~~~~~~~~~~~~
                    curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3187:24: error: no member named 'exc_type' in 'struct _ts'
      tmp_type = tstate->exc_type;
                 ~~~~~~  ^
  thriftpy/transport/cybase.c:3188:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
      tmp_value = tstate->exc_value;
                          ^~~~~~~~~
                          curexc_value
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
      PyObject *curexc_value;
                ^
  thriftpy/transport/cybase.c:3189:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
      tmp_tb = tstate->exc_traceback;
                       ^~~~~~~~~~~~~
                       curexc_traceback
  /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
      PyObject *curexc_traceback;
                ^
  thriftpy/transport/cybase.c:3190:13: error: no member named 'exc_type' in 'struct _ts'
      tstate->exc_type = type;
      ~~~~~~  ^
  fatal error: too many errors emitted, stopping now [-ferror-limit=]
  20 errors generated.
  error: command 'clang' failed with exit status 1

  ----------------------------------------
  Failed building wheel for thriftpy
  Running setup.py clean for thriftpy
Successfully built happybase
Failed to build thriftpy
Installing collected packages: ply, thriftpy, happybase, kafka-python
  Running setup.py install for thriftpy ... error
    Complete output from command /usr/local/opt/python/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-install-hy9tc71e/thriftpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-record-_zzfdmzn/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_py
    creating build
    creating build/lib.macosx-10.14-x86_64-3.7
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/server.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/hook.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/thrift.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/rpc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/utils.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/tornado.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    copying thriftpy/_compat.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport
    copying thriftpy/transport/_ssl.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport
    copying thriftpy/transport/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport
    copying thriftpy/transport/sslsocket.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport
    copying thriftpy/transport/socket.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/binary.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/compact.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/exc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/multiplex.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    copying thriftpy/protocol/json.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/protocol
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser
    copying thriftpy/parser/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser
    copying thriftpy/parser/parser.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser
    copying thriftpy/parser/exc.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser
    copying thriftpy/parser/lexer.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/parser
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib
    copying thriftpy/contrib/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/memory
    copying thriftpy/transport/memory/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/memory
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/buffered
    copying thriftpy/transport/buffered/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/buffered
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/framed
    copying thriftpy/transport/framed/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/transport/framed
    creating build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking
    copying thriftpy/contrib/tracking/__init__.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking
    copying thriftpy/contrib/tracking/tracker.py -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking
    copying thriftpy/contrib/tracking/tracking.thrift -> build/lib.macosx-10.14-x86_64-3.7/thriftpy/contrib/tracking
    running build_ext
    building 'thriftpy.transport.cybase' extension
    creating build/temp.macosx-10.14-x86_64-3.7
    creating build/temp.macosx-10.14-x86_64-3.7/thriftpy
    creating build/temp.macosx-10.14-x86_64-3.7/thriftpy/transport
    clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -I/usr/local/include -I/usr/local/opt/openssl/include -I/usr/local/opt/sqlite/include -I/usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c thriftpy/transport/cybase.c -o build/temp.macosx-10.14-x86_64-3.7/thriftpy/transport/cybase.o
    thriftpy/transport/cybase.c:3127:24: error: no member named 'exc_type' in 'struct _ts'
        tmp_type = tstate->exc_type;
                   ~~~~~~  ^
    thriftpy/transport/cybase.c:3128:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        tmp_value = tstate->exc_value;
                            ^~~~~~~~~
                            curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3129:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        tmp_tb = tstate->exc_traceback;
                         ^~~~~~~~~~~~~
                         curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3130:13: error: no member named 'exc_type' in 'struct _ts'
        tstate->exc_type = local_type;
        ~~~~~~  ^
    thriftpy/transport/cybase.c:3131:13: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        tstate->exc_value = local_value;
                ^~~~~~~~~
                curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3132:13: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        tstate->exc_traceback = local_tb;
                ^~~~~~~~~~~~~
                curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3154:24: error: no member named 'exc_type' in 'struct _ts'
        tmp_type = tstate->exc_type;
                   ~~~~~~  ^
    thriftpy/transport/cybase.c:3155:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        tmp_value = tstate->exc_value;
                            ^~~~~~~~~
                            curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3156:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        tmp_tb = tstate->exc_traceback;
                         ^~~~~~~~~~~~~
                         curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3157:13: error: no member named 'exc_type' in 'struct _ts'
        tstate->exc_type = *type;
        ~~~~~~  ^
    thriftpy/transport/cybase.c:3158:13: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        tstate->exc_value = *value;
                ^~~~~~~~~
                curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3159:13: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        tstate->exc_traceback = *tb;
                ^~~~~~~~~~~~~
                curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3178:21: error: no member named 'exc_type' in 'struct _ts'
        *type = tstate->exc_type;
                ~~~~~~  ^
    thriftpy/transport/cybase.c:3179:22: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        *value = tstate->exc_value;
                         ^~~~~~~~~
                         curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3180:19: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        *tb = tstate->exc_traceback;
                      ^~~~~~~~~~~~~
                      curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3187:24: error: no member named 'exc_type' in 'struct _ts'
        tmp_type = tstate->exc_type;
                   ~~~~~~  ^
    thriftpy/transport/cybase.c:3188:25: error: no member named 'exc_value' in 'struct _ts'; did you mean 'curexc_value'?
        tmp_value = tstate->exc_value;
                            ^~~~~~~~~
                            curexc_value
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:240:15: note: 'curexc_value' declared here
        PyObject *curexc_value;
                  ^
    thriftpy/transport/cybase.c:3189:22: error: no member named 'exc_traceback' in 'struct _ts'; did you mean 'curexc_traceback'?
        tmp_tb = tstate->exc_traceback;
                         ^~~~~~~~~~~~~
                         curexc_traceback
    /usr/local/Cellar/python/3.7.2_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m/pystate.h:241:15: note: 'curexc_traceback' declared here
        PyObject *curexc_traceback;
                  ^
    thriftpy/transport/cybase.c:3190:13: error: no member named 'exc_type' in 'struct _ts'
        tstate->exc_type = type;
        ~~~~~~  ^
    fatal error: too many errors emitted, stopping now [-ferror-limit=]
    20 errors generated.
    error: command 'clang' failed with exit status 1

    ----------------------------------------
Command "/usr/local/opt/python/bin/python3.7 -u -c "import setuptools, tokenize;__file__='/private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-install-hy9tc71e/thriftpy/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-record-_zzfdmzn/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /private/var/folders/0l/3k2zrhsn01b1sy7nktflll640000gn/T/pip-install-hy9tc71e/thriftpy/`
hesingon commented 5 years ago

My problem is similar, it's something wrong with cityhash. below are the terminal messages

 building 'cityhash' extension
    creating build
    creating build/temp.macosx-10.14-intel-2.7
    creating build/temp.macosx-10.14-intel-2.7/src
    cc -fno-strict-aliasing -fno-common -dynamic -g -Os -pipe -fno-common -fno-strict-aliasing -fwrapv -DENABLE_DTRACE -DMACOSX -DNDEBUG -Wall -Wstrict-prototypes -Wshorten-64-to-32 -DNDEBUG -g -fwrapv -Os -Wall -Wstrict-prototypes -DENABLE_DTRACE -arch i386 -arch x86_64 -pipe -Iinclude -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 -c src/city.cc -o build/temp.macosx-10.14-intel-2.7/src/city.o -O3 -Wno-unused-value -Wno-unused-function -msse4.2

and then follows with a lot of errors, obviously it can't import certain C language libraries


Command "/Users/Darius/.virtualenvs/frontera/bin/python -u -c "import setuptools, tokenize;__file__='/private/var/folders/ds/mm772gpj1zn9yy1sflxx_stw0000gn/T/pip-install-TmQYm7/cityhash/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /private/var/folders/ds/mm772gpj1zn9yy1sflxx_stw0000gn/T/pip-record-2gcw2r/install-record.txt --single-version-externally-managed --compile --install-headers /Users/Darius/.virtualenvs/frontera/include/site/python2.7/cityhash" failed with error code 1 in /private/var/folders/ds/mm772gpj1zn9yy1sflxx_stw0000gn/T/pip-install-TmQYm7/cityhash/```

Please help point out how to fix the problem, a lot of thanks!
sibiryakov commented 5 years ago

The solution is to switch to thriftpy2. But it seems happybase was abandoned, and we will need either to monkey patch thriftpy or fork happybase and use it for Frontera. I'm more keen to fork it.

gesof commented 4 years ago

So, still stuck in <No module named 'thriftpy'>. Jun 16th, 2020

sibiryakov commented 4 years ago

try master https://travis-ci.org/github/scrapinghub/frontera/builds/618649062, we have switched on thriftpy2 and tests are passing

gesof commented 4 years ago

Yes, it works by installing frontera directly from github:

pip install git+https://github.com/scrapinghub/frontera.git@master

Please make it available for installing it as described in the documentation.