ImportError: No module named 'ast2vec.model2.source2bow'

[freddieptf]

I keep getting this error when i try to run the example and i'm not sure how to solve this

Traceback (most recent call last):
  File "/usr/local/bin/tmsc", line 11, in <module>
    load_entry_point('tmsc==0.1.2a0', 'console_scripts', 'tmsc')()
  File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 572, in load_entry_point
    return get_distribution(dist).load_entry_point(group, name)
  File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2752, in load_entry_point
    return ep.load()
  File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2405, in load
    return self.resolve()
  File "/usr/local/lib/python3.5/dist-packages/pkg_resources/__init__.py", line 2411, in resolve
    module = __import__(self.module_name, fromlist=['__name__'], level=0)
  File "/usr/local/lib/python3.5/dist-packages/tmsc/__init__.py", line 1, in <module>
    from tmsc.topic_detector import TopicDetector
  File "/usr/local/lib/python3.5/dist-packages/tmsc/topic_detector.py", line 6, in <module>
    from ast2vec.model2.source2bow import Uasts2BOW
ImportError: No module named 'ast2vec.model2.source2bow'

Also the docker instructions are kinda outdated(bblfsh:server is bblfsh:bblfshd now), and i had to tweak the dockerfile because the build fails with

g++ -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -Ibblfsh/libuast/ -I/usr/local/include -I/usr/local/include/libxml2 -I/usr/include -I/usr/include/libxml2 -I/usr/include/python3.5m -c bblfsh/pyuast.c -o build/temp.linux-x86_64-3.5/bblfsh/pyuast.o -std=c++11
    unable to execute 'g++': No such file or directory
    error: command 'g++' failed with exit status 1

    ----------------------------------------
Command "/usr/bin/python3 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-ikekfqcx/bblfsh/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-7g031q7j-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-ikekfqcx/bblfsh/

which is fixed if you install g++.

[vmarkovtsev]

Thanks for reporting! I will update Dockerfile, README and the code.

[vmarkovtsev]

All right, please try to run it in docker and without docker now.

[freddieptf]

I'm running this in a digital ocean box with 512mb RAM and now i get a MemoryError, it maxed out lol. I have crappy internet so i might not be able to do this locally to see if it works

tmsc https://github.com/apache/spark

INFO:bblfsh:Detected bblfsh server: 0.0.0.0:9432
INFO:enry:Fetching https://api.github.com/repos/src-d/enry/releases/latest
INFO:enry:Latest release resolved to enry_v1.6.2_linux_amd64.tar.gz
INFO:enry:Fetching https://github.com/src-d/enry/releases/download/v1.6.2/enry_v1.6.2_linux_amd64.tar.gz
INFO:enry:Extracting the binary
INFO:enry:Downloaded /home/fred/tmsc/enry
INFO:gcs-backend:Fetching https://storage.googleapis.com/models.cdn.sourced.tech/index.json?ignoreCache=1...
INFO:gcs-backend:Fetching https://storage.googleapis.com/models.cdn.sourced.tech/models%2Ftopics%2Fc70a7514-9257-4b33-b468-27a8588d4dfa.asdf...
[################################] 1451/1451 - 00:00:00
INFO:topics:Reading /home/fred/.source{d}/topics/default.asdf...
Traceback (most recent call last):
  File "/usr/local/bin/tmsc", line 11, in <module>
    load_entry_point('tmsc==0.1.3a0', 'console_scripts', 'tmsc')()
  File "/usr/local/lib/python3.5/dist-packages/tmsc-0.1.3a0-py3.5.egg/tmsc/__main__.py", line 61, in main
    query = args.query
  File "/usr/local/lib/python3.5/dist-packages/tmsc-0.1.3a0-py3.5.egg/tmsc/topic_detector.py", line 45, in __init__
  File "/usr/local/lib/python3.5/dist-packages/modelforge/model.py", line 121, in load
    self._load_tree(tree)
  File "/usr/local/lib/python3.5/dist-packages/ast2vec/topics.py", line 47, in _load_tree
    assemble_sparse_matrix(tree["matrix"]))
  File "/usr/local/lib/python3.5/dist-packages/modelforge/model.py", line 355, in assemble_sparse_matrix
    matrix = matrix_class(tuple(subtree["data"]), shape=subtree["shape"])
  File "/usr/local/lib/python3.5/dist-packages/scipy/sparse/compressed.py", line 66, in __init__
    self.data = np.array(data, copy=copy, dtype=dtype)
  File "/usr/local/lib/python3.5/dist-packages/asdf/tags/core/ndarray.py", line 275, in __array__
    return self._make_array()
  File "/usr/local/lib/python3.5/dist-packages/asdf/tags/core/ndarray.py", line 254, in _make_array
    shape, self._dtype, block.data,
  File "/usr/local/lib/python3.5/dist-packages/asdf/block.py", line 1115, in data
    self._fd, self._size, self._data_size)
  File "/usr/local/lib/python3.5/dist-packages/asdf/block.py", line 1029, in _read_data
    fd, used_size, data_size, self.input_compression)
  File "/usr/local/lib/python3.5/dist-packages/asdf/compression.py", line 190, in decompress
    buffer = np.empty((data_size,), np.uint8)
MemoryError

[freddieptf]

I might be missing something but i don't think there's a --recommended option --> docker exec -it bblfshd bblfshctl driver install --recommended. Ended up doing docker exec -it bblfshd bblfshctl driver install --all

[vmarkovtsev]

Weird, took it from https://doc.bblf.sh/user/getting-started.html#installing-the-drivers @src-d/language-analysis Is it a bug?

[vmarkovtsev]

OK, considering this as fixed.