RasaHQ / rasa

💬 Open source machine learning framework to automate text- and voice-based conversations: NLU, dialogue management, connect to Slack, Facebook, and more - Create chatbots and voice assistants
https://rasa.com/docs/rasa/
Apache License 2.0
18.79k stars 4.62k forks source link

Getting RASA to Work on the Raspberry Pi 4 (Buster) #4603

Closed TheCedarPrince closed 4 years ago

TheCedarPrince commented 5 years ago

Rasa version: 1.3.8

Rasa X version: N/A

Python version: 3.5

Operating system (windows, osx, ...): Raspbian Buster

Issue:

I am in the process of trying to install Rasa on my Raspberry Pi 4 (4GB RAM edition) and I continue running into an error with installing or-tools stating:

  ERROR: Could not find a version that satisfies the requirement ortools (from mesh-tensorflow->tensor2tensor~=1.14.0->rasa) (from versions: none)
ERROR: No matching distribution found for ortools (from mesh-tensorflow->tensor2tensor~=1.14.0->rasa)


Any suggestions on any potential work-arounds?

Here is what I tried so far:

Another thought was to build a wheel for or-tools that could be compatible with arm7l architecture, but I do not know how feasible this is - it could be bundled with future releases of rasa.



Replication of Error Procedure

Here are the steps I took to get to this point; follow it completely to replicate my error:

  1. Freshly install a new version of Raspbian Lite (Buster) on an SD Card

  2. Run the following to update the Raspberry Pi:

    sudo apt-get update && 
    sudo apt-get upgrade -y &&
    sudo apt-get dist-upgrade -y
  3. Install Berryconda package manager:

    wget https://github.com/jjhelmus/berryconda/releases/download/v2.0.0/Berryconda3-2.0.0-Linux-armv7l.sh &&
    chmod +x Berryconda3-2.0.0-Linux-armv7l.sh &&
    ./Berryconda3-2.0.0-Linux-armv7l.sh && sudo reboot
  4. Add RPI channel into conda:

    conda config --add channels rpi

  5. Install Python3.5 into conda:

    conda install python=3.5 -y

  6. Set-Up conda environment (we set-up with 3.5 so as to not encounter an error with open-cv):

    A. Create conda environment: conda create --name alpha python=3.5 pip -y

    B. Update all packages in conda environment:

    conda update --all -y &&
    source activate alpha &&
    pip install --upgrade pip setuptools wheel
  7. Install rasa:

    A. Install required system packages:

    sudo apt-get install libatlas-base-dev python3-dev python3-pip libhdf5-dev -y

    B. Reboot the computer

    sudo reboot

    C. Install rasa

    source activate alpha && pip install rasa

Error (including full traceback):

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting rasa
  Using cached https://files.pythonhosted.org/packages/94/04/183500ce584fc4dce09ccdb80e192f90ae5ba61553ace9e17b3f77c94a04/rasa-1.3.9-py3-none-any.whl
Collecting python-socketio>=4.3.1 (from rasa)
  Downloading https://files.pythonhosted.org/packages/35/b0/22c3f785f23fec5c7a815f47c55d7e7946a67ae2129ff604148e939d3bdb/python_socketio-4.3.1-py2.py3-none-any.whl (49kB)
Collecting aiohttp~=3.5 (from rasa)
  Downloading https://files.pythonhosted.org/packages/c2/f7/f0ad3dbace4762fef5d80aa4124b41bf218e4c4dd6d387a86cede707d9a4/aiohttp-3.6.2-py3-none-any.whl (441kB)
Collecting packaging~=19.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/cf/94/9672c2d4b126e74c4496c6b3c58a8b51d6419267be9e70660ba23374c875/packaging-19.2-py2.py3-none-any.whl
Collecting async-generator~=1.10 (from rasa)
  Using cached https://files.pythonhosted.org/packages/71/52/39d20e03abd0ac9159c162ec24b93fbcaa111e8400308f2465432495ca2b/async_generator-1.10-py3-none-any.whl
Collecting scikit-learn~=0.20.0 (from rasa)
  Using cached https://www.piwheels.org/simple/scikit-learn/scikit_learn-0.20.4-cp35-cp35m-linux_armv7l.whl
Collecting networkx~=2.3 (from rasa)
  Downloading https://files.pythonhosted.org/packages/85/08/f20aef11d4c343b557e5de6b9548761811eb16e438cee3d32b1c66c8566b/networkx-2.3.zip (1.7MB)
Collecting tensorflow-probability~=0.7.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl
Collecting fbmessenger~=6.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/bd/e9/646684226176782b9e3b7dd5b35d7ecfd1d13cba24ad2e33255079921aab/fbmessenger-6.0.0-py2.py3-none-any.whl
Collecting tensorflow~=1.14.0 (from rasa)
  Downloading https://www.piwheels.org/simple/tensorflow/tensorflow-1.14.0-cp35-none-linux_armv7l.whl (100.7MB)
Collecting sanic-jwt~=1.3 (from rasa)
  Downloading https://www.piwheels.org/simple/sanic-jwt/sanic_jwt-1.3.2-py3-none-any.whl
Collecting SQLAlchemy~=1.3.0 (from rasa)
  Downloading https://www.piwheels.org/simple/sqlalchemy/SQLAlchemy-1.3.10-cp35-cp35m-linux_armv7l.whl (1.2MB)
Collecting jsonpickle~=1.1 (from rasa)
  Using cached https://files.pythonhosted.org/packages/07/07/c157520a3ebd166c8c24c6ae0ecae7c3968eb4653ff0e5af369bb82f004d/jsonpickle-1.2-py2.py3-none-any.whl
Collecting pymongo[srv,tls]~=3.8 (from rasa)
  Downloading https://www.piwheels.org/simple/pymongo/pymongo-3.9.0-cp35-cp35m-linux_armv7l.whl (433kB)
Collecting python-telegram-bot~=11.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/84/6c/47932a4041ee76650ad1f45a80e1422077e1e99c08a4d7a61cfbe5393d41/python_telegram_bot-11.1.0-py2.py3-none-any.whl (326kB)
Collecting sanic-cors~=0.9.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/99/a6/1f2dce79cb38f0c7d8c846f7456ccd2a471d504cedb645b585e25406d737/Sanic_Cors-0.9.9.post3-py2.py3-none-any.whl
Collecting mattermostwrapper~=2.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/93/70/203660597d12788e958dd691aa11c3c29caa075eadb2ce94d2eb53099d1b/mattermostwrapper-2.1-py2.py3-none-any.whl
Collecting tensor2tensor~=1.14.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/20/eb/7159c5a12880c496893a5bf7242c23fdb4eed47ade683d111f2f2d4009b4/tensor2tensor-1.14.1-py2.py3-none-any.whl
Collecting absl-py>=0.8.0 (from rasa)
  Downloading https://www.piwheels.org/simple/absl-py/absl_py-0.8.1-py3-none-any.whl (121kB)
Collecting pykwalify~=1.7.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/36/9f/612de8ca540bd24d604f544248c4c46e9db76f6ea5eb75fb4244da6ebbf0/pykwalify-1.7.0-py2.py3-none-any.whl (40kB)
Collecting coloredlogs~=10.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/08/0f/7877fc42fff0b9d70b6442df62d53b3868d3a6ad1b876bdb54335b30ff23/coloredlogs-10.0-py2.py3-none-any.whl (47kB)
Requirement already satisfied: setuptools>=41.0.0 in ./berryconda3/envs/alpha/lib/python3.5/site-packages (from rasa) (41.4.0)
Collecting apscheduler~=3.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/09/ff/d5b0e81846cd5e92d02e5f2682b78c73a5d9d61bc1eae32cea5ac15c0d47/APScheduler-3.6.1-py2.py3-none-any.whl
Collecting simplejson~=3.16 (from rasa)
  Using cached https://www.piwheels.org/simple/simplejson/simplejson-3.16.0-cp35-cp35m-linux_armv7l.whl
Collecting boto3~=1.9 (from rasa)
  Using cached https://files.pythonhosted.org/packages/8c/49/cc8d900df85f69bce673510aacd0473aba958244516829d422720f584632/boto3-1.9.248-py2.py3-none-any.whl
Collecting pydot~=1.4 (from rasa)
  Using cached https://files.pythonhosted.org/packages/33/d1/b1479a770f66d962f545c2101630ce1d5592d90cb4f083d38862e93d16d2/pydot-1.4.1-py2.py3-none-any.whl
Collecting attrs>=18 (from rasa)
  Using cached https://files.pythonhosted.org/packages/6b/e8/2ecaf86b128a34e225807f03b22664302937ab826bd3b7eccab6754d29ea/attrs-19.2.0-py2.py3-none-any.whl
Collecting numpy~=1.16 (from rasa)
  Downloading https://www.piwheels.org/simple/numpy/numpy-1.17.2-cp35-cp35m-linux_armv7l.whl (9.1MB)
Collecting twilio~=6.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/0b/1e/e5b2acb5f73578aef493ce79cf3bab92f2236a4da9693409fc804caebf63/twilio-6.31.1-py2.py3-none-any.whl (1.0MB)
Collecting python-dateutil~=2.8 (from rasa)
  Using cached https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl
Collecting requests>=2.20 (from rasa)
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Collecting colorclass~=2.2 (from rasa)
  Using cached https://www.piwheels.org/simple/colorclass/colorclass-2.2.0-py3-none-any.whl
Collecting rasa-sdk~=1.3.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/b2/95/ad9f2448be31be381ed2b1b01f0564ac505deac549423fa05a183aa9193b/rasa_sdk-1.3.3-py2.py3-none-any.whl
Collecting ruamel.yaml~=0.15.0 (from rasa)
  Using cached https://www.piwheels.org/simple/ruamel-yaml/ruamel.yaml-0.15.100-cp35-cp35m-linux_armv7l.whl
Collecting pika~=1.0.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/78/1a/28c98ee8b211be21d4a9f4ef1687c4d36f9302d47fcc28b81f9591abf6d8/pika-1.0.1-py2.py3-none-any.whl
Collecting pytz~=2019.1 (from rasa)
  Using cached https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl
Collecting python-engineio>=3.9.3 (from rasa)
  Using cached https://files.pythonhosted.org/packages/2b/20/8e3ba16102ae2e245d70d9cb9fa48b076253fdb036dc43eea142294c2897/python_engineio-3.9.3-py2.py3-none-any.whl
Collecting jsonschema~=2.6 (from rasa)
  Using cached https://files.pythonhosted.org/packages/77/de/47e35a97b2b05c2fadbec67d44cfcdcd09b8086951b331d82de90d2912da/jsonschema-2.6.0-py2.py3-none-any.whl
Collecting questionary>=1.1.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/c2/f9/8a6e7fce60566b3bcdc5ad0923916f38a65bca630ce3647251e672308bdf/questionary-1.3.0-py3-none-any.whl
Collecting slackclient~=1.3 (from rasa)
  Using cached https://www.piwheels.org/simple/slackclient/slackclient-1.3.2-py2.py3-none-any.whl
Collecting rocketchat-API~=0.6.0 (from rasa)
  Downloading https://files.pythonhosted.org/packages/e2/36/29d991028e2943b6a12abe12e3114a4cce13872283df7c7b784ee4cb59ed/rocketchat_API-0.6.34-py3-none-any.whl
Collecting terminaltables~=3.1 (from rasa)
  Using cached https://www.piwheels.org/simple/terminaltables/terminaltables-3.1.0-py3-none-any.whl
Collecting redis~=3.3.5 (from rasa)
  Downloading https://files.pythonhosted.org/packages/cc/ed/c7447328a3d9fb26961ca4ee877629a9514705b9442d3179456cb860c70f/redis-3.3.10-py2.py3-none-any.whl (66kB)
Collecting sanic~=19.3.1 (from rasa)
  Using cached https://files.pythonhosted.org/packages/90/11/31382617b33f89df0caca396f104628d256689a58a63c75d3e46663a8c8f/sanic-19.3.1-py3-none-any.whl
Collecting matplotlib~=3.0 (from rasa)
  Downloading https://www.piwheels.org/simple/matplotlib/matplotlib-3.0.3-cp35-cp35m-linux_armv7l.whl (10.9MB)
Collecting gast==0.2.2 (from rasa)
  Downloading https://www.piwheels.org/simple/gast/gast-0.2.2-py3-none-any.whl
Collecting colorhash~=1.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/0e/e1/50dbc513aa74e99eca4c47f2a8206711f0bec436fdddd95eebaf7eaaa1aa/colorhash-1.0.2-py2.py3-none-any.whl
Collecting sklearn-crfsuite~=0.3.6 (from rasa)
  Downloading https://files.pythonhosted.org/packages/25/74/5b7befa513482e6dee1f3dd68171a6c9dfc14c0eaa00f885ffeba54fe9b0/sklearn_crfsuite-0.3.6-py2.py3-none-any.whl
Collecting gevent~=1.4 (from rasa)
  Downloading https://www.piwheels.org/simple/gevent/gevent-1.4.0-cp35-cp35m-linux_armv7l.whl (5.6MB)
Collecting kafka-python~=1.4 (from rasa)
  Downloading https://files.pythonhosted.org/packages/49/c9/9863483a1353700ba87821b4f39085eb18fd1bcbb1e954c697177d67f03f/kafka_python-1.4.7-py2.py3-none-any.whl (266kB)
Collecting PyJWT~=1.7 (from rasa)
  Using cached https://files.pythonhosted.org/packages/87/8b/6a9f14b5f781697e51259d81657e6048fd31a113229cf346880bb7545565/PyJWT-1.7.1-py2.py3-none-any.whl
Collecting webexteamssdk~=1.1 (from rasa)
  Downloading https://www.piwheels.org/simple/webexteamssdk/webexteamssdk-1.2-py3-none-any.whl (82kB)
Collecting scipy~=1.2 (from rasa)
  Downloading https://www.piwheels.org/simple/scipy/scipy-1.3.1-cp35-cp35m-linux_armv7l.whl (37.9MB)
Collecting tqdm~=4.0 (from rasa)
  Using cached https://files.pythonhosted.org/packages/e1/c1/bc1dba38b48f4ae3c4428aea669c5e27bd5a7642a74c8348451e0bd8ff86/tqdm-4.36.1-py2.py3-none-any.whl
Collecting six>=1.9.0 (from python-socketio>=4.3.1->rasa)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting multidict<5.0,>=4.5 (from aiohttp~=3.5->rasa)
  Downloading https://www.piwheels.org/simple/multidict/multidict-4.5.2-cp35-cp35m-linux_armv7l.whl (301kB)
Collecting yarl<2.0,>=1.0 (from aiohttp~=3.5->rasa)
  Downloading https://www.piwheels.org/simple/yarl/yarl-1.3.0-cp35-cp35m-linux_armv7l.whl (227kB)
Collecting chardet<4.0,>=2.0 (from aiohttp~=3.5->rasa)
  Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB)
Collecting typing-extensions>=3.6.5; python_version < "3.7" (from aiohttp~=3.5->rasa)
  Downloading https://files.pythonhosted.org/packages/27/aa/bd1442cfb0224da1b671ab334d3b0a4302e4161ea916e28904ff9618d471/typing_extensions-3.7.4-py3-none-any.whl
Collecting async-timeout<4.0,>=3.0 (from aiohttp~=3.5->rasa)
  Downloading https://files.pythonhosted.org/packages/e1/1e/5a4441be21b0726c4464f3f23c8b19628372f606755a9d2e46c187e65ec4/async_timeout-3.0.1-py3-none-any.whl
Collecting idna-ssl>=1.0; python_version < "3.7" (from aiohttp~=3.5->rasa)
  Downloading https://www.piwheels.org/simple/idna-ssl/idna_ssl-1.1.0-py3-none-any.whl
Collecting pyparsing>=2.0.2 (from packaging~=19.0->rasa)
  Downloading https://files.pythonhosted.org/packages/11/fa/0160cd525c62d7abd076a070ff02b2b94de589f1a9789774f17d7c54058e/pyparsing-2.4.2-py2.py3-none-any.whl (65kB)
Collecting decorator>=4.3.0 (from networkx~=2.3->rasa)
  Downloading https://files.pythonhosted.org/packages/5f/88/0075e461560a1e750a0dcbf77f1d9de775028c37a19a346a6c565a257399/decorator-4.4.0-py2.py3-none-any.whl
Collecting cloudpickle>=0.6.1 (from tensorflow-probability~=0.7.0->rasa)
  Downloading https://files.pythonhosted.org/packages/c1/49/334e279caa3231255725c8e860fa93e72083567625573421db8875846c14/cloudpickle-1.2.2-py2.py3-none-any.whl
Collecting astor>=0.6.0 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/d1/4f/950dfae467b384fc96bc6469de25d832534f6b4441033c39f914efd13418/astor-0.8.0-py2.py3-none-any.whl
Collecting tensorflow-estimator<1.15.0rc0,>=1.14.0rc0 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/3c/d5/21860a5b11caf0678fbc8319341b0ae21a07156911132e0e71bffed0510d/tensorflow_estimator-1.14.0-py2.py3-none-any.whl (488kB)
Collecting keras-preprocessing>=1.0.5 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/28/6a/8c1f62c37212d9fc441a7e26736df51ce6f0e38455816445471f10da4f0a/Keras_Preprocessing-1.1.0-py2.py3-none-any.whl (41kB)
Collecting protobuf>=3.6.1 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/ad/c2/86c65136e280607ddb2e5dda19e2953c1174f9919b557d1d154574481de4/protobuf-3.10.0-py2.py3-none-any.whl (434kB)
Requirement already satisfied: wheel>=0.26 in ./berryconda3/envs/alpha/lib/python3.5/site-packages (from tensorflow~=1.14.0->rasa) (0.33.6)
Collecting grpcio>=1.8.6 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/14/ea/227e7055604ec319544def1b83a8506f084fc2e985b1ac4b84e04bf2c78e/grpcio-1.24.1-cp35-cp35m-linux_armv7l.whl (12.7MB)
Collecting tensorboard<1.15.0,>=1.14.0 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/91/2d/2ed263449a078cd9c8a9ba50ebd50123adf1f8cfbea1492f9084169b89d9/tensorboard-1.14.0-py3-none-any.whl (3.1MB)
Collecting termcolor>=1.1.0 (from tensorflow~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/termcolor/termcolor-1.1.0-py3-none-any.whl
Collecting keras-applications>=1.0.6 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/71/e3/19762fdfc62877ae9102edf6342d71b28fbfd9dea3d2f96a882ce099b03f/Keras_Applications-1.0.8-py3-none-any.whl (50kB)
Collecting wrapt>=1.11.1 (from tensorflow~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/wrapt/wrapt-1.11.2-cp35-cp35m-linux_armv7l.whl (66kB)
Collecting google-pasta>=0.1.6 (from tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/d0/33/376510eb8d6246f3c30545f416b2263eee461e40940c2a4413c711bdf62d/google_pasta-0.1.7-py3-none-any.whl (52kB)
Collecting dnspython<2.0.0,>=1.16.0; extra == "srv" (from pymongo[srv,tls]~=3.8->rasa)
  Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB)
Collecting future>=0.16.0 (from python-telegram-bot~=11.0->rasa)
  Downloading https://www.piwheels.org/simple/future/future-0.18.0-py3-none-any.whl (490kB)
Requirement already satisfied: certifi in ./berryconda3/envs/alpha/lib/python3.5/site-packages (from python-telegram-bot~=11.0->rasa) (2018.8.24)
Collecting cryptography (from python-telegram-bot~=11.0->rasa)
  Downloading https://www.piwheels.org/simple/cryptography/cryptography-2.7-cp35-cp35m-linux_armv7l.whl (782kB)
Collecting sanic-plugins-framework>=0.8.2 (from sanic-cors~=0.9.0->rasa)
  Downloading https://files.pythonhosted.org/packages/ac/a6/258bdd353c22c3ff7f130d1c788f874a88e48d306614d2f622f3bac2576b/Sanic_Plugins_Framework-0.8.2-py2.py3-none-any.whl
Collecting kfac (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/01/f0/4a7758f854a15b37d322827123ce58619d0f4270dd94f2dd30328f397339/kfac-0.2.0-py2.py3-none-any.whl (178kB)
Collecting gin-config (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/gin-config/gin_config-0.2.1-py3-none-any.whl
Collecting gunicorn (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl (112kB)
Collecting flask (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl (94kB)
Collecting mesh-tensorflow (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/f1/63/77b1b1e9a037aed5c6c42a264b6340093ddcf5a3cb06713f5b50bfb966b4/mesh_tensorflow-0.1.0-py2.py3-none-any.whl (221kB)
Collecting tensorflow-gan (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/3e/94/8903150ffafdd538b18e6d24dbd4f5a07105d9d31a29db49e728ae2a8b13/tensorflow_gan-1.0.0.dev0-py2.py3-none-any.whl (317kB)
Collecting bz2file (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/bz2file/bz2file-0.98-py3-none-any.whl
Collecting dopamine-rl (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/dopamine-rl/dopamine_rl-2.0.5-py3-none-any.whl (77kB)
Collecting gym (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/gym/gym-0.15.3-py3-none-any.whl (1.6MB)
Collecting pypng (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/pypng/pypng-0.0.20-py3-none-any.whl (67kB)
Collecting sympy (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/21/21/f4105795ca7f35c541d82c5b06be684dd2f5cb4f508fb487cd7aea4de776/sympy-1.4-py2.py3-none-any.whl (5.3MB)
Collecting oauth2client (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/95/a9/4f25a14d23f0786b64875b91784607c2277eff25d48f915e39ff0cff505a/oauth2client-4.1.3-py2.py3-none-any.whl (98kB)
Collecting tensorflow-datasets (from tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/6c/34/ff424223ed4331006aaa929efc8360b6459d427063dc59fc7b75d7e4bab3/tensorflow_datasets-1.2.0-py3-none-any.whl (2.3MB)
Collecting opencv-python (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/opencv-python/opencv_python-3.4.4.19-cp35-cp35m-linux_armv7l.whl (7.4MB)
Collecting google-api-python-client (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/google-api-python-client/google_api_python_client-1.7.11-py3-none-any.whl (56kB)
Collecting Pillow (from tensor2tensor~=1.14.0->rasa)
  WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))': /simple/pillow/
  Downloading https://www.piwheels.org/simple/pillow/Pillow-6.2.0-cp35-cp35m-linux_armv7l.whl (1.1MB)
Collecting h5py (from tensor2tensor~=1.14.0->rasa)
  Downloading https://www.piwheels.org/simple/h5py/h5py-2.10.0-cp35-cp35m-linux_armv7l.whl (3.5MB)
Collecting docopt>=0.6.2 (from pykwalify~=1.7.0->rasa)
  Downloading https://www.piwheels.org/simple/docopt/docopt-0.6.2-py2.py3-none-any.whl
Collecting PyYAML>=3.11 (from pykwalify~=1.7.0->rasa)
  Downloading https://www.piwheels.org/simple/pyyaml/PyYAML-5.1.2-cp35-cp35m-linux_armv7l.whl (45kB)
Collecting humanfriendly>=4.7 (from coloredlogs~=10.0->rasa)
  Downloading https://files.pythonhosted.org/packages/90/df/88bff450f333114680698dc4aac7506ff7cab164b794461906de31998665/humanfriendly-4.18-py2.py3-none-any.whl (73kB)
Collecting tzlocal>=1.2 (from apscheduler~=3.0->rasa)
  Downloading https://files.pythonhosted.org/packages/ef/99/53bd1ac9349262f59c1c421d8fcc2559ae8a5eeffed9202684756b648d33/tzlocal-2.0.0-py2.py3-none-any.whl
Collecting botocore<1.13.0,>=1.12.248 (from boto3~=1.9->rasa)
  Downloading https://files.pythonhosted.org/packages/50/5c/922cc8c2cdcb905e96bd73dc2064939bc9157e9e1d4c44e54e775cd2cddc/botocore-1.12.248-py2.py3-none-any.whl (5.7MB)
Collecting s3transfer<0.3.0,>=0.2.0 (from boto3~=1.9->rasa)
  Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB)
Collecting jmespath<1.0.0,>=0.7.1 (from boto3~=1.9->rasa)
  Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl
Collecting pysocks; python_version >= "3.0" (from twilio~=6.0->rasa)
  Downloading https://files.pythonhosted.org/packages/8d/59/b4572118e098ac8e46e399a1dd0f2d85403ce8bbaad9ec79373ed6badaf9/PySocks-1.7.1-py3-none-any.whl
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 (from requests>=2.20->rasa)
  Downloading https://files.pythonhosted.org/packages/e0/da/55f51ea951e1b7c63a579c09dd7db825bb730ec1fe9c0180fc77bfb31448/urllib3-1.25.6-py2.py3-none-any.whl (125kB)
Collecting idna<2.9,>=2.5 (from requests>=2.20->rasa)
  Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB)
Collecting flask-cors~=3.0 (from rasa-sdk~=1.3.0->rasa)
  Downloading https://files.pythonhosted.org/packages/78/38/e68b11daa5d613e3a91e4bf3da76c94ac9ee0d9cd515af9c1ab80d36f709/Flask_Cors-3.0.8-py2.py3-none-any.whl
Collecting ConfigArgParse~=0.14 (from rasa-sdk~=1.3.0->rasa)
  Downloading https://www.piwheels.org/simple/configargparse/ConfigArgParse-0.15.1-py3-none-any.whl
Collecting prompt-toolkit~=2.0 (from questionary>=1.1.0->rasa)
  Downloading https://files.pythonhosted.org/packages/87/61/2dfea88583d5454e3a64f9308a686071d58d59a55db638268a6413e1eb6d/prompt_toolkit-2.0.10-py3-none-any.whl (340kB)
Collecting websocket-client<0.55.0,>=0.35 (from slackclient~=1.3->rasa)
  Downloading https://files.pythonhosted.org/packages/26/2d/f749a5c82f6192d77ed061a38e02001afcba55fe8477336d26a950ab17ce/websocket_client-0.54.0-py2.py3-none-any.whl (200kB)
Collecting uvloop>=0.5.3; sys_platform != "win32" and implementation_name == "cpython" (from sanic~=19.3.1->rasa)
  Downloading https://www.piwheels.org/simple/uvloop/uvloop-0.13.0-cp35-cp35m-linux_armv7l.whl (3.2MB)
Collecting httptools>=0.0.10 (from sanic~=19.3.1->rasa)
  Downloading https://www.piwheels.org/simple/httptools/httptools-0.0.13-cp35-cp35m-linux_armv7l.whl (207kB)
Collecting aiofiles>=0.3.0 (from sanic~=19.3.1->rasa)
  Downloading https://files.pythonhosted.org/packages/cf/f2/a67a23bc0bb61d88f82aa7fb84a2fb5f278becfbdc038c5cbb36c31feaf1/aiofiles-0.4.0-py3-none-any.whl
Collecting ujson>=1.35; sys_platform != "win32" and implementation_name == "cpython" (from sanic~=19.3.1->rasa)
  Downloading https://www.piwheels.org/simple/ujson/ujson-1.35-cp35-cp35m-linux_armv7l.whl (72kB)
Collecting websockets<7.0,>=6.0 (from sanic~=19.3.1->rasa)
  Downloading https://www.piwheels.org/simple/websockets/websockets-6.0-cp35-cp35m-linux_armv7l.whl (92kB)
Collecting cycler>=0.10 (from matplotlib~=3.0->rasa)
  Downloading https://files.pythonhosted.org/packages/f7/d2/e07d3ebb2bd7af696440ce7e754c59dd546ffe1bbe732c8ab68b9c834e61/cycler-0.10.0-py2.py3-none-any.whl
Collecting kiwisolver>=1.0.1 (from matplotlib~=3.0->rasa)
  Downloading https://www.piwheels.org/simple/kiwisolver/kiwisolver-1.1.0-cp35-cp35m-linux_armv7l.whl (912kB)
Collecting python-crfsuite>=0.8.3 (from sklearn-crfsuite~=0.3.6->rasa)
  Downloading https://www.piwheels.org/simple/python-crfsuite/python_crfsuite-0.9.6-cp35-cp35m-linux_armv7l.whl (659kB)
Collecting tabulate (from sklearn-crfsuite~=0.3.6->rasa)
  Downloading https://www.piwheels.org/simple/tabulate/tabulate-0.8.5-py3-none-any.whl
Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" (from gevent~=1.4->rasa)
  Downloading https://www.piwheels.org/simple/greenlet/greenlet-0.4.15-cp35-cp35m-linux_armv7l.whl (42kB)
Collecting requests-toolbelt (from webexteamssdk~=1.1->rasa)
  Downloading https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl (54kB)
Collecting markdown>=2.6.8 (from tensorboard<1.15.0,>=1.14.0->tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/c0/4e/fd492e91abdc2d2fcb70ef453064d980688762079397f779758e055f6575/Markdown-3.1.1-py2.py3-none-any.whl (87kB)
Collecting werkzeug>=0.11.15 (from tensorboard<1.15.0,>=1.14.0->tensorflow~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/ce/42/3aeda98f96e85fd26180534d36570e4d18108d62ae36f87694b476b83d6f/Werkzeug-0.16.0-py2.py3-none-any.whl (327kB)
Collecting asn1crypto>=0.21.0 (from cryptography->python-telegram-bot~=11.0->rasa)
  Downloading https://files.pythonhosted.org/packages/6e/1e/fb0e487b5229e5fb7b15c6d00b4e8082a3414fe62b1da4c9a905b106e672/asn1crypto-1.1.0-py2.py3-none-any.whl (103kB)
Collecting cffi!=1.11.3,>=1.8 (from cryptography->python-telegram-bot~=11.0->rasa)
  Downloading https://www.piwheels.org/simple/cffi/cffi-1.12.3-cp35-cp35m-linux_armv7l.whl (310kB)
Collecting click>=5.1 (from flask->tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/fa/37/45185cb5abbc30d7257104c434fe0b07e5a195a6847506c074527aa599ec/Click-7.0-py2.py3-none-any.whl (81kB)
Collecting Jinja2>=2.10.1 (from flask->tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/65/e0/eb35e762802015cab1ccee04e8a277b03f1d8e53da3ec3106882ec42558b/Jinja2-2.10.3-py2.py3-none-any.whl (125kB)
Collecting itsdangerous>=0.24 (from flask->tensor2tensor~=1.14.0->rasa)
  Downloading https://files.pythonhosted.org/packages/76/ae/44b03b253d6fade317f32c24d100b3b35c2239807046a4c953c7b89fa49e/itsdangerous-1.1.0-py2.py3-none-any.whl
Collecting ortools (from mesh-tensorflow->tensor2tensor~=1.14.0->rasa)
  ERROR: Could not find a version that satisfies the requirement ortools (from mesh-tensorflow->tensor2tensor~=1.14.0->rasa) (from versions: none)
ERROR: No matching distribution found for ortools (from mesh-tensorflow->tensor2tensor~=1.14.0->rasa)

Command which led to error:

pip install rasa


If you have any questions, feel free to reach out - I hope we can get this working!

sara-tagger commented 5 years ago

Thanks for the issue, @wochinge will get back to you about it soon!

You may find help in the docs and the forum, too 🤗
wochinge commented 5 years ago

Hi @TheCedarPrince ,

thanks for your detailed description. However, I'm afraid we can't really do anything to help you with this problem. I know that @JulianGerhard21 managed to run it on a Raspberry PI, maybe he can help us out?

JulianGerhard21 commented 5 years ago

Hi,

indeed Rasa works properly on my Raspi 3 and 4 (Debian 9 and 10). However, I have not updated to 1.3.9 - I will do that tomorrow and give you some Feedback on it. I can for now say that I have built most of the main-dependencies like tf from source because these times compatibility is a Mess.

Sorry that I cant tell you more maybe I can tomorrow.

Regards Julian

TheCedarPrince commented 5 years ago

Hi @sara-tagger, @wochinge, and @JulianGerhard21.

First off, thanks all for the assistance and thoughts! @wochinge, I am glad you found the bug sufficiently documented - I'll keep it in that format for the future if I find more bugs!

@JulianGerhard21, interesting! I had no problem installing TF actually through a normal pip installation. It was just the or-tools that was giving me issues. If you can provide me the steps you took, I could try to replicate them on my end and do a write-up for your guys' wiki on how to install rasa on the Raspberry Pi 4. I can confirm that I was able to install and successfully run rasa-nlu on the Pi via the following steps:


  1. Freshly install a new version of Raspbian Lite (Buster) on an SD Card

  2. Run the following to update the Raspberry Pi:

    sudo apt-get update && 
    sudo apt-get upgrade -y &&
    sudo apt-get dist-upgrade -y
  3. Install system packages:

    sudo apt-get install libatlas-base-dev python3-dev python3-pip libhdf5-dev python-pip python-dev -y

  4. Reboot the computer using sudo reboot

  5. Install rasa-nlu using pip3 install rasa-nlu[spacy] (NOTE: This step takes a while as it is installing Spacy)

  6. Install spacy's models*

    python3 -m spacy link en_core_web_md en
    python3 -m spacy download en_core_web_md

    (NOTE: The model above is an

    English multi-task CNN trained on OntoNotes, with GloVe vectors trained on Common Crawl. Assigns word vectors, context-specific token vectors, POS tags, dependency parse and named entities.

    spacy has more information)

  7. Clone the Automaton repo onto the RPi and navigate to the rasa directory:

    git clone git@github.com:TheCedarPrince/automaton.git && 
    cd automaton/rasa
  8. Train the rasa-nlu model using make train-nlu

  9. Start the rasa-nlu server using make run-nlu

  10. Test by sending a local POST request to the server:


    curl XPOST localhost:5000/parse -d '{"query":"Hello", "project": "current"}'


    You should receive a json return listing the intents and their intent_ranking scheme (NOTE: You may receive an error saying something like curl: (6) Could not resolve host: XPOST - don't worry about this for now but continue to send a few additional queries to the server; it will take a moment to start working. Still trying to determine how to resolve this issue!)

Here is an example of what it looked like when I got everything running successfully:

image


But definitely agree, compatability is a mess; this was why I was trying to build it inside a conda environment (still seeing if that is a possibility as the architecture of conda may not allow us to sufficiently run rasa from a conda environment).

JulianGerhard21 commented 5 years ago

Hi @TheCedarPrince ,

I updated Rasa on my Raspi 4 with Debian 10 to version 1.3.9 "including" ortools. To achieve that, follow these steps:

  1. Clone this repo,
  2. Remove the dependency ortools (since it is not necessary for the usage of mesh in Rasa as far as I have seen) in setup.py
  3. Do a pip3 install --user . in the directory
  4. Update Rasa.

I think the more heavy thing is to install Spacy on the Pi :-)

I hope that helped!

Regards Julian

JulianGerhard21 commented 5 years ago

Small update: Installing SpaCy on a Raspi could be achieved by doing:

  1. Following these steps
  2. Clone the spaCy Repo
  3. Following these steps

and done!

Regards

TheCedarPrince commented 5 years ago

Hey @JulianGerhard21,

Thanks for the help! I shall attempt this sometime soon so - then, per your input, we can close the issue provided I can replicate everything. Which I think I will be able to!

With regards to spacy, I actually determined an easier way of doing that by using these steps:

  1. Install system packages:

    sudo apt-get install libatlas-base-dev python3-dev python3-pip libhdf5-dev python-pip python-dev -y

  2. Reboot the computer using sudo reboot

  3. Install rasa-nlu using pip3 install rasa-nlu[spacy]

  4. Install spacy's models*

    python3 -m spacy link en_core_web_md en
    python3 -m spacy download en_core_web_md

Using the python3-dev and python3-pip dependencies, it finds the latest Tensorflow ARM compatible Python wheel.

wochinge commented 5 years ago

@JulianGerhard21 Thanks for your fast and competent help! I have seen the question regarding running rasa on a raspberry PI a couple of times in the forum as well. Would you be up for writing a blog post with instructions? We could feature that in the community newsletter / forum / social media and so on, to give it visibility in the whole community.

JulianGerhard21 commented 5 years ago

Hi @wochinge ,

of course, I would be happy to do that. I am not quite sure if you are in the community@rasa.com dispatcher but Emma already asked the same about my SmartSpeaker project on the Raspi. Is there any template for a tutorial / blog entry that I should respect?

I should add that I won't be able to start this week but next week - is that okay?

Regards Julian

TheCedarPrince commented 5 years ago

Hey @JulianGerhard21, I can partially confirm your solution that you offered earlier.

Following your instructions for installing mesh, I am now past the or-tools issue! :tada: :tada: :tada:

However, I am still having issues with spaCy. As it would turn out, I was installing a very old version of spaCy onto my Raspberry Pi 4 using pip install rasa-nlu[spacy]. So this did work, but I am not up to date with all my systems.

I attempted the instructions you left on installing spaCy and was not able to replicate your solution for installing spaCy; could you enumerate the steps you took and how you accomplished this? I tested installing the package with Python 3.5 and 3.7 as well as inside of package management environments (i.e. conda and venv).

What I continue to get stuck on specifically is installing the blis package for spaCy - I tried out the instructions you suggested from here (see reference below) but no luck. I felt the instructions given in the issue you referenced were too vague for me to understand...

Here was the procedure I followed:

  1. Get the 2.1.8 version of spaCy: git clone -b v2.1.8 https://github.com/explosion/spaCy.git
  2. Update spaCy's requirements to the master branch: cd spaCy && rm requirements && wget https://raw.githubusercontent.com/explosion/spaCy/master/requirements.txt
  3. Assign the env variable: BLIS_ARCH=generic
  4. Inside of spaCy directory, set the Python path: export PYTHONPATH=pwd``
  5. Attempt install of requirements: pip --user install -r requirements.txt
  6. Build spaCy: python setup.py build_ext --inplace
  7. Install spaCy: pip install --user .

However, I continue getting stuck at installing blis on step 5. Any thoughts?

Thanks for the assistance @JulianGerhard21!

TheCedarPrince commented 5 years ago

https://github.com/explosion/cython-blis/issues/9#issuecomment-534876436

JulianGerhard21 commented 5 years ago

Hi @TheCedarPrince ,

thanks for your feedback. Unfortunately I am a bit busy today. I have cloned my SDCard and tomorrow I am going to reinstall Rasa 1.3.9 alongside with tensorflow 1.14.0 and Spacy with its newest version and document the whole process.

A keypoint is the fact that the most compatible python version at the moment is 3.6.8 which is not installed per default on Debian 10. After reinstalling that, several other issues did not happen again.

I am going to give you the documentation and feedback, but please dont expect until tomorrow evening. I want to make sure that it's done properly.

Another thing: removing a dependency from a library is not a neat solution so I'd suggest to rename the issue again: "Issues while installing Rasa on a Raspberry Pi4". This would lead more users with partially different/same problems to the issue.

@wochinge I am getting back to you as soon as I have verified, that everything works properly. Maybe you can release the documentation on the docs then until the tutorial is finished. I think a tutorial is more or less a clean guide for different experienced users whereas the docs might address technical facts directly. What do you guys think?

Regards

wochinge commented 5 years ago

@JulianGerhard21

of course, I would be happy to do that. I am not quite sure if you are in the community@rasa.com dispatcher

Good question 😄I saw your video of the smart speaker project because it was shared internally and that was so cool!

but Emma already asked the same about my SmartSpeaker project on the Raspi. Is there any template for a tutorial / blog entry that I should respect?

I sent Emma a link to our conversation here, so she can get in touch with you. Not sure, how much sense it makes to combine both blog posts, I guess this would be up to you. Also, there is no template you have to respect.

I should add that I won't be able to start this week but next

Sorry, I didn't mean to rush you. Just the fact that you consider doing that at all is so great!

Maybe you can release the documentation on the docs then until the tutorial is finished.

Pulling in @akelad for that one. Do you want to put that on the docs?

TheCedarPrince commented 5 years ago

@JulianGerhard21 - oh! I'll test things out with Python3.6.

Oh no rush at all! I just appreciate the help so far.

I agree with you on renaming the issue but I feel the name may be somewhat vague and may attract off-base requests. Since we already solved the or-tools problem, perhaps we could close this issue and open a more general post on the forum? What do you think @JulianGerhard21 and @wochinge?

Also, if you are strapped for time @JulianGerhard21, I could potentially help with the tutorial/docs you want to put together - I have been cataloging my progress anyhow and I could port it over to a tutorial somewhat quickly.

JulianGerhard21 commented 4 years ago

Hi guys,

this is one adventure.... I am nearly finished with a stable Installation guide. Last step is to compile python 3.6.8 with bz2 Support. I will give you feedback / the guide as soon as I finished that step.

Some of the major problems are the dependencies of tensor2tensor which mostly are not arm7 compatible per default and the fact that tensorflow currently has problems with python 3.7.x.

The good news: I managed to install tensorflow, rasa and spacy All with their latest versions (tf 1.14. And 2) without errors by following my documentation.

However, rasa uses bzip2 to handle some file IO operations and I used a python version without it. I am going to confirm everything asap and provide the tutorial.

We can then discuss how to publish it @wochinge @thecedarprince

Regards Julian

JulianGerhard21 commented 4 years ago

Hi guys, (@wochinge @TheCedarPrince @akelad)

I was now able to set everything up properly. I had the following setup, but tried to make everything as flexible as possible so at least Raspian 9 should be supported aswell:

Device: Raspberry Pi 4 OS: Raspbian 10 Python: 3.6.8 Frameworks to install: Tensorflow, SpaCy, Rasa

The following steps are necessary or recommended:

  1. Update your OS with sudo apt-get update && apt-get upgrade
  2. At the time writing this, Raspbian 10 ships with python 3.7.3 as default - which currently is not officially supported by Tensorflow. To workaround this, we are going to install python 3.6.8 in several steps. Note that bz2 is needed by Rasa and not per default compiled with python.

2.1 Install the bzip2 development library with sudo apt-get install libbz2-dev libssl-dev 2.2 Download the python 3.6.8 tarfile 2.3 Enter the following commands in your terminal:

tar -xvf Python-3.6.8.tar.xz
cd Python-3.6.8.tar.xz
sudo ./configure
sudo make -j4
sudo make install

To confirm that your installation works, you can do:

python3.6 -V
python3.6 -m pip -V
  1. Update pip and setuptools with python3.6 -m pip install --upgrade pip --user and python3.6 -m pip install --upgrade setuptools --user (necessary to be able to install the latest wheels/libs)
  2. Install Tensorflow with python3.6 -m pip install tensorflow==1.14.0 --user

There should not occur problems until this point. Now things are getting a bit more complicated since we are entering areas where compatibility is not given and we need to build/modify things. First of all we need to clone certain repositories (clone in separate directories is recommended):

a. git clone https://github.com/explosion/spaCy b. git clone https://github.com/tensorflow/tensor2tensor c. wget https://github.com/RasaHQ/rasa/archive/1.4.0.zip

  1. Follow this OpenCV tutorial from step 2 on until OpenCV is successfully installed. You should be able to do "import cv2" in python3.6 then. If there is an error, don't worry - OpenCV is not mandatory.

  2. Execute the following command in terminal: export BLIS_ARCH=generic (blis is used for computational purposes and mandatory for spacy)

  3. Move into the spaCy folder and use python3.6 -m pip install -r requirements.txt --user

  4. Move into the spaCy folder and execute python setup.py build_ext --inplace

  5. Move into the spaCy folder and execute python3.6 -m pip install . --user

  6. Execute the following command in terminal: python3.6 -m pip install dopamine-rl --user --no-deps (there is a cyclic behaviour in the dependencies of dopamine which is a dependency of tensor2tensor so we are just simplifying things here).

  7. Move into the tensor2tensor folder and remove the following dependency from setup.py: opencv-python (we have installed this previously but there is no symlink created so python won't find it easily)

  8. Move into the tensor2tensor folder and use python3.6 -m pip install . --user --force-reinstall

At this time, we have successfully installed Tensorflow, SpaCy and several dependencies that are getting important now.

  1. Execute the following commands in terminal: sudo apt install python3-dev && apt install libpq-dev/buster (per default, there is to postgres system installed but rasa has the requirement psycopg2)
  2. Execute the following command: python3.6 -m pip install psycopg2 --user
  3. Move into the Rasa-1.4.0 folder and remove the following dependency from requirements.txt and setup.py: tensor2tensor (we have installed this previously)
  4. Move into the Rasa-1.4.0 folder and execute: python3.6 -m pip install -r requirements.txt --user --force-reinstall
  5. Move into the Rasa-1.4.0 folder and execute: python3.6 -m pip install . --user --force-reinstall

Now you should be able to run: python3.6 -m rasa init and e.g. use pretrained_embeddings_spacy as a pipeline (assuming you downloaded at least one model from spacy).

General notes:

The main problems are missing wheels for Raspbian/Debian 10, compatibility issues with various python versions and the fact, that arm7 is not the widest supported architecture. However, the performance is really good in terms of training/usage time.

Of course this guide may not work for your specific scenario. Mostly this is because there are so many situations/systems/environments that I can't cover them all. This guide worked for me even after the third complete reinstall of my Pi so I assume you can either use it directly or getting the idea behind my approach. I am deeply sorry in case I missed a step. I needed 3 full days to work things out and tons of coffee and love of my wife that calmed be down everytime I was on a rampage :D So please use my guide and don't be shy to mention things I might have forgotten.

If there are questions, feel free to ask.

TheCedarPrince commented 4 years ago

this is one adventure.... I am nearly finished with a stable Installation guide. Last step is to compile python 3.6.8 with bz2 Support.

I am deeply sorry in case I missed a step. I needed 3 full days to work things out and tons of coffee and love of my wife that calmed be down everytime I was on a rampage :D

This is a fantastic write-up @JulianGerhard21 ! I will work on this over the next couple days to see if I can replicate things (am a bit busy right now but should have time to get things all set by Wednesday). Much thanks to you (and your wife!) for all the hard work; in short:

When I get time, I will let you know how things go!

One small note that is fantastic - I originally used Python 3.5 to avoid the OpenCV issue. You were able to work around it in Python3.6 which is awesome. The more recent the version, the better in my opinion!

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince,

thanks for your kind words. With every iteration we can make the guide more stable! Most likely you will have questions! :)

Let me know whenever I can be of help!

Regards Julian

TheCedarPrince commented 4 years ago

Ok, here is where I got to for tonight @JulianGerhard21,

I am currently stuck on Step 3. In Step 2 I was not able to run the command tar xJF Python-3.6.8.tar.xz as it kept giving this error:

pi@raspberrypi:~ $ tar xJF Python-3.6.8.tar.xz
tar: Cannot use multi-volume compressed archives
Try 'tar --help' or 'tar --usage' for more information.

I proceeded on and instead used tar -xvf Python-3.6.8.tar.xz.

Once I got to Step 3, I ran the command python3.6 -m pip install --upgrade pip and got the following traceback:

pi@raspberrypi:~ $ sudo python3.6 -m pip install --upgrade pip 
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=3, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=2, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)': /simple/pip/
Could not fetch URL https://www.piwheels.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='www.piwheels.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
Requirement already up-to-date: pip in /usr/local/lib/python3.6/site-packages (18.1)
pip is configured with locations that require TLS/SSL, however the ssl module in Python is not available.
Could not fetch URL https://pypi.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping
Could not fetch URL https://www.piwheels.org/simple/pip/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='www.piwheels.org', port=443): Max retries exceeded with url: /simple/pip/ (Caused by SSLError("Can't connect to HTTPS URL because the SSL module is not available.",)) - skipping

Any thoughts about what you think is going on? This was after a clean install of Raspbian (Buster) Lite after having only ran the commands:

sudo apt-get update && 
sudo apt-get upgrade -y && 
sudo apt-get dist-upgrade -y
JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince ,

please do:

sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev

and then repeat step 2.3 and build/configure/make/install Python again. I think the libssl was missing. If you can confirm that it works, I will update the guide accordingly.

Regards Julian

TheCedarPrince commented 4 years ago

@JulianGerhard21 - that worked!

I would suggest editing the guide on Step 2 to this:

2. Install the system dependencies with sudo apt-get install libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev

Also, I like to keep my installs as light as possible and was curious, do we need all the other system dependencies aside from libbz2-dev and libssl-dev? Or can we get away with simply, sudo apt-get install libbz2-dev libssl-dev?

Finally, in step 3 I do get an error if I run the pip install without the --user flag. I would suggest editing the guide on Step 3 to this:

3. Update pip and setuptools with python3.6 -m pip install --user --upgrade pip and python3.6 -m pip install --user --upgrade setuptools (necessary to be able to install the latest wheels/libs)

Will test out the rest of the steps later and keep you up to date.

Yours, TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince ,

Or can we get away with simply, sudo apt-get install libbz2-dev libssl-dev?

this depends mainly on what you want to be able to do later - because of that it was not part of the guide. I myself need most of those packages, but you are right: the minimal requirement are those two. I changed the guide accordingly.

Finally, in step 3 I do get an error if I run the pip install without the --user flag. I would suggest editing the guide on Step 3 to this:

Jeah - this might be a bit annoying but I think to keep things clean, it makes sense. I changed the guide accordingly.

Regards Julian

TheCedarPrince commented 4 years ago

Hey @JulianGerhard21,

Doing a small update/check-in right now that I want your opinion on. I will provide more info later but am running into an error on Step 16. Once I follow your instructions up to that point, I am still getting errors about the program shouting about opencv as well as dopamine-rl.

I removed opencv from the setup.py file in tensor2tensor; when that didn't work, I also tried removing dopamine-rl from the file. No luck either way.

Here is the traceback I am getting with only opencv removed from the file.

pi@raspberrypi:~/tensor2tensor $ python3.6 -m pip install . --user --force-reinstall
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/tensor2tensor
Collecting bz2file
  Using cached https://www.piwheels.org/simple/bz2file/bz2file-0.98-py3-none-any.whl
Collecting dopamine-rl
  Using cached https://www.piwheels.org/simple/dopamine-rl/dopamine_rl-2.0.5-py3-none-any.whl
Collecting flask
  Using cached https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl
Collecting future
  Using cached https://www.piwheels.org/simple/future/future-0.18.1-py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/35/a6/a0/c8441dbd28276aaff148c417279dee46f84936ba108cf32208/gevent-1.4.0-cp36-cp36m-linux_armv7l.whl
Collecting gin-config
  Using cached https://www.piwheels.org/simple/gin-config/gin_config-0.2.1-py3-none-any.whl
Collecting google-api-python-client
  Using cached https://www.piwheels.org/simple/google-api-python-client/google_api_python_client-1.7.11-py3-none-any.whl
Collecting gunicorn
  Using cached https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl
Collecting gym
  Using cached https://www.piwheels.org/simple/gym/gym-0.15.3-py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/bd/4f/8e/a608fe814bb1b784ca1680e7fac3cc16b4e071a46275997e69/h5py-2.10.0-cp36-cp36m-linux_armv7l.whl
Collecting kfac
  Using cached https://files.pythonhosted.org/packages/01/f0/4a7758f854a15b37d322827123ce58619d0f4270dd94f2dd30328f397339/kfac-0.2.0-py2.py3-none-any.whl
Collecting mesh-tensorflow
  Using cached https://files.pythonhosted.org/packages/42/10/7bbf9f19bbd57693b7cd66358ad432ea358e2af24d0f444ec6abf6385f49/mesh_tensorflow-0.1.4-py2.py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/5e/e9/4b/dd5a8eb53e97dfcc1314eca9c6769edd3cad379d6644c1ad94/numpy-1.17.3-cp36-cp36m-linux_armv7l.whl
Collecting oauth2client
  Using cached https://files.pythonhosted.org/packages/95/a9/4f25a14d23f0786b64875b91784607c2277eff25d48f915e39ff0cff505a/oauth2client-4.1.3-py2.py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/ad/d5/3d/65149b7a670d61cfe9636b75c1dcc94eb49bb33412b260f328/Pillow-6.2.1-cp36-cp36m-linux_armv7l.whl
Collecting pypng
  Using cached https://www.piwheels.org/simple/pypng/pypng-0.0.20-py3-none-any.whl
Collecting requests
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/4f/02/c1/598fa696113cc7be343f4da8ff69ed02e86ebf13ce420f2f20/scipy-1.3.1-cp36-cp36m-linux_armv7l.whl
Collecting six
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting sympy
  Using cached https://files.pythonhosted.org/packages/21/21/f4105795ca7f35c541d82c5b06be684dd2f5cb4f508fb487cd7aea4de776/sympy-1.4-py2.py3-none-any.whl
Collecting tensorflow-datasets
  Using cached https://files.pythonhosted.org/packages/e2/14/900746f9b8e1ed8ea93661f1a62b3937a5a25858d550ece38de8ea579f2d/tensorflow_datasets-1.3.0-py3-none-any.whl
Collecting tensorflow-gan
  Using cached https://files.pythonhosted.org/packages/3e/94/8903150ffafdd538b18e6d24dbd4f5a07105d9d31a29db49e728ae2a8b13/tensorflow_gan-1.0.0.dev0-py2.py3-none-any.whl
Collecting tensorflow-probability==0.7.0
  Using cached https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl
Collecting tqdm
  Using cached https://files.pythonhosted.org/packages/e1/c1/bc1dba38b48f4ae3c4428aea669c5e27bd5a7642a74c8348451e0bd8ff86/tqdm-4.36.1-py2.py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1) (from versions: none)
ERROR: No matching distribution found for opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1)

Here is also the output of my pip list (I do have OpenCV installed and can successfully import it - just don't know why cv2 is not popping up in my pip list):

Package                  Version   
------------------------ ----------
absl-py                  0.8.1     
astor                    0.8.0     
atomicwrites             1.3.0     
attrs                    19.3.0    
blis                     0.4.1     
bz2file                  0.98      
cachetools               3.1.1     
certifi                  2019.9.11 
chardet                  3.0.4     
Click                    7.0       
cloudpickle              1.2.2     
cymem                    2.0.2     
Cython                   0.29.13   
decorator                4.4.0     
dill                     0.3.1.1   
dopamine-rl              2.0.5     
flake8                   3.5.0     
Flask                    1.1.1     
future                   0.18.1    
gast                     0.3.2     
gevent                   1.4.0     
gin-config               0.2.1     
google-api-python-client 1.7.11    
google-auth              1.6.3     
google-auth-httplib2     0.0.3     
google-pasta             0.1.7     
googleapis-common-protos 1.6.0     
greenlet                 0.4.15    
grpcio                   1.24.1    
gunicorn                 19.9.0    
gym                      0.15.3    
h5py                     2.10.0    
httplib2                 0.14.0    
idna                     2.8       
importlib-metadata       0.23      
itsdangerous             1.1.0     
Jinja2                   2.10.3    
jsonschema               3.0.2     
Keras-Applications       1.0.8     
Keras-Preprocessing      1.1.0     
kfac                     0.2.0     
Markdown                 3.1.1     
MarkupSafe               1.1.1     
mccabe                   0.6.1     
mesh-tensorflow          0.1.4     
mock                     2.0.0     
more-itertools           7.2.0     
mpmath                   1.1.0     
murmurhash               1.0.2     
numpy                    1.17.3    
oauth2client             4.1.3     
packaging                19.2      
pbr                      5.4.3     
Pillow                   6.2.1     
pip                      19.3.1    
plac                     1.1.0     
pluggy                   0.13.0    
preshed                  3.0.2     
promise                  2.2.1     
protobuf                 3.10.0    
psycopg2                 2.8.4     
py                       1.8.0     
pyasn1                   0.4.7     
pyasn1-modules           0.2.7     
pycodestyle              2.3.1     
pyflakes                 1.6.0     
pyglet                   1.3.2     
pyparsing                2.4.2     
pypng                    0.0.20    
pyrsistent               0.15.4    
pytest                   5.2.1     
pytest-timeout           1.3.3     
requests                 2.22.0    
rsa                      4.0       
scipy                    1.3.1     
setuptools               41.4.0    
six                      1.12.0    
spacy                    2.2.1     
srsly                    0.1.0     
sympy                    1.4       
tensorboard              1.14.0    
tensorflow               1.14.0    
tensorflow-datasets      1.3.0     
tensorflow-estimator     1.14.0    
tensorflow-gan           1.0.0.dev0
tensorflow-metadata      0.15.0    
tensorflow-probability   0.7.0     
termcolor                1.1.0     
thinc                    7.2.0     
tqdm                     4.36.1    
uritemplate              3.0.0     
urllib3                  1.25.6    
wasabi                   0.2.2     
wcwidth                  0.1.7     
Werkzeug                 0.16.0    
wheel                    0.33.6    
wrapt                    1.11.2    
zipp                     0.6.0   

Any thoughts on what is going on? I may try a fresh install to see if that works. If you need more information, let me know.

Yours, TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince,

are you sure that you haven't accidently skipped this part:

python3.6 -m pip install dopamine-rl --user --no-deps (steps 10 and 11) ?

I'd suggest to repeat those steps first: 10 - 12 and 15, after that, you should be able to do 16.

Keep the following in mind: If you install something via pip, pip checks the dependencies of the package and those dependencies normally have their own dependencies. If you have modified a third-party library to be able to install something, this modification is only used if installed manually with the . - path. If it gets fetched from a remote pip index, your modifications won't work - this is actually what happens in step 16.

Regarding cv2: As I mentioned, there is no information-deployment in the site-packages (where pip show reads from) for cv2 because this step is missing in the OpenCV guide. It is something on my agenda that I have to experiment on - should not be that hard.

I hope it works now!

Regards Julian

TheCedarPrince commented 4 years ago

Hey @JulianGerhard21,

I am so so close!

image

I can boot up RASA, it pops me into the prompt, and when it starts to do its magic, it fails with the following traceback:

Welcome to Rasa! 🤖

To get started quickly, an initial project will be created.
If you need some help, check out the documentation at https://rasa.com/docs/rasa.
Now let's start! 👇🏽

? Please enter a path where the project will be created [default: current directory] .                                                                                                        
Created project directory at '/home/src'.
Finished creating project structure.
Training an initial model...
Training Core model...
Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/registry.py", line 21, in policy_from_module_path
    module_path, lookup_path="rasa.core.policies.registry"
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/utils/common.py", line 184, in class_from_module_path
    m = importlib.import_module(lookup_path)
  File "/usr/local/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/policies/registry.py", line 5, in <module>
    from rasa.core.policies.embedding_policy import EmbeddingPolicy
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/policies/embedding_policy.py", line 24, in <module>
    from rasa.utils import train_utils
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/utils/train_utils.py", line 9, in <module>
    from tensor2tensor.models.transformer import (
ModuleNotFoundError: No module named 'tensor2tensor'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/policies/ensemble.py", line 320, in from_dict
    constr_func = registry.policy_from_module_path(policy_name)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/registry.py", line 24, in policy_from_module_path
    raise ImportError("Cannot retrieve policy from path '{}'".format(module_path))
ImportError: Cannot retrieve policy from path 'MemoizationPolicy'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/__main__.py", line 87, in <module>
    main()
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 195, in run
    init_project(args, path)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 113, in init_project
    print_train_or_instructions(args, path)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/cli/scaffold.py", line 50, in print_train_or_instructions
    args.model = rasa.train(domain, config, training_files, output)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/train.py", line 45, in train
    kwargs=kwargs,
  File "uvloop/loop.pyx", line 1417, in uvloop.loop.Loop.run_until_complete
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/train.py", line 96, in train_async
    kwargs,
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/train.py", line 182, in _train_async_internal
    kwargs=kwargs,
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/train.py", line 217, in _do_training
    kwargs=kwargs,
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/train.py", line 346, in _train_core_with_validated_data
    kwargs=kwargs,
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/train.py", line 43, in train
    policies = config.load(policy_config)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/config.py", line 28, in load
    return PolicyEnsemble.from_dict(config_data)
  File "/home/pi/.local/lib/python3.6/site-packages/rasa/core/policies/ensemble.py", line 328, in from_dict
    "".format(policy_name)
rasa.core.policies.ensemble.InvalidPolicyConfig: Module for policy 'MemoizationPolicy' could not be loaded. Please make sure the name is a valid policy.

I am definitely executing through Steps 10 - 12 to install tensor2tensor but for some reason it keeps failing. Here is exactly what I am doing:

cd tensor2tensor
python3.6 -m pip install dopamine-rl --user --no-deps
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user --force-reinstall

And here is the following traceback I am getting from that:

pi@raspberrypi:~/tensor2tensor $ python3.6 -m pip install . --user --force-reinstall
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/tensor2tensor
Collecting bz2file
  Using cached https://www.piwheels.org/simple/bz2file/bz2file-0.98-py3-none-any.whl
Collecting dopamine-rl
  Using cached https://www.piwheels.org/simple/dopamine-rl/dopamine_rl-2.0.5-py3-none-any.whl
Collecting flask
  Using cached https://files.pythonhosted.org/packages/9b/93/628509b8d5dc749656a9641f4caf13540e2cdec85276964ff8f43bbb1d3b/Flask-1.1.1-py2.py3-none-any.whl
Collecting future
  Using cached https://www.piwheels.org/simple/future/future-0.18.1-py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/35/a6/a0/c8441dbd28276aaff148c417279dee46f84936ba108cf32208/gevent-1.4.0-cp36-cp36m-linux_armv7l.whl
Collecting gin-config
  Using cached https://www.piwheels.org/simple/gin-config/gin_config-0.2.1-py3-none-any.whl
Collecting google-api-python-client
  Using cached https://www.piwheels.org/simple/google-api-python-client/google_api_python_client-1.7.11-py3-none-any.whl
Collecting gunicorn
  Using cached https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl
Collecting gym
  Using cached https://www.piwheels.org/simple/gym/gym-0.15.3-py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/bd/4f/8e/a608fe814bb1b784ca1680e7fac3cc16b4e071a46275997e69/h5py-2.10.0-cp36-cp36m-linux_armv7l.whl
Collecting kfac
  Using cached https://files.pythonhosted.org/packages/01/f0/4a7758f854a15b37d322827123ce58619d0f4270dd94f2dd30328f397339/kfac-0.2.0-py2.py3-none-any.whl
Collecting mesh-tensorflow
  Using cached https://files.pythonhosted.org/packages/42/10/7bbf9f19bbd57693b7cd66358ad432ea358e2af24d0f444ec6abf6385f49/mesh_tensorflow-0.1.4-py2.py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/5e/e9/4b/dd5a8eb53e97dfcc1314eca9c6769edd3cad379d6644c1ad94/numpy-1.17.3-cp36-cp36m-linux_armv7l.whl
Collecting oauth2client
  Using cached https://files.pythonhosted.org/packages/95/a9/4f25a14d23f0786b64875b91784607c2277eff25d48f915e39ff0cff505a/oauth2client-4.1.3-py2.py3-none-any.whl
Collecting Pillow
  Using cached https://files.pythonhosted.org/packages/5b/bb/cdc8086db1f15d0664dd22a62c69613cdc00f1dd430b5b19df1bea83f2a3/Pillow-6.2.1.tar.gz
Collecting pypng
  Using cached https://www.piwheels.org/simple/pypng/pypng-0.0.20-py3-none-any.whl
Collecting requests
  Using cached https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl
Processing /home/pi/.cache/pip/wheels/4f/02/c1/598fa696113cc7be343f4da8ff69ed02e86ebf13ce420f2f20/scipy-1.3.1-cp36-cp36m-linux_armv7l.whl
Collecting six
  Using cached https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Collecting sympy
  Using cached https://files.pythonhosted.org/packages/21/21/f4105795ca7f35c541d82c5b06be684dd2f5cb4f508fb487cd7aea4de776/sympy-1.4-py2.py3-none-any.whl
Collecting tensorflow-datasets
  Using cached https://files.pythonhosted.org/packages/e2/14/900746f9b8e1ed8ea93661f1a62b3937a5a25858d550ece38de8ea579f2d/tensorflow_datasets-1.3.0-py3-none-any.whl
Collecting tensorflow-gan
  Using cached https://files.pythonhosted.org/packages/3e/94/8903150ffafdd538b18e6d24dbd4f5a07105d9d31a29db49e728ae2a8b13/tensorflow_gan-1.0.0.dev0-py2.py3-none-any.whl
Collecting tensorflow-probability==0.7.0
  Using cached https://files.pythonhosted.org/packages/3e/3a/c10b6c22320531c774402ac7186d1b673374e2a9d12502cbc8d811e4601c/tensorflow_probability-0.7.0-py2.py3-none-any.whl
Collecting tqdm
  Using cached https://files.pythonhosted.org/packages/e1/c1/bc1dba38b48f4ae3c4428aea669c5e27bd5a7642a74c8348451e0bd8ff86/tqdm-4.36.1-py2.py3-none-any.whl
Collecting absl-py>=0.2.2
  Using cached https://www.piwheels.org/simple/absl-py/absl_py-0.8.1-py3-none-any.whl
ERROR: Could not find a version that satisfies the requirement opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1) (from versions: none)
ERROR: No matching distribution found for opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1)

Also, here is the traceback for when I run the command for dopamine-rl after I had initially removed it - am I installing the correct version?:

pi@raspberrypi:~/tensor2tensor $ python3.6 -m pip install dopamine-rl --user --no-deps
Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Collecting dopamine-rl
  Using cached https://www.piwheels.org/simple/dopamine-rl/dopamine_rl-2.0.5-py3-none-any.whl
Installing collected packages: dopamine-rl
Successfully installed dopamine-rl-2.0.5

We are almost there I think! Any thoughts as to what you think is happening?

EDIT: Wait, I might have found the mistake. Hang on. EDIT 2: Nope. False alarm. Still Stuck.

Yours, TheCedarPrince

TheCedarPrince commented 4 years ago

@JulianGerhard21 ,

Ok, I tried tracking down my bug but still could not find it. Any thoughts?

Yours, TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince,

ok - let's try something:

Do a pip show dopamine-rl and if version 2.0.5 is shown (which is the correct one), move into tensor2tensor directory and use python3.6 -m pip install . --user without the --force-reinstall. This should prevent pip from digging too deep into the dependencies tree.

The error shown by rasa is "only" a consecutive error produced by the broken tensor2tensor libraby.

Did that work?

Regards Julian

TheCedarPrince commented 4 years ago

@JulianGerhard21 ,

I tried it again and still no luck; here is the return on python3.6 -m pip show dopamine-rl:

Name: dopamine-rl
Version: 2.0.5
Summary: Dopamine: A framework for flexible Reinforcement Learning research
Home-page: https://github.com/google/dopamine
Author: The Dopamine Team
Author-email: UNKNOWN
License: Apache 2.0
Location: /home/pi/.local/lib/python3.6/site-packages
Requires: opencv-python, gym, absl-py, gin-config
Required-by: 

And here is the output from when I execute python3.6 -m pip install . --user:

Looking in indexes: https://pypi.org/simple, https://www.piwheels.org/simple
Processing /home/pi/tensor2tensor
Collecting bz2file
  Using cached https://www.piwheels.org/simple/bz2file/bz2file-0.98-py3-none-any.whl
Requirement already satisfied: dopamine-rl in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (2.0.5)
Requirement already satisfied: flask in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (1.1.1)
Requirement already satisfied: future in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (0.18.1)
Requirement already satisfied: gevent in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (1.4.0)
Collecting gin-config
  Using cached https://www.piwheels.org/simple/gin-config/gin_config-0.2.1-py3-none-any.whl
Collecting google-api-python-client
  Using cached https://www.piwheels.org/simple/google-api-python-client/google_api_python_client-1.7.11-py3-none-any.whl
Collecting gunicorn
  Using cached https://files.pythonhosted.org/packages/8c/da/b8dd8deb741bff556db53902d4706774c8e1e67265f69528c14c003644e6/gunicorn-19.9.0-py2.py3-none-any.whl
Collecting gym
  Using cached https://www.piwheels.org/simple/gym/gym-0.15.3-py3-none-any.whl
Requirement already satisfied: h5py in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (2.10.0)
Collecting kfac
  Using cached https://files.pythonhosted.org/packages/01/f0/4a7758f854a15b37d322827123ce58619d0f4270dd94f2dd30328f397339/kfac-0.2.0-py2.py3-none-any.whl
Collecting mesh-tensorflow
  Using cached https://files.pythonhosted.org/packages/42/10/7bbf9f19bbd57693b7cd66358ad432ea358e2af24d0f444ec6abf6385f49/mesh_tensorflow-0.1.4-py2.py3-none-any.whl
Requirement already satisfied: numpy in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (1.17.3)
Collecting oauth2client
  Using cached https://files.pythonhosted.org/packages/95/a9/4f25a14d23f0786b64875b91784607c2277eff25d48f915e39ff0cff505a/oauth2client-4.1.3-py2.py3-none-any.whl
Collecting Pillow
  Using cached https://files.pythonhosted.org/packages/5b/bb/cdc8086db1f15d0664dd22a62c69613cdc00f1dd430b5b19df1bea83f2a3/Pillow-6.2.1.tar.gz
Collecting pypng
  Using cached https://www.piwheels.org/simple/pypng/pypng-0.0.20-py3-none-any.whl
Requirement already satisfied: requests in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (2.22.0)
Requirement already satisfied: scipy in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (1.3.1)
Requirement already satisfied: six in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (1.12.0)
Collecting sympy
  Using cached https://files.pythonhosted.org/packages/21/21/f4105795ca7f35c541d82c5b06be684dd2f5cb4f508fb487cd7aea4de776/sympy-1.4-py2.py3-none-any.whl
WARNING: Retrying (Retry(total=4, connect=None, read=None, redirect=None, status=None)) after connection broken by 'ProtocolError('Connection aborted.', RemoteDisconnected('Remote end closed connection without response',))': /simple/tensorflow-datasets/
Collecting tensorflow-datasets
  Using cached https://files.pythonhosted.org/packages/e2/14/900746f9b8e1ed8ea93661f1a62b3937a5a25858d550ece38de8ea579f2d/tensorflow_datasets-1.3.0-py3-none-any.whl
Collecting tensorflow-gan
  Using cached https://files.pythonhosted.org/packages/3e/94/8903150ffafdd538b18e6d24dbd4f5a07105d9d31a29db49e728ae2a8b13/tensorflow_gan-1.0.0.dev0-py2.py3-none-any.whl
Requirement already satisfied: tensorflow-probability==0.7.0 in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (0.7.0)
Requirement already satisfied: tqdm in /home/pi/.local/lib/python3.6/site-packages (from tensor2tensor==1.14.1) (4.36.1)
ERROR: Could not find a version that satisfies the requirement opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1) (from versions: none)
ERROR: No matching distribution found for opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1)

I have a theory that I am going to try based on the output from the pip show of dopamine-rl. The source code of dopamine in their setup.py file has open-cv as a requirement; I am going to remove that since we already have open-cv installed and attempt to install from source.

Will post results back later. Let me know if you have any other theories. But you never had a problem with dopamine-rl before when you were installing things on your RPi @JulianGerhard21 ?

Yours,

TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince,

hm - strange. I never had this problem. However we are going to solve it. Try your idea and give feedback - thats good. Another idea would be:

Remove dopamine from tensor2tensor's dependencies and install it again. Confirm with pip show tensor2tensor that it is installed and try Rasa.

Regards Julian

TheCedarPrince commented 4 years ago

VICTORY!!! :tada: :tada: :tada:

Julian. Julian. JULIAN!!!!

image

WE DID IT!!! @JulianGerhard21 !!! THIS IS HUGE!!!

All I did was the following:

git clone https://github.com/google/dopamine.git
cd dopamine/
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user

And now the bot WORKS!!! I haven't tested out all the functionality to see if everything is working completely, but it finished training on the RPi completely using spaCy and all the other packages.


Creating Your Own RasaPi - Instructions

@JulianGerhard21 and all others interested in getting RASA on an RPi4, I meticulously created a script that will automatically install everything needed onto your Pi based on the aforementioned exchange:

  1. First, prepare a fresh installation of Raspbian Buster (Lite) on your Pi.
  2. Copy the following script into a file called install.sh
# Update the Raspberry Pi
sudo apt-get update
sudo apt-get upgrade -y 
sudo apt-get dist-upgrade -y

# Install initial build dependencies
# Provides 
# Enables pip3.6 to access pypi
sudo apt-get install libbz2-dev libssl-dev -y 

# Get and install Python3.6
wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
tar -xvf Python-3.6.8.tar.xz
cd Python-3.6.8
sudo ./configure
sudo make -j4
sudo make install

# Update Python3.6 packages
python3.6 -m pip install --upgrade pip setuptools --user

# Install additional dependencies
# Enables access to Tensorflow whl
# Dependency for the h5py python package
sudo apt-get install python3-pip libhdf5-dev -y

# Install Tensorflow
python3.6 -m pip install tensorflow==1.14.0 --user

# Install OpenCV Dependencies
sudo apt-get install build-essential cmake unzip pkg-config -y
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev -y
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev -y
sudo apt-get install libxvidcore-dev libx264-dev -y
sudo apt-get install libgtk-3-dev -y
sudo apt-get install libcanberra-gtk* -y
sudo apt-get install libatlas-base-dev gfortran -y
sudo apt-get install python3-dev -y

# Download OpenCV and clarify naming scheme
cd ~
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.0.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.0.0.zip
unzip opencv.zip
unzip opencv_contrib.zip
mv opencv-4.0.0 opencv
mv opencv_contrib-4.0.0 opencv_contrib

cd ~/opencv
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
    -D ENABLE_NEON=ON \
    -D ENABLE_VFPV3=ON \
    -D BUILD_TESTS=OFF \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D INSTALL_PYTHON_EXAMPLES=OFF \
    -D BUILD_EXAMPLES=OFF ..

# Increasing swap size to make OpenCV
SWAPSIZE=2048
sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Make OpenCV
make -j4

# Install OpenCV
sudo make install
sudo ldconfig

# Reseting swap size
SWAPSIZE=100
sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Link cv2 to Python3.6
cd /usr/local/lib/python3.6/site-packages/
sudo ln -s /usr/local/python/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so cv2.so

# In case git is not installed
sudo apt-get install git -y

# Getting codebases for spaCy, tensor2tensor, and RASA
# NOTE: This is hard-coded for rasa-1.4.0 right now - let's make it more elegant soon
cd ~
git clone https://github.com/explosion/spaCy
git clone https://github.com/tensorflow/tensor2tensor
git clone https://github.com/google/dopamine.git
wget https://github.com/RasaHQ/rasa/archive/1.4.0.zip && unzip 1.4.0.zip 

# Installing spaCy
export BLIS_ARCH=generic
cd ~/spaCy
python3.6 -m pip install -r requirements.txt --user
python3.6 setup.py build_ext --inplace
python3.6 -m pip install . --user

# Installing dopamine-rl
cd ~/dopamine
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user

# Installing tensor2tensor
cd ~/tensor2tensor
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user --force-reinstall

# Installing other RASA dependencies
sudo apt install libpq-dev/buster -y 
python3.6 -m pip install psycopg2 --user

# Installing RASA
cd ~/rasa-1.4.0
sed -i '/tensor2tensor/d' setup.py
sed -i '/tensor2tensor/d' requirements.txt
python3.6 -m pip install -r requirements.txt --user --force-reinstall
python3.6 -m pip install . --user --force-reinstall

# Script exit
echo ""
echo ""
echo "------------------------------------------------------------"
echo "Congratulations! Rasa is now installed on your Raspberry Pi."
echo "To test rasa out, run python3.6 -m rasa init and start "
echo "creating your bot!"
  1. Execute the following: sudo chmod +x install.sh
  2. Execute the following: ./install.sh Please note that installation will take a very long time (~3 hours or so). Grab a tea, go for a run, read a book and come back later!
  3. You will have some folders and files leftover in your home directory so feel free to delete those now that everything is installed!

THANK YOU!

I would certainly like to thank @JulianGerhard21 for all the fantastic assistance, input, and massively heavy lifting. Without Julian's input, none of this would be possible. I have had a wonderful time working together here and learned a lot from you and this process - a "thank you" cannot do you enough justice but... THANK YOU

I would also like to thank @jrosebr for his awesome instructions that I adapted here for installing OpenCV.

Finally, a huge thank you to the RASA team! Thank you for making this wonderful product and I am excited to see how we can all use it on resource-constrained devices!


And with that, I shall now, close the issue. Issue: RESOLVED!

EDIT 10/24/19: Fixed a path error.

TheCedarPrince commented 4 years ago

So, what now @JulianGerhard21 , @wochinge , @akelad ?

I was thinking @wochinge and @akelad , perhaps the RASA team could make a separate, community-supported repo for the Raspberry Pi now that things are working? I could imagine things will become incompatible in the future and it would be a shame to see this support fall away as I think this opens RASA to a whole new community of makers, researchers, etc.

Perhaps we could start the Repo with the aforementioned script that was adapted from @JulianGerhard21 's guide and tweaked by me? Maybe you guys could also host a Pi image with RASA set up on it that we could download from the RASA site?

What do you think @JulianGerhard21 ?

Yours,

TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @TheCedarPrince ,

glad it worked - thanks for the kind words and thanks for the script - good idea - that makes life much easier.

I am up to everything that makes sense. I confirm that there definitely will be incompatibilites in the future and we need to keep track of them. A feature-branch or own repo needs to be updated/maintained and I am not quite sure if the Rasa team wants to provide that - but of course I would be in.

I think @wochinge and @akelad need to decide how to proceed. I think a tutorial on their blog would be a good start. In addition we could determine if your other suggestions make sense!

However, I am in - of course! :-)

Regards Julian

akelad commented 4 years ago

@JulianGerhard21 @TheCedarPrince awesome that you got it working! it's really great to see the community working together like this :) We will certainly discuss internally how we can best support you with the Raspberry Pi setup and will get back to you

TheCedarPrince commented 4 years ago

Hey @wochinge , @akelad , @JulianGerhard21 ,

I am reopening this issue as I want to see if the script Julian and I wrote can be independently verified. To test it, here is what you do:

  1. Download the latest version of Raspbian Lite onto a Raspberry Pi 4
  2. Copy this script onto the Raspberry Pi into install.sh:
# Update the Raspberry Pi
sudo apt-get update
sudo apt-get upgrade -y 
sudo apt-get dist-upgrade -y

# Install initial build dependencies
# Provides 
# Enables pip3.6 to access pypi
sudo apt-get install libbz2-dev libssl-dev -y 

# Get and install Python3.6
wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
tar -xvf Python-3.6.8.tar.xz
cd Python-3.6.8
sudo ./configure
sudo make -j4
sudo make install

# Update Python3.6 packages
python3.6 -m pip install --upgrade pip setuptools --user

# Install additional dependencies
# Enables access to Tensorflow whl
# Dependency for the h5py python package
sudo apt-get install python3-pip libhdf5-dev -y

# Install Tensorflow
python3.6 -m pip install tensorflow==1.14.0 --user

# Install OpenCV Dependencies
sudo apt-get install build-essential cmake unzip pkg-config -y
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev -y
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev -y
sudo apt-get install libxvidcore-dev libx264-dev -y
sudo apt-get install libgtk-3-dev -y
sudo apt-get install libcanberra-gtk* -y
sudo apt-get install libatlas-base-dev gfortran -y

# Download OpenCV and clarify naming scheme
cd ~
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.0.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.0.0.zip
unzip opencv.zip
unzip opencv_contrib.zip
mv opencv-4.0.0 opencv
mv opencv_contrib-4.0.0 opencv_contrib

cd ~/opencv
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
    -D ENABLE_NEON=ON \
    -D ENABLE_VFPV3=ON \
    -D BUILD_TESTS=OFF \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D INSTALL_PYTHON_EXAMPLES=OFF \
    -D BUILD_EXAMPLES=OFF ..

# Increasing swap size to make OpenCV
SWAPSIZE=2048
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Make OpenCV
make -j4

# Install OpenCV
sudo make install
sudo ldconfig

# Reseting swap size
SWAPSIZE=100
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Link cv2 to Python3.6
cd /usr/local/lib/python3.6/site-packages/
sudo ln -s /usr/local/python/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so cv2.so

# In case git is not installed
sudo apt-get install git -y

# Getting codebases for spaCy, tensor2tensor, and RASA
# NOTE: This is hard-coded for rasa-1.4.0 right now - let's make it more elegant soon
cd ~
git clone https://github.com/explosion/spaCy
git clone https://github.com/tensorflow/tensor2tensor
git clone https://github.com/google/dopamine.git
wget https://github.com/RasaHQ/rasa/archive/1.4.0.zip && unzip 1.4.0.zip 

# Installing spaCy
export BLIS_ARCH=generic
cd ~/spaCy
python3.6 -m pip install -r requirements.txt --user
python3.6 setup.py build_ext --inplace
python3.6 -m pip install . --user

# Installing dopamine-rl
cd ~/dopamine
sed -i '/opencv-python/d' setup.py
sed -i '/dopamine-rl/d' setup.py
python3.6 -m pip install . --user

# Installing tensor2tensor
cd ~/tensor2tensor
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user --force-reinstall

# Installing other RASA dependencies
sudo apt install libpq-dev/buster -y 
python3.6 -m pip install psycopg2 --user

# Installing RASA
cd ~/rasa-1.4.0
sed -i '/tensor2tensor/d' setup.py
sed -i '/tensor2tensor/d' requirements.txt
python3.6 -m pip install -r requirements.txt --user --force-reinstall
python3.6 -m pip install . --user --force-reinstall

# Script exit
echo ""
echo ""
echo "------------------------------------------------------------"
echo "Congratulations! Rasa is now installed on your Raspberry Pi."
echo "To test rasa out, run python3.6 -m rasa init and start "
echo "creating your bot!"
  1. Make the script executable by running sudo chmod +x install.sh
  2. Execute the script by running ./install.sh | tee log.txt
  3. Place share log.txt here and let me know if you ran into any issues while installing

This process will take upwards of 3 hours so don't worry if it is taking longer than expected. Thanks for the help and looking forward to making sure this script is good to go!

TheCedarPrince

wochinge commented 4 years ago

Hi @TheCedarPrince , @JulianGerhard21 ,

our team discussed how we can best add this to Rasa, and we think it would be best if you would set up a separate repository which includes the script and maybe some tests which verify that new releases didn't break things. If you need any help in setting this up, we are happy to support you there. We can then link to it from the docs so that it's visible to the community. How does that sound?

I am reopening this issue as I want to see if the script Julian and I wrote can be independently verified. To test it, here is what you do:

I'm currently trying to find out whether anyone in our engineering part owns a raspberry PI. However, you could also share it in our forum and get some feedback from the community? https://forum.rasa.com/c/rasa/tutorials would be a good place for that 🤔

TheCedarPrince commented 4 years ago

Hey @wochinge - that sounds good.

Unfortunately, I am going to have to say no about creating the separate repository right now as I do not have the bandwidth to take it on. @JulianGerhard21 - do you?

The forum is a good idea! I will pop over soon and make a post. :smile:

Yours, TheCedarPrince

JulianGerhard21 commented 4 years ago

Hi @wochinge, @TheCedarPrince

Sorry for the late reply. I had some private incidents pro manage. I think this is a good idea/approach and I would be willing to work on this. However this cant be done in a few days IMHO so I need some time to think about the things to prepare.

I would come back to you with feedback asap!

Thanks for the support!

Regards Julian

TheCedarPrince commented 4 years ago

Hey @wochinge and @JulianGerhard21,

I added the news to the Rasa Forum and am now waiting to hear from the community about the script.

Sounds good @JulianGerhard21 - hope all is well! Chat later.

Yours, TheCedarPrince

wochinge commented 4 years ago

@JulianGerhard21 Sure, take all the time you need!

@TheCedarPrince Thanks for creating the post!

rgstephens commented 4 years ago

@TheCedarPrince thanks for this script. I'd really like to get Rasa working on the Pi. The tensor2tensor install is failing and I confirmed that opencv-python has been removed from the setup.py.

ERROR: Could not find a version that satisfies the requirement opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1) (from versions: none)
ERROR: No matching distribution found for opencv-python>=3.4.1.15 (from dopamine-rl->tensor2tensor==1.14.1)
TheCedarPrince commented 4 years ago

Hey @rgstephens

If it is there, could you try removing dopamine-rl from the setup.py for tensor2tensor as well?

Let me know if that works.

Also, I am closing this issue and encourage everyone who has a question to see this forum and post your questions here.

Thank you! TheCedarPrince

Suniron commented 4 years ago

Hi,

I got this problem with the install.sh:


ERROR: Could not find a version that satisfies the requirement tensorflow~
=1.14.0 (from rasa==1.4.0) (from versions: none)                          
ERROR: No matching distribution found for tensorflow~=1.14.0 (from rasa==1
.4.0)```
Suniron commented 4 years ago

Hey @wochinge , @akelad , @JulianGerhard21 ,

I am reopening this issue as I want to see if the script Julian and I wrote can be independently verified. To test it, here is what you do:

  1. Download the latest version of Raspbian Lite onto a Raspberry Pi 4
  2. Copy this script onto the Raspberry Pi into install.sh:
# Update the Raspberry Pi
sudo apt-get update
sudo apt-get upgrade -y 
sudo apt-get dist-upgrade -y

# Install initial build dependencies
# Provides 
# Enables pip3.6 to access pypi
sudo apt-get install libbz2-dev libssl-dev -y 

# Get and install Python3.6
wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
tar -xvf Python-3.6.8.tar.xz
cd Python-3.6.8
sudo ./configure
sudo make -j4
sudo make install

# Update Python3.6 packages
python3.6 -m pip install --upgrade pip setuptools --user

# Install additional dependencies
# Enables access to Tensorflow whl
# Dependency for the h5py python package
sudo apt-get install python3-pip libhdf5-dev -y

# Install Tensorflow
python3.6 -m pip install tensorflow==1.14.0 --user

# Install OpenCV Dependencies
sudo apt-get install build-essential cmake unzip pkg-config -y
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev -y
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev -y
sudo apt-get install libxvidcore-dev libx264-dev -y
sudo apt-get install libgtk-3-dev -y
sudo apt-get install libcanberra-gtk* -y
sudo apt-get install libatlas-base-dev gfortran -y

# Download OpenCV and clarify naming scheme
cd ~
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.0.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.0.0.zip
unzip opencv.zip
unzip opencv_contrib.zip
mv opencv-4.0.0 opencv
mv opencv_contrib-4.0.0 opencv_contrib

cd ~/opencv
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
    -D ENABLE_NEON=ON \
    -D ENABLE_VFPV3=ON \
    -D BUILD_TESTS=OFF \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D INSTALL_PYTHON_EXAMPLES=OFF \
    -D BUILD_EXAMPLES=OFF ..

# Increasing swap size to make OpenCV
SWAPSIZE=2048
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Make OpenCV
make -j4

# Install OpenCV
sudo make install
sudo ldconfig

# Reseting swap size
SWAPSIZE=100
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Link cv2 to Python3.6
cd /usr/local/lib/python3.6/site-packages/
sudo ln -s /usr/local/python/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so cv2.so

# In case git is not installed
sudo apt-get install git -y

# Getting codebases for spaCy, tensor2tensor, and RASA
# NOTE: This is hard-coded for rasa-1.4.0 right now - let's make it more elegant soon
cd ~
git clone https://github.com/explosion/spaCy
git clone https://github.com/tensorflow/tensor2tensor
git clone https://github.com/google/dopamine.git
wget https://github.com/RasaHQ/rasa/archive/1.4.0.zip && unzip 1.4.0.zip 

# Installing spaCy
export BLIS_ARCH=generic
cd ~/spaCy
python3.6 -m pip install -r requirements.txt --user
python3.6 setup.py build_ext --inplace
python3.6 -m pip install . --user

# Installing dopamine-rl
cd ~/dopamine
sed -i '/opencv-python/d' setup.py
sed -i '/dopamine-rl/d' setup.py
python3.6 -m pip install . --user

# Installing tensor2tensor
cd ~/tensor2tensor
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user --force-reinstall

# Installing other RASA dependencies
sudo apt install libpq-dev/buster -y 
python3.6 -m pip install psycopg2 --user

# Installing RASA
cd ~/rasa-1.4.0
sed -i '/tensor2tensor/d' setup.py
sed -i '/tensor2tensor/d' requirements.txt
python3.6 -m pip install -r requirements.txt --user --force-reinstall
python3.6 -m pip install . --user --force-reinstall

# Script exit
echo ""
echo ""
echo "------------------------------------------------------------"
echo "Congratulations! Rasa is now installed on your Raspberry Pi."
echo "To test rasa out, run python3.6 -m rasa init and start "
echo "creating your bot!"
  1. Make the script executable by running sudo chmod +x install.sh
  2. Execute the script by running ./install.sh | tee log.txt
  3. Place share log.txt here and let me know if you ran into any issues while installing

This process will take upwards of 3 hours so don't worry if it is taking longer than expected. Thanks for the help and looking forward to making sure this script is good to go!

TheCedarPrince

How can i share you my log.txt file ?

phandisubburaman commented 4 years ago

Hey @wochinge , @akelad , @JulianGerhard21 , I am reopening this issue as I want to see if the script Julian and I wrote can be independently verified. To test it, here is what you do:

  1. Download the latest version of Raspbian Lite onto a Raspberry Pi 4
  2. Copy this script onto the Raspberry Pi into install.sh:
# Update the Raspberry Pi
sudo apt-get update
sudo apt-get upgrade -y 
sudo apt-get dist-upgrade -y

# Install initial build dependencies
# Provides 
# Enables pip3.6 to access pypi
sudo apt-get install libbz2-dev libssl-dev -y 

# Get and install Python3.6
wget https://www.python.org/ftp/python/3.6.8/Python-3.6.8.tar.xz
tar -xvf Python-3.6.8.tar.xz
cd Python-3.6.8
sudo ./configure
sudo make -j4
sudo make install

# Update Python3.6 packages
python3.6 -m pip install --upgrade pip setuptools --user

# Install additional dependencies
# Enables access to Tensorflow whl
# Dependency for the h5py python package
sudo apt-get install python3-pip libhdf5-dev -y

# Install Tensorflow
python3.6 -m pip install tensorflow==1.14.0 --user

# Install OpenCV Dependencies
sudo apt-get install build-essential cmake unzip pkg-config -y
sudo apt-get install libjpeg-dev libpng-dev libtiff-dev -y
sudo apt-get install libavcodec-dev libavformat-dev libswscale-dev libv4l-dev -y
sudo apt-get install libxvidcore-dev libx264-dev -y
sudo apt-get install libgtk-3-dev -y
sudo apt-get install libcanberra-gtk* -y
sudo apt-get install libatlas-base-dev gfortran -y

# Download OpenCV and clarify naming scheme
cd ~
wget -O opencv.zip https://github.com/opencv/opencv/archive/4.0.0.zip
wget -O opencv_contrib.zip https://github.com/opencv/opencv_contrib/archive/4.0.0.zip
unzip opencv.zip
unzip opencv_contrib.zip
mv opencv-4.0.0 opencv
mv opencv_contrib-4.0.0 opencv_contrib

cd ~/opencv
mkdir build
cd build

cmake -D CMAKE_BUILD_TYPE=RELEASE \
    -D CMAKE_INSTALL_PREFIX=/usr/local \
    -D OPENCV_EXTRA_MODULES_PATH=~/opencv_contrib/modules \
    -D ENABLE_NEON=ON \
    -D ENABLE_VFPV3=ON \
    -D BUILD_TESTS=OFF \
    -D OPENCV_ENABLE_NONFREE=ON \
    -D INSTALL_PYTHON_EXAMPLES=OFF \
    -D BUILD_EXAMPLES=OFF ..

# Increasing swap size to make OpenCV
SWAPSIZE=2048
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Make OpenCV
make -j4

# Install OpenCV
sudo make install
sudo ldconfig

# Reseting swap size
SWAPSIZE=100
sudo sed -i "s/^CONF_SWAPSIZE.*/CONF_SWAPSIZE=${SWAPSIZE}/" /etc/dphys-swapfile

# Restarting swap service
sudo /etc/init.d/dphys-swapfile stop
sudo /etc/init.d/dphys-swapfile start

# Link cv2 to Python3.6
cd /usr/local/lib/python3.6/site-packages/
sudo ln -s /usr/local/python/cv2/python-3.6/cv2.cpython-36m-arm-linux-gnueabihf.so cv2.so

# In case git is not installed
sudo apt-get install git -y

# Getting codebases for spaCy, tensor2tensor, and RASA
# NOTE: This is hard-coded for rasa-1.4.0 right now - let's make it more elegant soon
cd ~
git clone https://github.com/explosion/spaCy
git clone https://github.com/tensorflow/tensor2tensor
git clone https://github.com/google/dopamine.git
wget https://github.com/RasaHQ/rasa/archive/1.4.0.zip && unzip 1.4.0.zip 

# Installing spaCy
export BLIS_ARCH=generic
cd ~/spaCy
python3.6 -m pip install -r requirements.txt --user
python3.6 setup.py build_ext --inplace
python3.6 -m pip install . --user

# Installing dopamine-rl
cd ~/dopamine
sed -i '/opencv-python/d' setup.py
sed -i '/dopamine-rl/d' setup.py
python3.6 -m pip install . --user

# Installing tensor2tensor
cd ~/tensor2tensor
sed -i '/opencv-python/d' setup.py
python3.6 -m pip install . --user --force-reinstall

# Installing other RASA dependencies
sudo apt install libpq-dev/buster -y 
python3.6 -m pip install psycopg2 --user

# Installing RASA
cd ~/rasa-1.4.0
sed -i '/tensor2tensor/d' setup.py
sed -i '/tensor2tensor/d' requirements.txt
python3.6 -m pip install -r requirements.txt --user --force-reinstall
python3.6 -m pip install . --user --force-reinstall

# Script exit
echo ""
echo ""
echo "------------------------------------------------------------"
echo "Congratulations! Rasa is now installed on your Raspberry Pi."
echo "To test rasa out, run python3.6 -m rasa init and start "
echo "creating your bot!"
  1. Make the script executable by running sudo chmod +x install.sh
  2. Execute the script by running ./install.sh | tee log.txt
  3. Place share log.txt here and let me know if you ran into any issues while installing

This process will take upwards of 3 hours so don't worry if it is taking longer than expected. Thanks for the help and looking forward to making sure this script is good to go! TheCedarPrince

How can i share you my log.txt file ?

I am also facing same issue Any help?

JulianGerhard21 commented 4 years ago

Hi all,

I will be able to provide feedback this thursday afternoon with Rasa 1.8.

Ill post the details here asap.

Regards Julian

Suniron commented 4 years ago

Hi,

With the last rasa version (1.8.0), i got this on my Raspberry 4 4GB with Ubuntu server 64 bits 18.04:


ERROR: Could not find a version that satisfies the requirement tensorflow<2.2,>=2.1 (from rasa) (from versions: none)
ERROR: No matching distribution found for tensorflow<2.2,>=2.1 (from rasa)```

I think that it's really not possible :'(
sd-z commented 4 years ago

In case anybody is following the guide and running into the error: ERROR: Could not find a version that satisfies the requirement tensorflow-addons (from tensor2tensor==) (from versions: none) ERROR: No matching distribution found for tensorflow-addons (from tensor2tensor==) It can be solved by doing: git checkout v1.14.1 after entering the tensor2tensor directory.

This is due to tensorflow-addons being added in later versions of tensorflow after the release of the guide. However Python3.6.8 pip on Jessie only installs tensorflow 1.14.1. Thanks for the guide it helped a lot!

rgstephens commented 4 years ago

There's also a recent forum post with a Docker image for Raspberry Pi here