produvia / kryptos

Kryptos AI is a virtual investment assistant that manages your cryptocurrency portfolio
http://twitter.com/kryptos_ai
MIT License
48 stars 8 forks source link

ImportError: cannot import name 'choices' #136

Closed ghost closed 5 years ago

ghost commented 5 years ago

@slavakurilyak thank you for your work.
When I try to run the app by docker-compose up I get ImportError: cannot import name 'choices' error. Output is below. Do you have an idea why this is happening?

Terminal Output

Starting redis ... done
Starting db       ... done
Recreating ml      ... done
Recreating worker  ... done
Recreating web     ... done
Recreating updater ... done
Attaching to redis, db, worker, ml, updater, web
db         | 2019-03-15 18:34:48.991 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
db         | 2019-03-15 18:34:48.991 UTC [1] LOG:  listening on IPv6 address "::", port 5432
db         | 2019-03-15 18:34:49.000 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
redis      | 1:C 15 Mar 18:34:48.101 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
redis      | 1:C 15 Mar 18:34:48.101 # Redis version=4.0.5, bits=64, commit=00000000, modified=0, pid=1, just started
redis      | 1:C 15 Mar 18:34:48.101 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
redis      | 1:M 15 Mar 18:34:48.103 * Running mode=standalone, port=6379.
redis      | 1:M 15 Mar 18:34:48.103 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
redis      | 1:M 15 Mar 18:34:48.103 # Server initialized
redis      | 1:M 15 Mar 18:34:48.104 # WARNING you have Transparent Huge Pages (THP) support enabled in your kernel. This will create latency and memory usage issues with Redis. To fix this issue run the command 'echo never > /sys/kernel/mm/transparent_hugepage/enabled' as root, and add it to your /etc/rc.local in order to retain the setting after a reboot. Redis must be restarted after THP is disabled.
redis      | 1:M 15 Mar 18:34:48.104 * DB loaded from disk: 0.000 seconds
redis      | 1:M 15 Mar 18:34:48.104 * Ready to accept connections
worker     | monitor.1 started (pid=8)
db         | 2019-03-15 18:34:49.032 UTC [18] LOG:  database system was shut down at 2019-03-15 18:29:43 UTC
db         | 2019-03-15 18:34:49.043 UTC [1] LOG:  database system is ready to accept connections
worker     | worker.1 started (pid=9)
web        | dev.1 started (pid=7)
worker     | [2019-03-15 18:34:53 +0000] [10] [INFO] Starting gunicorn 19.9.0
worker     | [2019-03-15 18:34:53 +0000] [10] [INFO] Listening at: http://0.0.0.0:8080 (10)
worker     | [2019-03-15 18:34:53 +0000] [10] [INFO] Using worker: sync
worker     | [2019-03-15 18:34:53 +0000] [19] [INFO] Booting worker with pid: 19
web        |  * Serving Flask app "autoapp.py" (lazy loading)
web        |  * Environment: docker-dev
web        |  * Debug mode: on
web        |  * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)
updater    | updater.1 started (pid=8)
updater    | Traceback (most recent call last):
updater    |   File "updater.py", line 10, in <module>
updater    |     from app.settings import get_from_datastore
updater    |   File "/app/app/settings.py", line 20, in <module>
updater    |     class Config(object):
updater    |   File "/app/app/settings.py", line 41, in Config
updater    |     CLIENT_ACCESS_TOKEN = get_from_datastore("CLIENT_ACCESS_TOKEN", "production")
updater    |   File "/app/app/settings.py", line 11, in get_from_datastore
updater    |     ds = datastore.Client()
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 205, in __init__
updater    |     project=project, credentials=credentials, _http=_http)
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 215, in __init__
updater    |     _ClientProjectMixin.__init__(self, project=project)
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 169, in __init__
updater    |     project = self._determine_default(project)
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 222, in _determine_default
updater    |     return _determine_default_project(project)
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 75, in _determine_default_project
updater    |     project = _base_default_project(project=project)
updater    |   File "/usr/local/lib/python3.6/site-packages/google/cloud/_helpers.py", line 179, in _determine_default_project
updater    |     _, project = google.auth.default()
updater    |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 294, in default
updater    |     credentials, project_id = checker()
updater    |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 165, in _get_explicit_environ_credentials
updater    |     os.environ[environment_vars.CREDENTIALS])
updater    |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 89, in _load_credentials_from_file
updater    |     'File {} was not found.'.format(filename))
updater    | google.auth.exceptions.DefaultCredentialsError: File /app/Kryptos-Staging-3deb8860d570.json was not found.
updater    | updater.1 stopped (rc=1)
web        |  * Restarting with stat
updater exited with code 1
worker     | [2019-03-15 18:34:56 +0000] [19] [ERROR] Exception in worker process
worker     | Traceback (most recent call last):
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/arbiter.py", line 583, in spawn_worker
worker     |     worker.init_process()
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 129, in init_process
worker     |     self.load_wsgi()
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/workers/base.py", line 138, in load_wsgi
worker     |     self.wsgi = self.app.wsgi()
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/base.py", line 67, in wsgi
worker     |     self.callable = self.load()
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 52, in load
worker     |     return self.load_wsgiapp()
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/app/wsgiapp.py", line 41, in load_wsgiapp
worker     |     return util.import_app(self.app_uri)
worker     |   File "/usr/local/lib/python3.6/site-packages/gunicorn/util.py", line 350, in import_app
worker     |     __import__(module)
worker     |   File "/core/monitor.py", line 5, in <module>
worker     |     from kryptos.settings import REDIS_HOST, REDIS_PORT
worker     |   File "/core/kryptos/__init__.py", line 2, in <module>
worker     |     from .logger import logger_group, setup_logging, add_logger
worker     |   File "/core/kryptos/logger.py", line 16, in <module>
worker     |     cloud_client = google.cloud.logging.Client()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/logging/client.py", line 104, in __init__
worker     |     project=project, credentials=credentials, _http=_http)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 215, in __init__
worker     |     _ClientProjectMixin.__init__(self, project=project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 169, in __init__
worker     |     project = self._determine_default(project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 182, in _determine_default
worker     |     return _determine_default_project(project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/_helpers.py", line 179, in _determine_default_project
worker     |     _, project = google.auth.default()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 294, in default
worker     |     credentials, project_id = checker()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 165, in _get_explicit_environ_credentials
worker     |     os.environ[environment_vars.CREDENTIALS])
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 89, in _load_credentials_from_file
worker     |     'File {} was not found.'.format(filename))
worker     | google.auth.exceptions.DefaultCredentialsError: File Kryptos-Staging-3deb8860d570.json was not found.
worker     | [2019-03-15 18:34:56 +0000] [19] [INFO] Worker exiting (pid: 19)
worker     | Traceback (most recent call last):
worker     |   File "kryptos/worker/manager.py", line 18, in <module>
worker     |     from kryptos import logger_group, setup_logging
worker     |   File "/core/kryptos/__init__.py", line 2, in <module>
worker     |     from .logger import logger_group, setup_logging, add_logger
worker     |   File "/core/kryptos/logger.py", line 16, in <module>
worker     |     cloud_client = google.cloud.logging.Client()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/logging/client.py", line 104, in __init__
worker     |     project=project, credentials=credentials, _http=_http)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 215, in __init__
worker     |     _ClientProjectMixin.__init__(self, project=project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 169, in __init__
worker     |     project = self._determine_default(project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 182, in _determine_default
worker     |     return _determine_default_project(project)
worker     |   File "/usr/local/lib/python3.6/site-packages/google/cloud/_helpers.py", line 179, in _determine_default_project
worker     |     _, project = google.auth.default()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 294, in default
worker     |     credentials, project_id = checker()
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 165, in _get_explicit_environ_credentials
worker     |     os.environ[environment_vars.CREDENTIALS])
worker     |   File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 89, in _load_credentials_from_file
worker     |     'File {} was not found.'.format(filename))
worker     | google.auth.exceptions.DefaultCredentialsError: File Kryptos-Staging-3deb8860d570.json was not found.
worker     | [2019-03-15 18:34:57 +0000] [10] [INFO] Shutting down: Master
worker     | [2019-03-15 18:34:57 +0000] [10] [INFO] Reason: Worker failed to boot.
worker     | monitor.1 stopped (rc=3)
worker     | sending SIGTERM to worker.1 (pid 9)
worker     | worker.1 stopped (rc=-15)
worker exited with code 3
web        |  * Debugger is active!
web        |  * Debugger PIN: 302-198-789
ml         | [2019-03-15 18:35:00.129441] INFO: ML_INDICATOR: Starting initial ML worker
ml         | /usr/local/lib/python3.6/site-packages/sklearn/ensemble/weight_boosting.py:29: DeprecationWarning: numpy.core.umath_tests is an internal NumPy module and should not be imported. It will be removed in a future NumPy release.
ml         |   from numpy.core.umath_tests import inner1d
web        | 172.18.0.1 - - [15/Mar/2019 18:35:39] "GET / HTTP/1.1" 500 -
web        | Traceback (most recent call last):
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 325, in __call__
web        |     self._flush_bg_loading_exception()
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 313, in _flush_bg_loading_exception
web        |     reraise(*exc_info)
web        |   File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 35, in reraise
web        |     raise value
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 302, in _load_app
web        |     self._load_unlocked()
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 317, in _load_unlocked
web        |     self._app = rv = self.loader()
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 372, in load_app
web        |     app = locate_app(self, import_name, name)
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 242, in locate_app
web        |     '\n\n{tb}'.format(name=module_name, tb=traceback.format_exc())
web        | flask.cli.NoAppException: While importing "autoapp", an ImportError was raised:
web        | 
web        | Traceback (most recent call last):
web        |   File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 235, in locate_app
web        |     __import__(module_name)
web        |   File "/app/autoapp.py", line 4, in <module>
web        |     from app.app import create_app
web        |   File "/app/app/app.py", line 11, in <module>
web        |     from app import api, bot, models, task
web        |   File "/app/app/bot/__init__.py", line 1, in <module>
web        |     from . import assistant
web        |   File "/app/app/bot/assistant.py", line 9, in <module>
web        |     from app.utils import build
web        |   File "/app/app/utils/build.py", line 9, in <module>
web        |     from app.utils import choices
web        | ImportError: cannot import name 'choices'
web        | 172.18.0.1 - - [15/Mar/2019 18:35:39] "GET /?__debugger__=yes&cmd=resource&f=style.css HTTP/1.1" 200 -
web        | 172.18.0.1 - - [15/Mar/2019 18:35:39] "GET /?__debugger__=yes&cmd=resource&f=debugger.js HTTP/1.1" 200 -
web        | 172.18.0.1 - - [15/Mar/2019 18:35:39] "GET /?__debugger__=yes&cmd=resource&f=jquery.js HTTP/1.1" 200 -
web        | 172.18.0.1 - - [15/Mar/2019 18:35:39] "GET /?__debugger__=yes&cmd=resource&f=ubuntu.ttf HTTP/1.1" 200 -
web        | 172.18.0.1 - - [15/Mar/2019 18:35:40] "GET /?__debugger__=yes&cmd=resource&f=console.png HTTP/1.1" 200 -

Flask output


flask.cli.NoAppException
flask.cli.NoAppException: While importing "autoapp", an ImportError was raised:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 235, in locate_app
    __import__(module_name)
  File "/app/autoapp.py", line 4, in <module>
    from app.app import create_app
  File "/app/app/app.py", line 11, in <module>
    from app import api, bot, models, task
  File "/app/app/bot/__init__.py", line 1, in <module>
    from . import assistant
  File "/app/app/bot/assistant.py", line 9, in <module>
    from app.utils import build
  File "/app/app/utils/build.py", line 9, in <module>
    from app.utils import choices
ImportError: cannot import name 'choices'

Traceback (most recent call last)
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 330, in __call__
rv = self._load_unlocked()
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 317, in _load_unlocked
self._app = rv = self.loader()
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 372, in load_app
app = locate_app(self, import_name, name)
File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 242, in locate_app
'\n\n{tb}'.format(name=module_name, tb=traceback.format_exc())
flask.cli.NoAppException: While importing "autoapp", an ImportError was raised: Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/flask/cli.py", line 235, in locate_app __import__(module_name) File "/app/autoapp.py", line 4, in <module> from app.app import create_app File "/app/app/app.py", line 11, in <module> from app import api, bot, models, task File "/app/app/bot/__init__.py", line 1, in <module> from . import assistant File "/app/app/bot/assistant.py", line 9, in <module> from app.utils import build File "/app/app/utils/build.py", line 9, in <module> from app.utils import choices ImportError: cannot import name 'choices'
slavakurilyak commented 5 years ago

@canercak Thanks for bringing this issue to my attention! Here is a commit file to resolve this issue. You can find the latest code release on the develop branch.

ghost commented 5 years ago

thank you, I no longer get that error. However was getting this since I don't have the google cloud json.

google.auth.exceptions.DefaultCredentialsError
google.auth.exceptions.DefaultCredentialsError: File /app/Kryptos-Staging-3deb8860d570.json was not found.

Traceback (most recent call last)
File "/app/autoapp.py", line 4, in <module>
from app.app import create_app
File "/app/app/app.py", line 14, in <module>
from app.settings import DockerDevConfig, ProdConfig
File "/app/app/settings.py", line 20, in <module>
class Config(object):
File "/app/app/settings.py", line 41, in Config
CLIENT_ACCESS_TOKEN = get_from_datastore("CLIENT_ACCESS_TOKEN", "production")
File "/app/app/settings.py", line 11, in get_from_datastore
ds = datastore.Client()
File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 205, in __init__
project=project, credentials=credentials, _http=_http)
File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 215, in __init__
_ClientProjectMixin.__init__(self, project=project)
File "/usr/local/lib/python3.6/site-packages/google/cloud/client.py", line 169, in __init__
project = self._determine_default(project)
File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 222, in _determine_default
return _determine_default_project(project)
File "/usr/local/lib/python3.6/site-packages/google/cloud/datastore/client.py", line 75, in _determine_default_project
project = _base_default_project(project=project)
File "/usr/local/lib/python3.6/site-packages/google/cloud/_helpers.py", line 179, in _determine_default_project
_, project = google.auth.default()
File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 294, in default
credentials, project_id = checker()
File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 165, in _get_explicit_environ_credentials
Open an interactive python shell in this frameos.environ[environment_vars.CREDENTIALS])
File "/usr/local/lib/python3.6/site-packages/google/auth/_default.py", line 89, in _load_credentials_from_file
'File {} was not found.'.format(filename))
google.auth.exceptions.DefaultCredentialsError: File /app/Kryptos-Staging-3deb8860d570.json was not found.

I commented out the get_from_datastore parts on settings.py, could see the main page but then when I was registering I was getting this

sqlalchemy.exc.ProgrammingError
sqlalchemy.exc.ProgrammingError: (psycopg2.ProgrammingError) relation "users" does not exist
LINE 2: FROM users 
             ^
 [SQL: 'SELECT users.is_active AS users_is_active, users.id AS users_id, users.email AS users_email, users.email_confirmed_at AS users_email_confirmed_at, users.password AS users_password, users.telegram_id AS users_telegram_id, users.telegram_username AS users_telegram_username, users.telegram_photo AS users_telegram_photo, users.telegram_auth_date AS users_telegram_auth_date \nFROM users \nWHERE users.email ILIKE %(email_1)s \n LIMIT %(param_1)s'] [parameters: {'email_1': 'canercak@gmail.com', 'param_1': 1}] (Background on this error at: http://sqlalche.me/e/f405) 

then I created postgresql://localhost/kryptos manually to get this error.

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
    Is the server running on host "localhost" (127.0.0.1) and accepting
    TCP/IP connections on port 5432?

then I changed postgres config from localhost to accept all connections, tried some more, etc but could not make it work.

slavakurilyak commented 5 years ago

@canercak did you try the command $ docker-compose up -d after running $ docker-compose build?

We are currently relying on Google Cloud Platform (GCP) for the deployment of this AI agent. We need to move away from GCP and implement a one-click deploy to Heroku. If anyone is up for the challenge, take a look at #39.

ghost commented 5 years ago

I tried. Getting the same result. I think not relying on deployment credentials just to see the system should be a bigger priority. And just want to add my idea because what you've done looks like nice and runs on proper foundations such as zipline/catalyst and I also want to apply alphazero or gym as you do. However I've run my own bot based on technical indicators and used almost all systems such as haas, gunbot, 3commas, cryptohopper, gekko, freqtrade just to lose money as all others do. Anyone who will get into this will understand that ml is the way to go but it's complicated so people need a simple bot who runs certain ml models. For example ml strategies in gekko work well but the bot is complicated, amateur and not-trustable (crashes, cannot buy bid/ask, cannot run multiple pairs with partial amounts, etc). People need something simple that they can control that doesn't rely on 16th century indicators. That's all. Otherwise no one will ever hand his cryptocurrency portfolio to a virtual assistant. I've handed several times and have seen what happens. Going to write my own bot anyway. Wish you good luck.

slavakurilyak commented 5 years ago

I think not relying on deployment credentials just to see the system should be a bigger priority

I agree. We need to move towards Heroku to simplify Kryptos. See: #39

And just want to add my idea because what you've done looks like nice and runs on proper foundations such as zipline/catalyst and I also want to apply alphazero or gym as you do.

Reinforcement learning has a lot of promise within Kryptos. See: #104

However I've run my own bot based on technical indicators and used almost all systems such as haas, gunbot, 3commas, cryptohopper, gekko, freqtrade just to lose money as all others do. Anyone who will get into this will understand that ml is the way to go but it's complicated so people need a simple bot who runs certain ml models. For example ml strategies in gekko work well but the bot is complicated, amateur and not-trustable (crashes, cannot buy bid/ask, cannot run multiple pairs with partial amounts, etc). People need something simple that they can control that doesn't rely on 16th century indicators. That's all. Otherwise no one will ever hand his cryptocurrency portfolio to a virtual assistant. I've handed several times and have seen what happens.

We believe that the only way to survive in the cryptocurrency-investment space is to use AI and machine learning. We began describing our visions for the application of these technologies here: machine learning, deep learning, data science, and reinforcement learning.

Thanks to the work completed by @bukosabino, we already use gradient tree boosting algorithms (XGBoost and LiteGBM) to perform machine learning strategies at Kryptos.