Localhost login error after running Pytest

valentierra commented 5 years ago

Once the command:

DOMAIN=backend sh ./script-test.sh

is executed, the containers are stopped and removed. To start the containers again I have used the command:

docker-compose up -d

Through localhost, Traefik, Celery, pgAdmin and Swagger services seem to work without problem, but Vue doesn't allow the login, reporting the error: "Incorrect email or password". I have followed this procedure a couple of times with identical result.

Is it because of a superuser restriction, like having to create another user to keep using the service? How could I recover the access to Vue service without reinstalling the full-stack?

headshothottie commented 5 years ago

Did you try: docker-compose logs ?

That helped me debug when I had a somewhat similar prob with Flower.

valentierra commented 5 years ago

Yes, it shows the service is up and working, but the user-password combination seems to be corrupted, as I have tried to log in using other browsers with the same error.

(...) frontend_1 | 172.22.0.4 - - [28/Oct/2018:08:40:32 +0000] "GET /js/app.af067547.js HTTP/1.1" 304 0 "http://localhost/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36" "172.22.0.1" frontend_1 | 172.22.0.4 - - [28/Oct/2018:08:40:34 +0000] "GET /img/icons/favicon-32x32.png HTTP/1.1" 200 1271 "http://localhost/login" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36" "172.22.0.1" frontend_1 | 172.22.0.4 - - [28/Oct/2018:08:40:35 +0000] "GET /service-worker.js HTTP/1.1" 200 965 "http://localhost/service-worker.js" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36" "172.22.0.1" frontend_1 | 172.22.0.4 - - [28/Oct/2018:08:41:27 +0000] "GET /favicon.ico HTTP/1.1" 200 1150 "http://localhost/api/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36" "172.22.0.1" frontend_1 | 172.22.0.4 - - [28/Oct/2018:08:41:29 +0000] "GET /login HTTP/1.1" 304 0 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) AppleWebKit/537.36 (KHTML,like Gecko) Chrome/70.0.3538.77 Safari/537.36 Vivaldi/2.1.1337.36" "172.22.0.1" (...)

Before running the Pytest, access the Vue service worked like a charm. Have you tried running the Pytest routine without any problem?

valentierra commented 5 years ago

Well, I found an easy way to have the Vue service back.

After running DOMAIN=backend sh ./script-test.sh to execute the Pytest on the backend, run:

docker-compose build

And then,

docker-compose up -d

The authorization token is restored from the image and Vue login is back to work as expected

tiangolo commented 5 years ago

Thanks @headshothottie for your tip! :+1: :tada:

@valentierra the script ./script-test.sh is mainly to be used by CI (continuous integration) systems. That's why it removes all the containers, builds them, runs the tests and then kills them. So that if you run it in your CI system it won't leave dangling containers.

Now, for local development, do this:

Start everything and develop locally with:

docker-compose up -d

Execute a bash session in the backend-tests container with:

docker-compose exec backend-tests bash

Inside that container's bash session, run the tests with:

pytest /app/app/tests/

If you update your backend code and reload/restart your backend container, you will be able to re-run the tests right away, without having to re-build the containers. Because, during development, the app code is mounted as a host volume inside the containers. That way you can iterate quickly without having to rebuild the containers.

Also, if you want to test some specific tests inside a specific directory or a specific file, you can do that with:

pytest /app/app/tests/my_specific_tests/

As an additional note, I recently updated the hosts handling to be able to run everything as localhost. But doing that, I introduced a little bug, that would make the tests not run properly during development, because they would be trying to communicate with the API at localhost instead of doing it at the sibling container with a host of backend. Because of that, if you ran the tests as I described above using a project generated between a couple weeks ago and 30 min ago, it wouldn't work.

I just updated the Docker Compose files to solve that. But you probably have the previous version. To update it, replicate these changes locally: https://github.com/tiangolo/full-stack/commit/d993cc623d4b58373fe28e821acaff44aa9a9f97

I.e.:

remove the - SERVER_NAME=${DOMAIN} from docker-compose.dev.env.yml
change - SERVER_NAME=${DOMAIN} to - SERVER_NAME=backend in docker-compose.test.yml

valentierra commented 5 years ago

Thank you Tiangolo!

That was a pretty fast and impeccable answer!

I really appreciate all the hard work you've done with the full-stack project. I have noticed you have another full-stack for Couchbase DB. What would be in your appreciation, the fair scale of data suitable to prefer Couchbase over Postgres?

tiangolo commented 5 years ago

I'm glad it helped you!

Short answer, by default, I would use a NoSQL system. As of now, I would go with Couchbase.

I have already worked on other projects, or experiments, with some relational DBs as MySQL, Postgres, and even some SQLite, and with several NoSQL databases as MongoDB, CouchDB, ElasticSearch, Solr, Cassandra. And my favorite as of now is Couchbase when the choice is available.

In my opinion, NoSQL systems have several advantages over RDBMSs for most modern applications. So, by default, I would try to go with one of those.

If I was you, I would use Postgres if:

You already have Postgres with previous data that you want to keep and you are building a new backend for it.
You are already very, very familiar with Postgres, its command line, internals, etc.
You depend on plugins or integrations specifically made for Postgres.
You need to use SQLAlchemy, or you are already very familiar with SQLAlchemy, or similar ORM tools, like Django's ORM or Peewee.

I wouldn't use it, even if the previous is true, if:

The data doesn't fit in one server.
The speed of the data is very fast (as in IoT, sensors, etc)
The schema (the tables and columns) might change in the future. This is almost always true.
You want to be able to grow easily adding more machines if necessary and have the possibility of having failover systems set up easily.
You need some special features as full text search or data synchronization with a web frontend or mobile applications, to enable offline first apps.

And again, being all the same, I would still go with a NoSQL system.

valentierra commented 5 years ago

The last third argument was definitive to make the decision of restarting my project with the Couchbase full-stack version. I see one of the Couchbase projects contains a sync gateway manager. Is there an appreciable advantage in opting for "Full-stack-flask-Couchbase" instead of "Full-stack-flask-CouchDB"?

tiangolo commented 5 years ago

First, if you already have some code based on this project, keep it around in a backup directory. All these project generators I have share the same structure and a lot of the code. So, if you have some code made already, a lot of the migration to the new project will be just copy-paste.

Couchbase was originally made out of CouchDB. But with Memcached on top. So Couchbase would be like having CouchDB (or MongoDB, for this comparison) with a caching Redis layer on top. But all integrated and handled by the same DB.

CouchDB created the synchronization protocol, in Couchbase it is implemented with the Sync Gateway. It allows mobile apps and frontends to synchronize data with the backend. So they can work offline with all the database functionality. The synchronization could also be used for things like messaging / chat systems, to get the new messages in near-realtime. For the front end you could use PouchDB. For mobile apps Couchbase Lite.

Couchbase Sync Gateway has an advantage over plain CouchDB, with it you can filter which documents in a database a user can see and synchronize. With CouchDB you can validate and filter writes, but reads are unlimited for the users that have access.

Another difference, Couchbase uses a query language syntax very similar or equivalent to SQL. MongoDB and CouchDB use a different query language, based on JSON.

valentierra commented 5 years ago

Thank you Tiangolo! Pretty fair explanation of each project version. Couchbase features look quite powerful and promising and so, I'll give it a try.

I have some machine learning code developed in python waiting to see the light. It is properly backed-up so I appreciate your concerns. My project is still at an early stage, so choosing the right deployment platform is crucial to what comes next. You certainly have saved me tons of time and work, and I hope to contribute to your project as things are getting the shape.

To keep things organized, I will move my comments and issues to the proper project thread. Thanks again!

tiangolo commented 5 years ago

Cool!

acnebs commented 5 years ago

I thought I'd piggyback this issue since I'm having a very similar problem and this issue is still open.

I am unable to login to the Vue dashboard once I get everything setup with cookiecutter + built with docker-compose, but everything else is working fine: Traefik, Flower, pgAdmin, etc. Although, with pgAdmin, I'm not actually seeing the database (as far as I can tell, but I also haven't used pgAdmin before so maybe I'm just missing something).

Anyway, back to the Vue Dashboard issue. In the browser, I'm getting a 502 Bad Gateway error in my console when I try to login. The request looks fine, and I've made sure that the payload being sent (user/pass) is the same as the values of FIRST_SUPERUSER and FIRST_SUPERUSER_PASSWORD in the backend env file.

This is also being picked up by Sentry, which is kicking off an error of

BadRequest: 400 Bad Request: Incorrect email or password
  File "flask/app.py", line 1813, in full_dispatch_request
    rv = self.dispatch_request()
  File "flask/app.py", line 1799, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "flask_apispec/annotations.py", line 118, in wrapped
    return wrapper(*args, **kwargs)
  File "flask_apispec/wrapper.py", line 29, in __call__
    response = self.call_view(*args, **kwargs)
  File "flask_apispec/wrapper.py", line 48, in call_view
    return self.func(*args, **kwargs)
  File "app/api/api_v1/endpoints/token.py", line 40, in route_login_access_token
    abort(400, "Incorrect email or password")
  File "werkzeug/exceptions.py", line 707, in abort
    return _aborter(status, *args, **kwargs)
  File "werkzeug/exceptions.py", line 687, in __call__
    raise self.mapping[code](*args, **kwargs)
error: unterminated character set at position 0
  File "flask/app.py", line 2292, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1815, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "flask_cors/extension.py", line 161, in wrapped_function
    return cors_after_request(app.make_response(f(*args, **kwargs)))
  File "flask_cors/extension.py", line 181, in cors_after_request
    set_cors_headers(resp, res_options)
  File "flask_cors/core.py", line 240, in set_cors_headers
    headers_to_set = get_cors_headers(options, request.headers, request.method)
  File "flask_cors/core.py", line 172, in get_cors_headers
    origins_to_set = get_cors_origins(options, request_headers.get('Origin'))
  File "flask_cors/core.py", line 123, in get_cors_origins
    elif try_match_any(request_origin, origins):
  File "flask_cors/core.py", line 267, in try_match_any
    return any(try_match(inst, pattern) for pattern in patterns)
  File "flask_cors/core.py", line 267, in <genexpr>
    return any(try_match(inst, pattern) for pattern in patterns)
  File "flask_cors/core.py", line 275, in try_match
    return re.match(maybe_regex, request_origin, flags=re.IGNORECASE)
  File "python3.6/re.py", line 172, in match
    return _compile(pattern, flags).match(string)
  File "python3.6/re.py", line 301, in _compile
    p = sre_compile.compile(pattern, flags)
  File "python3.6/sre_compile.py", line 562, in compile
    p = sre_parse.parse(p, flags)
  File "python3.6/sre_parse.py", line 855, in parse
    p = _parse_sub(source, pattern, flags & SRE_FLAG_VERBOSE, 0)
  File "python3.6/sre_parse.py", line 416, in _parse_sub
    not nested and not items))
  File "python3.6/sre_parse.py", line 523, in _parse
    source.tell() - here)

Any thoughts?

tiangolo commented 5 years ago

I can spot this section in your logs:

File "app/api/api_v1/endpoints/token.py", line 40, in route_login_access_token
    abort(400, "Incorrect email or password")

But this also calls my attention:

error: unterminated character set at position 0

What might that refer to?

Also, if you are using this project generator, with Flask and all these plug-ins, you might enjoy FastAPI: https://github.com/tiangolo/fastapi

It's made from the learnings of using all these plug-ins and other tools. To improve the development of APIs.

There's also an equivalent project generator: https://github.com/tiangolo/full-stack-fastapi-postgresql

tiangolo commented 4 years ago

Hi @valentierra , I'll assume you were able to solve your problem and I'll close this issue now.

If you are still using this project, I suggest you check the equivalent project generator for FastAPI that solves the same use cases in a much better way.

Because of that, this Flask-based project generator is now going to be deprecated. You are still free to use it, but it won't receive any new features, changes, or bug fixes.

tiangolo / full-stack

Localhost login error after running Pytest #11