Unable to login - Githubissues

adamsmesher commented 5 years ago

[emulation/Frameworks/reternal-quickstart]$ docker-compose up -d --build                                                          *[development]
Starting reternal-quickstart_redis-service_1 ... done
Starting reternal-quickstart_mongodb_1       ... done
Starting reternal-quickstart_compiler_1      ... done
Starting reternal-quickstart_api-socket_1    ... done
Starting reternal-quickstart_c2_1            ... done
Starting reternal-quickstart_api_1           ... done
Starting reternal-quickstart_ui_1            ... done

[emulation/Frameworks/reternal-quickstart]$ python3 manage.py -a install -t all                                             *[development]
Username: admin
Password:
Role (User/Admin): Admin
{'result': 'created', 'message': 'Succesfully created user'}
{'result': 'success', 'message': 'Finished importing base commands'}
{'result': 'success', 'message': 'Finished loading Mitre techniques'}
emulation/Frameworks/reternal-quickstart/cli/imports/import_mapping.py:14: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_object = yaml.load(yamlfile)
{'result': 'success', 'message': 'Finished loading mapped techniques'}
{'result': 'success', 'message': 'Finished importing all objects'}

[emulation/Frameworks/reternal-quickstart]$

"Succesfully created user" but a cant login, i have - "Unable to login"

adamsmesher commented 5 years ago

additionally, i have: Access to XMLHttpRequest at 'http://127.0.0.1:5000/api/v1/login' from origin 'http://ternal.local' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. in the Chrome dev tools

d3vzer0 commented 5 years ago

Could be an issue with the CORS environment setting in the docker-compose file. Can you check if the CORS_DOMAIN key is set to the correct domain/host that you're visiting? :)

PS. I also updated quite a lot of issues, be sure to create a backup/copy of your compose file and pull the latest changes ^^

adamsmesher commented 5 years ago

Thank you for replying! Now, i have fresh (without any changes) install, with default docker-compose file. And problem still present. I need to add domain to all containers to the hosts file ?...

d3vzer0 commented 5 years ago

The CORS_DOMAIN key should only be needed for the API service :) However, it could be that your host is starting the old/previously built container. Try running the below commands to rebuild and recreate the containers:

docker-compose stop
docker-compose build
docker-compose up -d --force-recreate

d3vzer0 commented 5 years ago

Ps. I updated the install guide with some extra details: https://github.com/d3vzer0/reternal-quickstart/wiki/Install--Guide

adamsmesher commented 5 years ago

my current compose file:

docker-compose.yml

#### ```version: '2' services: mongodb: image: mongo restart: unless-stopped networks: - rtn-backend ports: - 127.0.0.1:27017:27017 volumes: - mongodb-data:/data/db redis-service: image: redis restart: unless-stopped networks: - rtn-backend ports: - 127.0.0.1:6379:6379 api: build: ./backend depends_on: - mongodb - redis-service networks: - rtn-frontend - rtn-backend ports: - 127.0.0.1:5000:5000 environment: JWT_SECRET: WcdVkn0qfoh5aEJzweLzOqkrr FLASK_SECRET: qovYUxjZIgTu6KpFQWwfK6xEs CORS_DOMAIN: http://localhost C2_DEST: http://localhost:9000/api/v1/ping api-socket: build: ./backend depends_on: - redis-service networks: - rtn-backend environment: JWT_SECRET: 4Ura5MSJmejuyOTGatifn6907 FLASK_SECRET: atVVrBKmUruhlJfl5yWGagY0M command: - celery - -A - app.tasks.listener.celery - worker - -Q - api compiler: build: ./agent depends_on: - redis-service networks: - rtn-backend ui: build: context: ./ui args: VUE_APP_BASEAPI: http://127.0.0.1:5000/api/v1 VUE_APP_SOCKETHOST: http://127.0.0.1:5000 depends_on: - api networks: - rtn-frontend volumes: - ./navigator-reternal.json:/usr/share/nginx/html/mitre-navigator/assets/config.json ports: - 80:80 c2: build: ./c2 depends_on: - redis-service - mongodb environment: C2_SECRET: JrAzsTndludmDI4n8V2ob7Zu3 C2_PORT: 9000 networks: - rtn-backend ports: - 9000:9000 volumes: mongodb-data: driver: local networks: rtn-frontend: driver: bridge rtn-backend: driver: bridge ```

docker-compose build docker-compose up -d --force-recreate

and after that, create user and importing all objects:

[emulation/Frameworks/reternal-quickstart]$ python manage.py -a install -t all
Username: admin
Password:
Role (User/Admin): Admin
{'result': 'failed', 'message': 'User already exists'}
{'result': 'success', 'message': 'Finished importing base commands'}
{'result': 'success', 'message': 'Finished loading Mitre techniques'}
emulation/Frameworks/reternal-quickstart/cli/imports/import_mapping.py:14: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_object = yaml.load(yamlfile)
{'result': 'success', 'message': 'Finished loading mapped techniques'}
{'result': 'success', 'message': 'Finished importing all objects'}

...and i still have "Unable to login". What else am I going to do?

d3vzer0 commented 5 years ago

Strange. Can you share your current compose config (without the secret keys) and the URL you're trying to visit?

adamsmesher commented 5 years ago

source of docker-compose.yml in previous message (spoiler, hide by markdown) :) url is: http://127.0.0.1

d3vzer0 commented 5 years ago

Ah I totally missed it, my bad :) This may sounds strange, but I think the issue is that CORS_DOMAIN is set to 'http://localhost' while you're visiting 'http://127.0.0.1'. According to the CORS security policy you're supposed to originate from the 'localhost' domain which is different than '127.0.0.1' as seen by browsers. Can you try visiting http://localhost instead and see if it works?

PS. If all fails try setting CORS_DOMAIN to "*" and see if it works for now. Note that this isn't secure but we can rule out some options :)

rowham commented 5 years ago

Solution: Just add CORS Everywhere extension to your browser. Here you can find CORS Everywhere extension for Firefox: https://addons.mozilla.org/en-US/firefox/addon/cors-everywhere/

adamsmesher commented 5 years ago

Thanks all! Problem solved!

d3vzer0 commented 5 years ago

Thanks all! Problem solved!

Good to hear :) I'll clarify some of the CORS options on the Wiki ^^

jordisk commented 5 years ago

@d3vzer0 hi, I'm having the same problem. I'm unable to login successfully on the platform. I have changed de CORS domain but still not success to log in.

root@redTeamTools:/opt/reternal-quickstart# cat docker-compose.yml
version: '2'
services:
  mongodb:
    image: mongo
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 127.0.0.1:27017:27017
    volumes:
    - mongodb-data:/data/db
  redis-service:
    image: redis
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 127.0.0.1:6379:6379
  api:
    build: ./backend
    depends_on:
    - mongodb
    - redis-service
    networks:
    - rtn-frontend
    - rtn-backend
    ports:
    - 127.0.0.1:5000:5000
    environment:
      JWT_SECRET: snNRR7tlKxi2t2Nw3FHT99Xu4
      FLASK_SECRET: mJ4wKJxQZBjBeEY9C6YnCHpMb
      CORS_DOMAIN: http://10.202.27.2
      C2_DEST: http://127.0.0.1:9000/api/v1/ping
  api-socket:
    build: ./backend
    depends_on:
    - redis-service
    networks:
    - rtn-backend
    environment:
      JWT_SECRET: n7TqTHo6JFCreoAYb9wliD4Gv
      FLASK_SECRET: m4BB59f90bmZX8pDYZSLSBemw
    command:
    - celery
    - -A
    - app.tasks.listener.celery
    - worker
    - -Q
    - api
  compiler:
    build: ./agent
    depends_on:
    - redis-service
    networks:
    - rtn-backend
  ui:
    build:
      context: ./ui
      args:
        VUE_APP_BASEAPI: http://127.0.0.1:5000/api/v1
        VUE_APP_SOCKETHOST: http://127.0.0.1:5000
    depends_on:
    - api
    networks:
    - rtn-frontend
    volumes:
    - ./navigator-reternal.json:/usr/share/nginx/html/mitre-navigator/assets/config.json
    ports:
    - 0.0.0.0:80:80
  c2:
    build: ./c2
    depends_on:
    - redis-service
    - mongodb
    environment:
      C2_SECRET: PJeOWqEWpZX6wFfFnYDHPvtWu
      C2_PORT: 9000
    networks:
    - rtn-backend
    ports:
    - 0.0.0.0:9000:9000
volumes:
  mongodb-data:
    driver: local
networks:
  rtn-frontend:
    driver: bridge
  rtn-backend:
    driver: bridge

d3vzer0 commented 5 years ago

@jordisk Do you happen to have some details regarding the host/ip that is running the other services? Is that the host @ 10.202.27.2? If so, you will also have to change the following options in the compose file to match the proper IP:

C2_DEST VUE_APP_BASEAPI VUE_APP_SOCKETHOST

When this is the case and you changed the values, try rebuilding the containers :) Short howto can be found @ https://github.com/d3vzer0/reternal-quickstart/wiki/Rebuilding---Updating

DefenceLogic commented 5 years ago

Dear D3vzer0,

I have same issue as not being able to login.

I am running on Ubuntu 18.04, all docker containers are up.

Here is my docker-compose.yml

version: '2'
services:
  mongodb:
    image: mongo
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:27017:27017
    volumes:
    - mongodb-data:/data/db
  redis-service:
    image: redis
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:6379:6379
  api:
    build: ./backend
    depends_on:
    - mongodb
    - redis-service
    networks:
    - rtn-frontend
    - rtn-backend
    ports:
    - 172.16.4.176:5000:5000
    environment:
      JWT_SECRET: ILFKLSDTrQLQQoLj04UDpCJL1
      FLASK_SECRET: m5AIBrBJlTKknnQpSeeGAP7jI
      CORS_DOMAIN: http://172.16.4.176
      C2_DEST: http://172.16.4.176:9000/api/v1/ping
  api-socket:
    build: ./backend
    depends_on:
    - redis-service
    networks:
    - rtn-backend
    environment:
      JWT_SECRET: nQUUvRLQ6FfWnfGtzmtOhmba0
      FLASK_SECRET: qZoJPy7vVCXCPhqiAAM9uU3Gf
    command:
    - celery
    - -A
    - app.tasks.listener.celery
    - worker
    - -Q
    - api
  compiler:
    build: ./agent
    depends_on:
    - redis-service
    networks:
    - rtn-backend
  ui:
    build:
      context: ./ui
      args:
        VUE_APP_BASEAPI: http://172.16.4.176:5000/api/v1
        VUE_APP_SOCKETHOST: http://172.16.4.176:5000
    depends_on:
    - api
    networks:
    - rtn-frontend
    volumes:
    - ./navigator-reternal.json:/usr/share/nginx/html/mitre-navigator/assets/config.json
    ports:
    - 172.16.4.176:80:80
  c2:
    build: ./c2
    depends_on:
    - redis-service
    - mongodb
    environment:
      C2_SECRET: XseYMSHUA4XYHEPS3GEd6aGM5
      C2_PORT: 9000
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:9000:9000
volumes:
  mongodb-data:
    driver: local
networks:
  rtn-frontend:
    driver: bridge
  rtn-backend:
    driver: bridge

Is there anything incorrect in there?

I have tried CORS anywhere extension but no luck.

I have a user on the system

Output of user creation

nihon@nihon-siem-dev:~/reternal-quickstart$ python3 manage.py -a install -t all
Username: tester
Password: 
Role (User/Admin): Admin
{'result': 'created', 'message': 'Succesfully created user'}
{'result': 'success', 'message': 'Finished importing base commands'}
{'result': 'success', 'message': 'Finished loading Mitre techniques'}
/home/nihon/reternal-quickstart/cli/imports/import_mapping.py:14: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_object = yaml.load(yamlfile)
{'result': 'success', 'message': 'Finished loading mapped techniques'}
{'result': 'success', 'message': 'Finished importing all objects'}

Any ideas?

d3vzer0 commented 5 years ago

Dear D3vzer0,

I have same issue as not being able to login.

I am running on Ubuntu 18.04, all docker containers are up.

Here is my docker-compose.yml

version: '2'
services:
  mongodb:
    image: mongo
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:27017:27017
    volumes:
    - mongodb-data:/data/db
  redis-service:
    image: redis
    restart: unless-stopped
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:6379:6379
  api:
    build: ./backend
    depends_on:
    - mongodb
    - redis-service
    networks:
    - rtn-frontend
    - rtn-backend
    ports:
    - 172.16.4.176:5000:5000
    environment:
      JWT_SECRET: ILFKLSDTrQLQQoLj04UDpCJL1
      FLASK_SECRET: m5AIBrBJlTKknnQpSeeGAP7jI
      CORS_DOMAIN: http://172.16.4.176
      C2_DEST: http://172.16.4.176:9000/api/v1/ping
  api-socket:
    build: ./backend
    depends_on:
    - redis-service
    networks:
    - rtn-backend
    environment:
      JWT_SECRET: nQUUvRLQ6FfWnfGtzmtOhmba0
      FLASK_SECRET: qZoJPy7vVCXCPhqiAAM9uU3Gf
    command:
    - celery
    - -A
    - app.tasks.listener.celery
    - worker
    - -Q
    - api
  compiler:
    build: ./agent
    depends_on:
    - redis-service
    networks:
    - rtn-backend
  ui:
    build:
      context: ./ui
      args:
        VUE_APP_BASEAPI: http://172.16.4.176:5000/api/v1
        VUE_APP_SOCKETHOST: http://172.16.4.176:5000
    depends_on:
    - api
    networks:
    - rtn-frontend
    volumes:
    - ./navigator-reternal.json:/usr/share/nginx/html/mitre-navigator/assets/config.json
    ports:
    - 172.16.4.176:80:80
  c2:
    build: ./c2
    depends_on:
    - redis-service
    - mongodb
    environment:
      C2_SECRET: XseYMSHUA4XYHEPS3GEd6aGM5
      C2_PORT: 9000
    networks:
    - rtn-backend
    ports:
    - 172.16.4.176:9000:9000
volumes:
  mongodb-data:
    driver: local
networks:
  rtn-frontend:
    driver: bridge
  rtn-backend:
    driver: bridge

Is there anything incorrect in there?

I have tried CORS anywhere extension but no luck.

I have a user on the system

Output of user creation

nihon@nihon-siem-dev:~/reternal-quickstart$ python3 manage.py -a install -t all
Username: tester
Password: 
Role (User/Admin): Admin
{'result': 'created', 'message': 'Succesfully created user'}
{'result': 'success', 'message': 'Finished importing base commands'}
{'result': 'success', 'message': 'Finished loading Mitre techniques'}
/home/nihon/reternal-quickstart/cli/imports/import_mapping.py:14: YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the default Loader is unsafe. Please read https://msg.pyyaml.org/load for full details.
  yaml_object = yaml.load(yamlfile)
{'result': 'success', 'message': 'Finished loading mapped techniques'}
{'result': 'success', 'message': 'Finished importing all objects'}

Any ideas?

Sorry for the late reply, totally missed your question. I updated the installation guide almost entirely. You can check out the new Wiki for a howto :)

albmj commented 4 years ago

Hi D3vzer0, Love the project, but I have run into an issue:

Following the Guide from 1.A, I ran through all the steps and the installation seemed to complete without errors. All the docker containers are running, and the website is up, but when logging in I get the "Unable to login" error. Just looking at the request in chrome shows a 500 Internal Server Error. After digging through the docker logs, the error seems to originate from the reternal/backend docker image, named reternal-backend ("/bin/sh -c 'python …").

The final line from the python traceback is: pymongo.errors.OperationFailure: command find requires authentication

It looks like a permission error, but I am unsure how to go about fixing it, any ideas?

Here is a full dump from the docker log: 2019-11-25T11:29:07.839711146Z 172.18.0.8 - - [25/Nov/2019 11:29:03] "POST /api/v1/login HTTP/1.1" 500 48581 0.016001 2019-11-25T11:29:07.839738797Z (10) accepted ('172.18.0.8', 57892) 2019-11-25T11:29:07.839743422Z Traceback (most recent call last): 2019-11-25T11:29:07.839746403Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2449, in wsgi_app 2019-11-25T11:29:07.839749609Z response = self.handle_exception(e) 2019-11-25T11:29:07.839752530Z File "/usr/local/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function 2019-11-25T11:29:07.839773166Z return cors_after_request(app.make_response(f(*args, **kwargs))) 2019-11-25T11:29:07.839792756Z File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router 2019-11-25T11:29:07.839796341Z return original_handler(e) 2019-11-25T11:29:07.839799230Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1866, in handle_exception 2019-11-25T11:29:07.839802418Z reraise(exc_type, exc_value, tb) 2019-11-25T11:29:07.839805552Z File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise 2019-11-25T11:29:07.839808533Z raise value.with_traceback(tb) 2019-11-25T11:29:07.839811321Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 2446, in wsgi_app 2019-11-25T11:29:07.839814328Z response = self.full_dispatch_request() 2019-11-25T11:29:07.839817138Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1951, in full_dispatch_request 2019-11-25T11:29:07.839820093Z rv = self.handle_user_exception(e) 2019-11-25T11:29:07.839822891Z File "/usr/local/lib/python3.6/site-packages/flask_cors/extension.py", line 161, in wrapped_function 2019-11-25T11:29:07.839825909Z return cors_after_request(app.make_response(f(*args, **kwargs))) 2019-11-25T11:29:07.839828800Z File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 269, in error_router 2019-11-25T11:29:07.839831789Z return original_handler(e) 2019-11-25T11:29:07.839834519Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1820, in handle_user_exception 2019-11-25T11:29:07.839837468Z reraise(exc_type, exc_value, tb) 2019-11-25T11:29:07.839840274Z File "/usr/local/lib/python3.6/site-packages/flask/_compat.py", line 38, in reraise 2019-11-25T11:29:07.839843205Z raise value.with_traceback(tb) 2019-11-25T11:29:07.839845966Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request 2019-11-25T11:29:07.839848873Z rv = self.dispatch_request() 2019-11-25T11:29:07.839851740Z File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1935, in dispatch_request 2019-11-25T11:29:07.839854689Z return self.view_functions[rule.endpoint](**req.view_args) 2019-11-25T11:29:07.839857557Z File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 458, in wrapper 2019-11-25T11:29:07.839862095Z resp = resource(*args, **kwargs) 2019-11-25T11:29:07.839865132Z File "/usr/local/lib/python3.6/site-packages/flask/views.py", line 89, in view 2019-11-25T11:29:07.839868063Z return self.dispatch_request(*args, **kwargs) 2019-11-25T11:29:07.839870883Z File "/usr/local/lib/python3.6/site-packages/flask_restful/__init__.py", line 573, in dispatch_request 2019-11-25T11:29:07.839873843Z resp = meth(*args, **kwargs) 2019-11-25T11:29:07.839876659Z File "/reternal-backend/app/api_generic.py", line 53, in post 2019-11-25T11:29:07.839879541Z validate = User(args.username).login(args.password) 2019-11-25T11:29:07.839885077Z File "/reternal-backend/app/operations/op_user.py", line 13, in login 2019-11-25T11:29:07.839915110Z user_object = Users.objects.get(username=self.username) 2019-11-25T11:29:07.839918275Z File "/usr/local/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 266, in get 2019-11-25T11:29:07.839921243Z result = six.next(queryset) 2019-11-25T11:29:07.839924055Z File "/usr/local/lib/python3.6/site-packages/mongoengine/queryset/base.py", line 1485, in __next__ 2019-11-25T11:29:07.839927090Z raw_doc = six.next(self._cursor) 2019-11-25T11:29:07.839929896Z File "/usr/local/lib/python3.6/site-packages/pymongo/cursor.py", line 1156, in next 2019-11-25T11:29:07.839932860Z if len(self.__data) or self._refresh(): 2019-11-25T11:29:07.839935691Z File "/usr/local/lib/python3.6/site-packages/pymongo/cursor.py", line 1073, in _refresh 2019-11-25T11:29:07.839938606Z self.__send_message(q) 2019-11-25T11:29:07.839941335Z File "/usr/local/lib/python3.6/site-packages/pymongo/cursor.py", line 955, in __send_message 2019-11-25T11:29:07.839955637Z address=self.__address) 2019-11-25T11:29:07.839959054Z File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1347, in _run_operation_with_response 2019-11-25T11:29:07.839962119Z exhaust=exhaust) 2019-11-25T11:29:07.839964878Z File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1465, in _retryable_read 2019-11-25T11:29:07.839967894Z return func(session, server, sock_info, slave_ok) 2019-11-25T11:29:07.839970738Z File "/usr/local/lib/python3.6/site-packages/pymongo/mongo_client.py", line 1341, in _cmd 2019-11-25T11:29:07.839973628Z unpack_res) 2019-11-25T11:29:07.839976389Z File "/usr/local/lib/python3.6/site-packages/pymongo/server.py", line 136, in run_operation_with_response 2019-11-25T11:29:07.839979396Z _check_command_response(first) 2019-11-25T11:29:07.839982162Z File "/usr/local/lib/python3.6/site-packages/pymongo/helpers.py", line 159, in _check_command_response 2019-11-25T11:29:07.839985119Z raise OperationFailure(msg % errmsg, code, response) 2019-11-25T11:29:07.839987919Z pymongo.errors.OperationFailure: command find requires authentication

habbdt commented 4 years ago

I am also having the same issue as mentioned by @albmj. I tried it twice in two separate environments. In both cases, I have seen the same result. Any thoughts on how to resolve this issue. Thanks.

d3vzer0 commented 4 years ago

@habbdt @albmj Hiya! It probably involves the API container not setting the environment variables properly for the Username/Password in order to connect with the mongo database. If it's not set, the API will attempt to connect with mongo without using authentication. This is probably because I made some changes and have to sync the QuickStart repo with the latest version of the API repo that set these variables :) I'll take a look and see if the documentation is still up to date with the Ansible playbooks as well

d3vzer0 / reternal-quickstart

Unable to login #5