Upgrade version procedure don't imported data

[radiusmk]

Hi everyone;

I upgraded my Vectr installation following the procedures described in https://docs.vectr.io/VECTR_v5_5_Migration/, but the old data didn't was imported.

I runned the MongoDB Upgrade Tool and copy data to folder ./user/mongo, but any data was imported. I tried remove Mongo container and restart all, but in logs I can't see anything about the importation of data.

The MongoDB Upgrade Tool runned well and created folders and files of my databases, so I think the problem is the new installation that don't read data on .user/mongo directory.

Anyone have idea that I can do?

Thanks. Marco Aurélio

[thebleucheese]

Your user/mongo folder should contain folders like this:

Your folder will not have these exact names, the names of the folders will match your database names.

If that's what your user/mongo folder looks like and it's still not loading you'll need to delete your existing VECTR data directory. I recommend backing this folder up somewhere prior to deleting. This is in one of a few places depending on your version with the two most likely being:

<vectr_deploy_dir>/data

OR

/var/data/<docker_project_name>

If it's not in either of these places, you can check the mounted volume for the mongo container's data folder in the docker-compose.yml file.

[radiusmk]

Hello, thanks for reply;

In Vectr user/mongo there are files like your example:

[root@myserver]# pwd
/opt/vectr/user/mongo
[root@myserver]# ls -l
total 32
drwxr-xr-x 2 root root   67 Aug 12 16:15 admin
drwxr-xr-x 2 root root 4096 Aug 12 16:15 assessmentGroup
drwxr-xr-x 2 root root 4096 Aug 12 16:15 DEMO_ATOMIC_RED_CE
drwxr-xr-x 2 root root 4096 Aug 12 16:15 DEMO_PURPLE_CE
drwxr-xr-x 2 root root 4096 Aug 12 16:15 GoldStandard
drwxr-xr-x 2 root root 4096 Aug 12 16:15 PurpleTeam
drwxr-xr-x 2 root root 8192 Aug 12 16:15 TempImport
[root@myserver]

I tried stop containers, delete the data directory (/opt/vectr/data) and start containers again, and the data wasn’t imported.

[root@myserver]# rm * -rf
[root@myserver]# pwd
/opt/vectr/data
[root@myserver]# ls -l
total 0
[root@myserver]# cd ..
[root@myserver]# docker-compose up

I confirmed that user/mongo folder is mounted inside Tomcat container:

[root@myserver]# docker exec -it bb8cf47461d9 bash
root@bb8cf47461d9:/usr/local/tomcat# cd /opt/vectr/user/mongo/
root@bb8cf47461d9:/opt/vectr/user/mongo# ls -l
total 32
drwxr-xr-x 2 root root 4096 Aug 12 19:15 DEMO_ATOMIC_RED_CE
drwxr-xr-x 2 root root 4096 Aug 12 19:15 DEMO_PURPLE_CE
drwxr-xr-x 2 root root 4096 Aug 12 19:15 GoldStandard
drwxr-xr-x 2 root root 4096 Aug 12 19:15 PurpleTeam
drwxr-xr-x 2 root root 8192 Aug 12 19:15 TempImport
drwxr-xr-x 2 root root   67 Aug 12 19:15 admin
drwxr-xr-x 2 root root 4096 Aug 12 19:15 assessmentGroup
root@bb8cf47461d9:/opt/vectr/user

I didn't any changes in docker-compose.yml:

[root@myserver]# more docker-compose.yml
version: '3'
services:
  mongo:
    image: mongo:4.2
    networks:
      vectr_bridge:
        aliases:
          - vectr-mongo
          - oauth2-mongo
    volumes:
      - ${VECTR_DATA_DIR:-./data/}${COMPOSE_PROJECT_NAME}:/data/db
    env_file:
      - ./.env
    restart: unless-stopped

  tomcat:
    image: securityriskadvisors/vectr_tomcat:5.5.6
    networks:
      vectr_bridge:
        aliases:
          - ${VECTR_HOSTNAME}
    volumes:
      - ./resources:/opt/vectr/resources
      - ./user:/opt/vectr/user
      - ./logs:/usr/local/tomcat/logs
    env_file:
      - ./.env
    ports:
      - "${VECTR_PORT:-8081}:8443"
    restart: unless-stopped
    depends_on:
      - mongo

networks:
  vectr_bridge:
    ipam:
      driver: default
      config:
        - subnet: "${VECTR_NETWORK_SUBNET:-10.0.27.0/24}"
[root@myserver]

Are there some procedures that I can tried to manually start the import process?

[carlvonderheid]

In your .env file, did you change the VECTR_DATA_DIR to /opt/vectr/data or is it set to /var/data/?

You'll need to stop the containers, delete whatever path VECTR_DATA_DIR/COMPOSE_PROJECT_NAME is set to, then restart the containers to force the system to load whatever is in /opt/vectr/user/mongo.

[radiusmk]

I tried it:

[root@myserver]# pwd
/opt/vectr
[root@myserver]# cat .env | grep DATA_DIR
VECTR_DATA_DIR=/opt/vectr/data/
[root@myserver]# docker ps
CONTAINER ID        IMAGE               COMMAND             CREATED             STATUS              PORTS               NAMES
[root@myserver]# rm data/* -rf
[root@myserver]# ls data / -l
total 0
[root@myserver]# ls user/mongo/ -l
total 32
drwxr-xr-x 2 root root   67 Aug 12 16:15 admin
drwxr-xr-x 2 root root 4096 Aug 12 16:15 assessmentGroup
drwxr-xr-x 2 root root 4096 Aug 12 16:15 DEMO_ATOMIC_RED_CE
drwxr-xr-x 2 root root 4096 Aug 12 16:15 DEMO_PURPLE_CE
drwxr-xr-x 2 root root 4096 Aug 12 16:15 GoldStandard
drwxr-xr-x 2 root root 4096 Aug 12 16:15 PurpleTeam
drwxr-xr-x 2 root root 8192 Aug 12 16:15 TempImport
[root@myserver]#
[root@myserver]# docker-compose up -d
[root@myserver]# ls data/ -l
total 4
drwxr-xr-x 4 polkitd root 4096 Aug 14 12:09 XXXXvectr_5_5
[root@myserver]#

but in the vectr interface no database is founded:

[radiusmk]

Hello guys;

I tried remove all containers and images, and used "docker-compose.yml" of 5.6.3 version, but the data migration didn't work. I checked inside the Tomcat container and user/mongo folder was mapped correctly:

root@65584ddd770f:/usr/local/tomcat# cd /opt/vectr/user/mongo/
root@65584ddd770f:/opt/vectr/user/mongo#
root@65584ddd770f:/opt/vectr/user/mongo# ls -l
total 32
drwxr-xr-x 2 root root 4096 Aug 12 19:15 DEMO_ATOMIC_RED_CE
drwxr-xr-x 2 root root 4096 Aug 12 19:15 DEMO_PURPLE_CE
drwxr-xr-x 2 root root 4096 Aug 12 19:15 GoldStandard
drwxr-xr-x 2 root root 4096 Aug 12 19:15 PurpleTeam
drwxr-xr-x 2 root root 8192 Aug 12 19:15 TempImport
drwxr-xr-x 2 root root   67 Aug 12 19:15 admin
drwxr-xr-x 2 root root 4096 Aug 12 19:15 assessmentGroup
root@65584ddd770f:/opt/vectr/user/mongo#

The Vectr work well, but any data was migrated. Are there some log that I can see what is the problem? I can't find anything.

Thanks. Marco Aurélio

[SRAPSpencer]

It would be easiest to review this on a call. Can you email us at vectrops@securityriskadvisors.com so we can assist further?

[radiusmk]

Thanks for reply.

My team decided this week migrate all data manually. I think that problem is because we tried import data from an very old Vectr version (5.3.1).

Marco Aurélio

[SRAPSpencer]

Please let us know if you need anything further.