search

SecurityRiskAdvisors / VECTR

VECTR is a tool that facilitates tracking of your red and blue team testing activities to measure detection and prevention capabilities across different attack scenarios
1.39k stars 164 forks source link

Cannot Restore Vectr DB file using mongodump #57

Closed sanjeewai closed 4 years ago

sanjeewai commented 4 years ago

I'm unable to restore vectr JSON file to a newly created vectr application. Appreciate anyone can help with a script to do mongorestore or any other method to restore the file.

sanjeewai commented 4 years ago

my Docker container details are ; xxx@xxx:/srv/vectr$ sudo docker images REPOSITORY TAG IMAGE ID CREATED SIZE securityriskadvisors/vectr_tomcat 5.5.6 8dc4f4f6c3be 6 days ago 1.37GB mongo 4.2 c5e5843d9f5f 2 weeks ago 387MB mongo latest c5e5843d9f5f 2 weeks ago 387MB hello-world latest fce289e99eb9 15 months ago 1.84kB

xxx@xxx:/srv/vectr$ sudo docker ps [sudo] password for xxx CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 23239b5720c4 securityriskadvisors/vectr_tomcat:5.5.6 "/docker-entrypoint.…" 38 hours ago Up 16 hours 8080/tcp, 0.0.0.0:8081->8443/tcp sandbox1_tomcat_1 c79b310e6880 mongo:4.2 "docker-entrypoint.s…" 38 hours ago Up 16 hours 27017/tcp sandbox1_mongo_1

carlvonderheid commented 4 years ago

Hi, thanks for reaching out. Can you explain a little more on the process you did? Does "unable to restore" mean that you attempted and there was an error?

We have directions here to migrate to the new release: https://docs.vectr.io/VECTR_v5_5_Migration/

sanjeewai commented 4 years ago

Thank you for reaching out, Basically i have no idea to which folder i should restore the zip file. The folder has json & bson files. any easy utility which i can use to mount this file ? I used Portainer utility to navigate among the images but coudn't move the file in to the container.

folders in my mongo image are :

root@c79b310e6880:/# ls bin boot data dev docker-entrypoint-initdb.d etc home js-yaml.js lib lib64 media mnt opt proc root run sbin srv sys tmp usr var root@c79b310e6880:/#

SRAPSpencer commented 4 years ago

Where exactly did you get this mongodump from? Is this from a prior version of VECTR?

All prior versions of VECTR use a older MongoDB version. You won't be able to just import the mongodump, you need to upgrade the data as per the migration path. Our migration path assumes your old VECTR instance is still up and running. Do you have access to that instance where the data came from?

sanjeewai commented 4 years ago

No , I don’t have access to it. Due to the confidentiality I cannot share it with you to have a look. But would it be possible to have a zoom meeting with me? This would be much easier to resolve the issue and I greatly appreciate your help.

On Wed, Apr 15, 2020 at 5:23 PM Paul Spenc notifications@github.com wrote:

Where exactly did you get this mongodump from? Is this from a prior version of VECTR?

All prior versions of VECTR use a older MongoDB version. You won't be able to just import the mongodump, you need to upgrade the data as per the migration path. Our migration path assumes your old VECTR instance is still up and running. Do you have access to that instance where the data came from?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SecurityRiskAdvisors/VECTR/issues/57#issuecomment-614070404, or unsubscribe https://github.com/notifications/unsubscribe-auth/APE5KWBWDWVLBOCZMVJVKO3RMW7OBANCNFSM4MINUA6Q .

thebleucheese commented 4 years ago

Are you trying to get data from an older VECTR instance like 5.4 to the current 5.5.6?

Before we can assist any further you will need access to the filesystem of the original image - or a complete mongodump from the instance you're trying to restore.

If you performed a backup via the "backup databases" screen in the UI, that does not backup enough data to restore an entire instance or upgrade to a new version. It solely backs up individual VECTR session databases and does not have full access to the underlying mongo data structure. Therefore, backed up data from there will be missing dependencies and user data. If that is the only data you have, it's unfortunately incomplete and the instance or your data will not be able to be easily recovered or upgraded that way.

The raw mongodb data from your previous instance(normally at /var/data/sandbox1 or your /data directory) will have everything needed to migrate from 5.4 to 5.5.6.

Once you have access to this we'll be able to assist.

sanjeewai commented 4 years ago

Agree, I think if i can get a docker image backup of the old mongoDB container and load it in to my vectr db container that might work.

thebleucheese commented 4 years ago

There's a specific process that needs to be followed to upgrade data so it works with the new version. The Mongo major version upgrades included a lot of changes, and we had to upgrade from Mongo 3.4 to 4.2. We also added a password to the database. Mounting the data in the new container as-is probably won't work, but we can go through the steps to upgrade and get it running.

If you can get a backup of the old mongo db container please reach out to us and we can assist in the next steps with a zoom call: vectr@securityriskadvisors.com

sanjeewai commented 4 years ago

On Wed, Apr 15, 2020 at 6:27 PM thebleucheese notifications@github.com wrote:

There's a specific process that needs to be followed to upgrade data so it works with the new version. The Mongo major version upgrades included a lot of changes, and we had to upgrade from Mongo 3.4 to 4.2. We also added a password to the database. Mounting the data in the new container as-is probably won't work, but we can go through the steps to upgrade and get it running.

If you can get a backup of the old mongo db container please reach out to us and we can assist in the next steps with a zoom call: vectr@securityriskadvisors.com

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/SecurityRiskAdvisors/VECTR/issues/57#issuecomment-614107443, or unsubscribe https://github.com/notifications/unsubscribe-auth/APE5KWFJEO724QJWNPVKPJTRMXG4NANCNFSM4MINUA6Q .

Hi,

I still couldn’t get the raw database . But I’m able to see matching mongo db json and bson files as per the current db tables . Would it be possible to restore some data in my new instance if we do a zoom call? appreciate your help.

sanjeewai commented 4 years ago

Hi,

I’ve received the raw mongodb data file and I’m ready for a zoom call to fix this. sanjeewai88@gmail.com Thank you

sanjeewai commented 4 years ago

Issue resolved. Thanks to Carl & vectr team to helped out to the complete DB restoration and migration.

Cheers!