Closed MukuFlash03 closed 6 days ago
Checking in the first initial set of changes for consolidating the differences in the external and internal versions. These are based on the differences identified here
Currently included these changes:
Next are various changes to consolidate differences; made in this commit.
Changed sed to jq in start.sh.
Changed dash version to latest v 2.16.1 Detailed information available in the differences table mentioned at the top.
Got steps from @jiji14 which are applicable on existing code on main branch (before redesign code changes). For prod like testing, use docker-compose-prod.yml file instead of docker-compose-dev.yml in the below commands.
# Copy config file
cp config-fake.py config.py
# Run build script
and containers
docker compose -f docker-compose-dev.yml build
docker compose -f docker-compose-dev.yml up -d
# Load data
./docker/load_mongodump.sh tar_data_file (name of the file containing data)
# Change line 10 in start.sh
sed "s-localhost-${DB_HOST}-" conf/storage/db.conf.sample > conf/storage/db.conf
# Change DB_HOST in docker-compose-dev.yml
DB_HOST=mongodb://db/openpath_stage
# Run docker containers
docker compose -f docker-compose-dev.yml up -d
# Go to localhost:8050 and login
username = hello
password = world
Steps after redesign No need to copy config file or modify sed command as these are not used anymore in redesigned code changes.
# Run build script
and containers
docker compose -f docker-compose-dev.yml build
docker compose -f docker-compose-dev.yml up -d
# Load data
./docker/load_mongodump.sh tar_data_file (name of the file containing data)
# Change DB_HOST in docker-compose-dev.yml
DB_HOST=mongodb://db/openpath_stage
# Run docker containers
docker compose -f docker-compose-dev.yml up -d
# Go to localhost:8050 and login
username = hello
password = world
Tested external repo code both with redesign code changes. Admin-dash working with data and values can be seen in dashboard.
Screenshots available with me showing values being loaded: Data tab and Tokens tab
Testing build from Mukul's server code:
Changed the build image in the Dockerfile to the consolidate-differences image:
Ran $ docker build -t mukul-admindash .
to build the image.
Had to turn off the VPN momentarily due to an HTTP request being blocked + keeping it from building.
Loaded cortezebikes snapshot tar.gz data into the database using:
./load_mongodump.sh ../data/openpath_prod_cortezebikes_*_*.tar.gz
I did not want to completely rebuild a db container and rename, so I changed the name of the database in load_mongodump.sh to just be db:
Confirmed that the data was loaded in by using docker exec -it db /bin/bash
and running:
mongo
show dbs
The data that I loaded was there:
Ran docker run:
$ docker run --name mukul-int-admin-dash-1 -d -e DASH_SERVER_PORT=8050 -e DB_HOST=mongodb://db/openpath_prod_cortezebikes -e WEB_SERVER_HOST=0.0.0.0 -e CONFIG_PATH="https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/" -e STUDY_CONFIG=cortezebikes -e DASH_SILENCE_ROUTES_LOGGING=False -e SERVER_BRANCH=master -e REACT_VERSION="18.2.0" -e AUTH_TYPE="basic" -p 8050:8050 --network emission mukul-admindash
Of note: I kept getting an error when the STUDY_CONFIG variable was enclosed in quotes. This is strange, because the CONFIG_PATH variable was fine to be enclosed in quotes, and the CONFIG_PATH is concatenated with STUDY_CONFIG after being read in as environment variables.
After waiting for a moment for the data to load, I was able to access the login screen at localhost:8050. I logged in with hello:world, and saw that the data were present!
(intentionally sharing the least consequential columns)
The sign-ups trend graph was also shown, I just don't want to show any actual data. I have the screenshot to show it if necessary.
Internally testing admin dash build:
Cd’ed into local version of internal nrelopenpath multi-tier branch . Changed the Dockerfile to build off of the local image I created in the last test:
Built the image with
docker build -t mukul-internal-admin-dash ./admin_dashboard
Ran the container with:
docker run --name mukul-int-admin-dash-2 -d -e DASH_SERVER_PORT=8050 -e DB_HOST=mongodb://db/openpath_prod_cortezebikes -e WEB_SERVER_HOST=0.0.0.0 -e CONFIG_PATH="https://raw.githubusercontent.com/e-mission/nrel-openpath-deploy-configs/main/configs/" -e STUDY_CONFIG=cortezebikes -e DASH_SILENCE_ROUTES_LOGGING=False -e SERVER_BRANCH=master -e REACT_VERSION="18.2.0" -e AUTH_TYPE="basic" -p 8050:8050 --network emission mukul-internal-admin-dash
Note that I again had to remove the quotations around the value for STUDY_CONFIG. After going to localhost:8050, everything looked exactly the same as before!
User numbers blocked out to protect user privacy.
Merged changes from the tag automation branch, so now going to cleanup + prepare for merge with main/implementation
To respond to this we ran into issues with the COPY function in the dockerfile. We can keep it as is, or I can change all of the docker composes to build off of the Dockerfile in the main directory. The two docker files are identical aside from their locations.
Other than that, I have merged the tag update functionality + prepared all of the files to access the correct branches and directories for the automated tag updating.
Squash merging again, with 90 commits for 13 files.
This PR serves as the implementation for this issue for redesigning our build and deployment processes across e-mission-server, join, admin-dash, public-dash primarily.
Collaborating on this along with @nataliejschultz .
All four redesign PRs for the external repos: E-mission-server: https://github.com/e-mission/e-mission-server/pull/961 Join: https://github.com/e-mission/nrel-openpath-join-page/pull/29 Admin-dash: https://github.com/e-mission/op-admin-dashboard/pull/113 Public-dash: https://github.com/e-mission/em-public-dashboard/pull/125
A rough plan can be:
1st phase
2nd phase: