We now have both a prod db and a dev db: courseplannerdb and courseplannerdb-dev. They both live in the folder /opt/cp-course-db
added a --prod option for webscraping/node/course-seq/storer.js and webscraping/node/course-info/storer.js which will cause each of them to write to the prod DB. They write to the dev DB by default.
scrapeTheWeb.sh passes its first argument directly to the storers so you can perform a full production webscraping run via sh scrapeTheWeb.sh --prod
I restructured the Java code a little bit, adding an inheritance structure instead of using Util.java. More importantly, I have added courseplanner.properties which holds useful properties for the webapp. The DB used by default is courseplannerdb-dev.
See bottom of PR @stumash 👀
Changes on the vm not tracked by repo:
moved webscraping source files from /home/david/CoursePlanner/webscraping into /opt/cp-webscraping/
I needed to make sure the owner of this new dir was david and not root:
sudo chown -r david:david /opt/cp-webscraping
If we're deploying to prod, deploy.sh must replace the dbName property value with courseplannerdb inside courseplanner.properties. This should happen right before mvn clean install is executed.
Also when were deploying to prod, deploy.sh must sync up the webscraping source code between the repository and the remote directory: /opt/cp-webscraping. Perhaps use rsync and (important) exclude the node_modules
resolves #58 and resolves #59
Summary
courseplannerdbandcourseplannerdb-dev. They both live in the folder/opt/cp-course-dbwebscraping/node/course-seq/storer.jsandwebscraping/node/course-info/storer.jswhich will cause each of them to write to the prod DB. They write to the dev DB by default.scrapeTheWeb.shpasses its first argument directly to the storers so you can perform a full production webscraping run viash scrapeTheWeb.sh --prodUtil.java. More importantly, I have addedcourseplanner.propertieswhich holds useful properties for the webapp. The DB used by default iscourseplannerdb-dev.Changes on the vm not tracked by repo:
/home/david/CoursePlanner/webscrapinginto/opt/cp-webscraping/sudo chown -r david:david /opt/cp-webscraping0 0 * * * sh /opt/cp-webscraping/scrapeTheWeb.sh --prodChanges that must be made moving forward
If we're deploying to prod,
deploy.shmust replace thedbNameproperty value withcourseplannerdbinsidecourseplanner.properties. This should happen right beforemvn clean installis executed.Also when were deploying to prod,
deploy.shmust sync up the webscraping source code between the repository and the remote directory:/opt/cp-webscraping. Perhaps usersyncand (important) exclude thenode_modules