config/default.json
andconfig/custom-environment-variables.json
,./constants.js
npm run download-data
to download all available datasets.npm run import-data
to import all data using the downloaded files from the previous step.Before starting the application, make sure that PostgreSQL is running and you have configured everything correctly in config/default.json
npm i
npm run lint
npm start
. This will run all tools in the following sequence:npm run download-data
=> npm run import-data
The application will print progress information and the results in the terminal.
max_old_space_size
has been set to 4096MB to allow parse/process such huge data files without any issues. The app will clean the memory right after using the data to prevent memory/heap leaks.FOREIGN ADDRESSES
doesn't have a header in the CSV file and it has slightly different format (it has an extra column). The app handles all datasets without any issue.