cisagov / cyhy-core

Core code for Cyber Hygiene (CyHy)
Other
7 stars 9 forks source link

Always import data in the GNIS data importer script #100

Closed mcdonnnj closed 1 month ago

mcdonnnj commented 1 month ago

🗣 Description

This pull request removes logic around checking if GNIS data is imported already. It changes the base behavior to always import GNIS data, thus making the --force option's behavior the default.

💭 Motivation and context

This change's core reason for being is that with the merge of #83 the check for existing records will now throw an error. This is because the csv.DictReader object begins reading the file to pull the CSV header before the record checking logic is run. This results in the following error: IOError: telling position disabled by next() call. In addition to the aforementioned bug the checking logic only looked for the first and last records in the source data. This meant that if records changed other than the first and last ones they would not be imported. This is undesirable behavior as it results in out-of-date or missing records.

🧪 Testing

I used this to redeploy the database instance in my testing environment without issue.

✅ Pre-approval checklist