search

newyorkmyc / keys-typeahead

Creative Commons Zero v1.0 Universal
0 stars 0 forks source link

How do you add additional keys? #3

Open newyorkmyc opened 1 week ago

newyorkmyc commented 1 week ago

How do we make it easy to add additoinal key information to the sheet/application?

This should be part of the README.md

mrozanoff commented 1 week ago

I have them stored in a google sheets, the python builds the json and populates what we have from the data. This way multiple people can add together and it can be updated as we go. This way on the actual website all we have to do is update the json

newyorkmyc commented 1 week ago

Got it.

This is problematic with potential data pollution. How do we plan to force review of information?

Can we convert the .xsl to .csv and make it part of the application? We can surely use a good spreadsheet for submissions, but the actual implementation should be done elsewhere so it may be sanitized and reviewed.

What is the process to get it built out, does someone need to run the python script or is that automated?

Lets document how that is completed in the README.md

mrozanoff commented 1 week ago

Yeah I'll definitely add it all in the readme.

Data pollution from people who aren't us entering in data? Well we can review everything that is not entered by us. Were you thinking to send this out on a mass scale? Excel to CSV is no problem.

As of right now someone needs to run the python script. It can definitely be automated, but I don't know if that makes sense.

newyorkmyc commented 1 week ago

Yes, I think thats a lot of overhead for us to manage. There are a ton of keys out there, so it would be great to crowdsource this.

Doing so, we would need a way to aggregate the data as well as validate.

The python script could be ran on merge and even deployed as part of a CI/CD process.

mrozanoff commented 1 week ago

That makes sense. I don't see how we can reliably validate except manually.

Does it make sense to just have a separate google sheets that everyone can add to, and once we validate that we push it to the 'actual' sheet? We can have a separate locked tab for all the keys that we already have in the database so anyone can cross reference.

newyorkmyc commented 6 days ago

My concern with using a public data sheet is possible nefarious code slipping in. Taking a vary cautious approach here, we are basically looking to use the spreadsheet as our DB, so precautions need to be taken.

We could set up an e-mail account for keys submissions, or enable it so its comments only.

Regardless, we will then have to export the updated .csv file and submit a PR. That PR is where we can run some unit tests and or sanitizing of the code. The build process would automate the python script build and ssh & scp it to the server.