search

onyxfish / votersdaily

A project to parse the content of diverse government schedules into a consistent format.
GNU General Public License v3.0
12 stars 3 forks source link

Define standard logic for escaping JSON string delimiters in fields. #56

Open onyxfish opened 15 years ago

onyxfish commented 15 years ago

I can't find a canonical reference on this issue, but it looks like we need to be careful to escape in single or double quotes in our strings. I'm not sure, but I think the Python library might already be handling this. Need a standard approach so that data is not lost (however minor) is not lost.

onyxfish commented 15 years ago

According to this: http://www.webmonkey.com/tutorial/Get_Started_With_JSON

It looks like quotes just need to be escaped for the string to remain valid Javascript. I haven't found any indications that there are other problem characters, either for JSON or for CouchDB. We should probably make this standard practice for all text fields.

chaunceyt commented 15 years ago

So as a rule escape all string fields regardless. PHP - addslashes() within the scraper.

I will create a _escape_str() method for the PHP abstract class.

onyxfish commented 15 years ago

Sounds good. I'm going to see if couchdb-python is already doing this and if not I'll escape all the strings myself.