Documentation and cleanup

[gregallensworth]

This is the result of my first looking-over of the data:

• Analysis of duplicate points and lines.
• A list of cleanup items, e.g. deletion of unnecessary fields.
• Documentation of tables and fields.

Desired review:

• The tables and fields documentation, and the proposed cleanup documentation, could use a review by jbranigan, to see if he notices any discrepancies or incorrect ideas.

• Duplicate points and lines have been removed, into new CARTO tables under the greeninfo account: ecglines_clean_unique and ecgpoints_clean_unique The techniques to detect them are documented in some new "NOTES" in the sql/ folder. If the techniques seem sound, then these new tables may be used moving forward, and the "NOTES" may be obsolete.

[clhenrick]

@gregallensworth not sure how busy @jbranigan is as he recently started a new job. To answer your question on the n_turn_type and s_turn_type fields: those are "normalized" direction values which can be used by the app to display an arrow icon next to cue text, so they are needed.

[gregallensworth]

Thanks for the feedback about those two fields. I'll update the docs.

[clhenrick]

@gregallensworth also, with the current data I'm not sure we can say for sure that with cue points pline_id is the northern most point of the corresponding line and that pline_id2 is the southern most point of the other corresponding line. In the screenshot below it seems to be the opposite. It could be that pline_id is south and pline_id2 is north for the whole route, but we haven't verified that.

Above is for the area outside of Providence, RI

[clhenrick]

For the final data schema it might make sense to name the fields pline_id and pline_id2 to something more semantic like line_id_n and line_id_s.

[clhenrick]

And here's a case where pline_id actually is north and pline_id actually is south to confirm that they don't have explicit associations with the start or end of a line in a particular direction.

[gregallensworth]

Thanks for the review. 3daf09b updates a few things:

• Cleanup todo items doc, the removal of duplicates is checked off as done
• Docs listing the dupes, removed as now obsolete.
• Note about pline_id and pline__id2 updated, to not imply the false north-south relationship.

[clhenrick]

Going to merge this in, if there's more data QA SQL or notes to add in later we can add that directly to the repo.