Add ability to track multiple direct data links on Election records?

[zstumgoren]

Should we update the Election model to add the ability to track multiple links to raw data files? Currently we only allow a single url in Election.direct_link, but it's quite common to have multiple data files associated with a single Election record. For example, in VA, we have separate files for a pair of state legislative races associated with the Sept. 4, 2012 Election record (right now I'm dropping these links into the Election.note field, which is gross).

Adding support for multiple data source links would allow us to better leverage the metadata API in the core framework, hopefully reducing complexity and increasing consistency in datasource.py implementations.

Two possible implementations:

• Quick & dirty - Turn Election.direct_link into a text field to allow longer strings and include help_text saying that multiple links should be inserted with a comma separator (or some other unique delimiter we don't expect to encounter in URLs).
• Clean way - Add a SourceLink model and reference as an M2M on Election model, such as Election.source_links?

[zstumgoren]

Ok, I'm also remembering why I punted on this in the original implementation. Adding an M2M to inline Election records on State admin pages was cumbersome. So if we do want to implement this, I'm thinking the quick & dirty route (change Election.direct_link to a text field is the path of least resistance. @dwillis @ghing If that doesn't make you all itch too much, I'd prefer to go that route to avoid getting bogged down in the implementation. Thoughts?

[dwillis]

Fine by me.

[ghing]

Let's try going that route if it ends up causing problems when consuming the metadata, we can revisit. I think you're right that having all metadata, but dirtier is better than having less data but cleaner.

I can also imagine an intermediate solution where we have both a text field and M2M fields and the text field gets parsed out into M2M fields in a post_save hook, avoiding the pain of making things work in the Django admin, but not adding work on the consumer end.

[zstumgoren]

Ok all, I've pushed Election.direct_links changes based on this thread and our offline conversation. Changes include:

• Adding direct_links field to admin and metadata API
• Copy data from Election.direct_link to Election.direct_links
• Hide Election.direct_link (singular) field in the admin, but keep it in the API for now to avoid breaking any datasource.py that might rely on it.

Let me know if you have questions or find bugs. Otherwise I'll close this ticket.

[ghing]

I'm happy you thought of splitting direct_links into a list in the API response. That makes a lot of sense and avoids having to decide where the parsing would go in downstream code. Nice design! :smile:

[zstumgoren]

@ghinb Excellent. Glad that works for downstream. Should we remove direct_link API field now or just create a placeholder ticket about removing it after we've ported datasource.py for various states? Of course, if you already know that it won't break anything, let me know and I can just remove it.

[ghing]

We'll need to update a few of the existing datasources, but that should be pretty lightweight. I'll do that today.

[zstumgoren]

Closing this ticket since work has been deployed. The API currently has direct_link and direct_links, for backwards compatibility.