search

andrewxhill / MOL

The Map of Life
mol.colorado.edu/
19 stars 4 forks source link

Use the field specification to set the datatypes for metadata fields in the database #121

Open gaurav opened 12 years ago

gaurav commented 12 years ago

At the moment, the metadata is stored in two places: on the backend in a PostgreSQL database, and on the frontend in the Google App Engine datastore. For conveniences' sake, I've currently set up all the fields as TEXT fields. However, it won't take too long to use the 'Type' column in the field specification [1] to set up these types instead.

Since indexed TEXT fields will probably/hopefully be fast enough for us now, I won't bother; but I'm just adding this bug to the issues list to remind me to make that change at some point.

[1] http://www.google.com/fusiontables/DataSource?dsrcid=1326977

tucotuco commented 12 years ago

It's not so much the speed I would worry about with indexed fields, but rather the sorting characteristics that differ between text and numeric data types. The results of queries would be affected. Why not do it correctly up front? Curious, not annoyed. ;-)

gaurav commented 12 years ago

I'm worried about downstream problems (having to modify metadata in the fields because it won't fit the spec type, having to pick spec types for tricky fields), but you're right, this won't take more than 30-45 minutes to get squared away. I'll probably take a bash at it today.

gaurav commented 12 years ago

This is essentially done as per @56f81b4, but I'll keep it in the same branch as issue #118 and leave this issue open until the metadata database is working smoothly. Most of the types ended up as TEXT anyway, except for a few DATEs, INTEGERs and SHORTINTs. But this will be a good foundation for more complex data types for metadata database fields, if necessary.