schema in PG.Table option object

[robfallows]

I'm thinking it would be cool to start building out the options object by adding a schema object. This could (on the server) create the table if it doesn't exist and validate it if it does. We could move closer to the simplicity of the Mongo.Collection with this functionality and reduce the need for migrations to set stuff up.

I've been spitballing a few ideas and I think it would be easy to create the table (less easy to validate, but still doable).

[Slava]

I think the database should be the main source of schema, not the application code. This is why migrations are important - the database migrations are driving the schema changes.

[stubailo]

Yeah we actually had this at first and then removed it because it turns out that having migrations be separate is a good idea. But it could be cool to verify that the scheme matches and remind the developer to run migrations!

[robfallows]

I would like to make a plea to rethink this.

We have an application which allows our customers to write scripts for data collection. The data collected is then sent to a central (MySQL) database which warehouses it. We have no prior knowledge of our customers' scripts or data, other than some structure around the transmitted data, which allows us to identify the "name" for this new thing and the column names and types required. We then create a table on-the-fly to store this data. We have no way of using migrations to set up the necessary table(s) beforehand.

Ironically, we identified a while back that the best way to store this data was in MongoDB. However, for commercial reasons we are migrating the warehousing and analytics to SAP HANA (another SQL database). Once again, we are creating tables on the fly to accomodate new customer scripts.

We also have a separate database for this application which contains configuration data, customer metadata, user accounts, etc. In contrast to the customer data tables, the schema for this database is tightly managed: change control and migrations are strictly followed.

On the scale of kindergarten to MIT, Meteor's core, its functionality, reliability, performance and integration sits at the MIT end. On the other hand, its speed of development and ease of use lies in the other direction. Anything which "gets in the way" of rapid application development, or proof of concept is a bad thing. One of the truly great things about Meteor is that it manages this apparent dichotomy really well.

To add hurdles (install a database and write/run migrations) is fine while functionality is still being developed, but is a real disconnect when coming from "current" Meteor. I understand that database installation will likely be addressed, but making it difficult to add schema operations in the code is only going to result in a (more likely many) community package(s) to put this functionality in.

I'm fully behind migrations - they're a requirement of any serious database solution, but they are sometimes insufficient, and it becomes necessary to allow schema operations in code. Surely it's much better to grasp the nettle now and do it definitively in the core package?