Enable relational lenses by default?

[jamescheney]

Once #894 is fixed, are relational lenses stable enough to be enabled by default?

It would also be good to have a bit more documentation and some tutorial examples (which would be easiest to accommodate using sqlite3).

[rudihorn]

I guess this depends on what our guidelines are for enabling features by default. I don't think they disturb much else which is most important. A few more examples would probably be nice though. The most important issues of #894 will be fixed with #897, though unit tests will probably not be looked at for a while.

[rudihorn]

It might be nice to have language integrated create table if not exists though, especially for serial columns. This would help with creating concise Sqlite examples. Maybe an intern project?

[rudihorn]

I'm also wondering if it would be possible / useful to use the Links Serial type defined in the prelude for non lens tables as well. The current lensserial syntax isn't great.

[jamescheney]

I think I can live with the unit tests not being generalized, as long as the actual end-to-end behavior is being tested for all three databases, so that we know if we break something major.. The unit tests to be more about testing the logic of relational lens delta translation.

Ur/Web has an initialization mode which automatically creates the table mentioned in the program (or checks that there are existing tables with the right schema). I'm a bit ambivalent about this, and tend to prefer making an actual update type system that tracks the side-effect of updates on the available table types (possibly as a DSL for scripts that are typically run once for initialization rather than within a normal Links program). But obviously this is a much bigger extension.

I don't think having lenses enabled by default should stop further development or cleanup of the interface - as long as we're clear it is still an experimental feature. But I'm also happy with keeping it clearly marked as "experimental" by guarding it by a flag, so that the dozens of Links developers out there don't get over-reliant on the current syntax.

[jamescheney]

To be more specific, what I'm ambivalent about regarding Ur/Web's approach is that a single Links table type could correspond to many different SQL tables, including different ways of indexing and different "subtypes" or Links's high-level base types. Currently, Links doesn't need to care about this as long as the strings we get back from queries are correctly convertible to the corresponding Links types.

I think it's good for these to be under the control of whoever configures the database rather than having a one-size-fits-all schema generated from Links types. Alternatively, one could make Links types richer to express the distinctions SQL schemas provide but doing this in a portable way seems problematic. (Database-dependent types, anyone?)

I suppose these are also reasons why my idea of having a Links-like DSL for type-safe schema-changing updates would be more complicated than I'd hope as well.

[rudihorn]

What I was suggesting was mainly a "get started quickly" function, which should not try to do anything fancy or support creating all SQL table variants. I was thinking of something that more targets creating small examples and helps users wanting to experiment with links. The only "advanced" thing it should do, is try to detect single id keys and turn them into serial columns.

[rudihorn]

My lens unit test code did something like:

  let create_table test_ctx db tablename (primary_key : string list)
      (fields : string list) =
    let colfn col =
      let open Database in
      db.quote_field col ^ " INTEGER NOT NULL"
    in
    let query =
      let open Database in
      "CREATE TABLE "
      ^ db.quote_field tablename
      ^ " ( "
      ^ List.fold_left (fun a b -> a ^ colfn b ^ ", ") "" fields
      ^ "CONSTRAINT "
      ^ db.quote_field ("PK_" ^ tablename)
      ^ " PRIMARY KEY ("
      ^ List.fold_left
          (fun a b -> a ^ ", " ^ b)
          (List.hd primary_key) (List.tl primary_key)
      ^ "));"
    in
    print_table_query test_ctx query;
    let open Database in
    db.execute query

  let create_table_easy test_ctx db tablename str =
    let fd = fundep_of_string str in
    let left = Lens.Fun_dep.left fd in
    let right = Lens.Fun_dep.right fd in
    let cols = Lens.Alias.Set.union left right in
    create_table test_ctx db tablename
      (Lens.Alias.Set.elements left)
      (Lens.Alias.Set.elements cols)

I was thinking of something like this that uses pretty printing, supports bools, strings and floats and can be called from Links code.

[jamescheney]

I'm increasingly leaning towards making sqlite3 the default database to use for "getting started with Links". Now that relational lenses work with sqlite3, we can use them in the tutorial. Also, sqlite3 supports creating temporary in-memory databases - though actually initializing them within a Links program would still require some way of running "CREATE TABLE" statements from within Links. The easiest way to do this would of course be to expose the query statement exec function as a Links library call...

Perhaps something like the following would be a good compromise (which would incidentally also work well with Sqlite3 in-memory databases):

• When we execute a database expression, connect to the database server and check that a database with the given name actually exists there. If not, create it (with appropriate permissions for the links user).
• When we execute a table expression, check that a table (or view) with a matching schema exists on the database. ("Matching" should mean: all of the fields exist in the table, and perhaps, the SQL types match the Links types. But there are many DB-specific SQL types to consider.) If such a table does not exist, create one with a default schema, e.g. interpreting integers, strings, etc. as the most liberal corresponding database type.

After that, the DB administrator is free to update (in a way that preserves the alignment with Links), the next time the Links program runs we will check to make sure things stayed in sync.

Note that this is also a limited form of bidirectionality...

[jamescheney]

I think we agreed to do this for 0.9.4, simply by changing the default value of the relational lenses flag to true, and maybe we just get rid of the flag later. Issue #959 documents additional improvements for relational lenses that may or may not be prioritized for 0.9.4.