Discuss related work

[3noch]

Here's is some related work that would be helpful to review and discuss. Perhaps some it has valuable insights/techniques that can be used in Project:M36:

• https://github.com/stepcut/relatable
• https://github.com/Gabriel439/Haskell-Bears-Library

[agentm]

Thanks! I hadn't seen the relatable prototype before; it reminds me of vinyl-records.

[agentm]

One immediate issue that jumps out at me is whether or not relatable support nested relations. I would need to experiment with it to confirm, but I would guess not. Also, I am wondering how far we might otherwise get without Template Haskell.

The new GHC 8 custom error reporting would also come in handy for type errors.

[agentm]

@3noch, was there anything specific about these modules that you wish to discuss?

One issue that is probably intractable is mixing static relvar/Haskell record declarations with dynamic runtime queries. I do recall seeing a template-haskell-based solution which actually queried the database to validate the types against the actual remote schema at compile-time, but I question the worth of this approach since it seems to offer a false sense of security: in production, the schema could be anything.

What do you think?

[3noch]

@agentm I thinking having the ability to check a schema against a running database is pretty cool, and could add a layer of safety before deployments. However, I think that having a good migration system is still required regardless, and it gets you 98% of the way there if you run your tests against a migration-created database. If you can compare your code schema against the schema generated by your migration sequence, that would get you 99% of the way there.

[3noch]

A type-safe SQL DSL: https://github.com/valderman/selda

[agentm]

I took a look at Selda and it indeed does look promising. One thing that turns me off (and this is not the only framework with the problem) is that the DSL assumes the data types I want back are the unpacked tuples. This precludes support for relation-valued attributes which is essential to the relational algebra (much less so for SQL which doesn't support it at all). Also, the NULL handling appears awkward, but that is to be expected due to the impedance mismatch.

In summary, I would like to see a similar framework for Project:M36 but with these properties:

• marshal data types using Tupleable instance via Generics
• convenient insert/update/delete methods
• clear transaction semantics
• some sort of schema validation/versioning at startup to ensure that the convenience functions above can't blow up due to schema changes

But, slightly off-topic, how did you come across this framework? If there is a listing of interesting and upcoming Haskell frameworks, I would like to subscribe.

[3noch]

@agentm Great thoughts!!

Haha, I am the listing. ;) This one happened to hit Haskell reddit today. But I found it a while ago because I follow a lot of people and I religiously look through my GitHub activity feeds.

[3noch]

https://github.com/ocharles/rel8

[agentm]

Here's another SQL-inspired JS database relying on web storage: https://github.com/google/lovefield . It might inspire a similar interface for this project.

[3noch]

https://hackage.haskell.org/package/beam

[3noch]

http://khibino.github.io/haskell-relational-record/

[3noch]

On the git-like storage and branching model: https://github.com/mirage/irmin

[3noch]

Type-safe API: https://www.morphism.tech/announcing-squeal/

[agentm]

Do you think that squeal is an API direction we should pursue?

What turns me off to it is the strong SQL-specific bindings type-wise and data wise. While I can appreciate being able improve code confidence at compile-time, I wonder how the code might handle a schema changing over time or if the expected types don't match. I imagine those must be thrown as exceptions.

In short, the current Atomable and Tupleable interfaces provide most of the same services, but don't offer the same compile-time guarantees. In addition, it looks like there is no static guarantee between the Haskell type and the db type in squeal, but do correct me if I am wrong.

Because of the SQL impedance mismatch, in squeal, the user is forced to define an SQL mapping, which we don't require, so I think we win there as well.

As you work with the new Tupleable interface, please do open any new issues on things that could be easier or offer better compile-time guarantees.

[3noch]

@agentm I'm just keeping a log of everything I see that could be useful down the road. I definitely agree that no amount of type-safety will make SQL ideal to work with. Project M36 has a big advantage already in that regard, even without the powerful type-safety. However, I'm sure we'll want to add more type-safety in the future and we can use these as examples.