clojurewerkz / titanium

Clojure graph library built on top of Titan
106 stars 20 forks source link

How should the views in here effect the organization of Titanium? #5

Closed zmaril closed 11 years ago

zmaril commented 11 years ago

http://stuartsierra.com/2013/03/29/perils-of-dynamic-scope

michaelklishin commented 11 years ago

Don't worry about it.

zmaril commented 11 years ago

That's not a good answer. He specifically calls out the main abstraction in Titanium as an antipattern and one which causes problems when you try to use Clojure to do certain things.

1.) You can’t dispatch to another thread. 
Say goodbye to Agents, Futures, thread pools, non-blocking I/O, or any other kind of asynchrony. 
The resource is only valid on the current thread.

2.) You can’t return a lazy sequence backed by the resource because the resource will be destroyed as soon as body returns.

3.) You can’t have more than one resource at a time. 
Hence the “singleton” in the name of this pattern. 
Using a thread-bound Var throughout the API means that you can never operate on more than one instance of the resource in a single thread.
Lots of apps need to work with multiple databases, which really sucks using this kind of library.

1 and 3 are only solved when you use titanium.graph/transact!. And that might only work based on which backend you choose to use.

2 is much more problematic. This has happened to me on multiple occasions and it is definitely a pain point. Transactions provide a nice "conceptual scope" of work, but if the end result is a lazy sequence over something, you have to remember to use a doall before the transaction gets closed. If we could find a way to either force transactions to always be used and returned lazy sequences to always be evaluated, it would make working with Titanium much easier.

michaelklishin commented 11 years ago

See this discussion. This pattern works well for most users. Monger provides extra arities that accept a specific DB instance you want to work on, the problem is that with type hinting, you only have up to 4 arities and varargs reduce that number even further.

Titanium specifically does not use varargs heavily so it may be a good option. But Elastisch, Neocons, Langohr do use them and so it becomes nearly impossible to do. Explicit arguments punish people who have basic needs in favor of people who return lazy sequences from separate threads. I'd argue the latter accounts for a really small % of developers and use cases.

michaelklishin commented 11 years ago

Lazy sequences are only really used when you traverse the graph. How often do you traverse a graph concurrently? Shouldn't Titan and your query language largely do the heavy lifting so you don't have to make traversals concurrency and/or parallel?

I really don't see #2 as a strong enough argument and the other two are solved with clojurewerkz.titanium.graph/transact! and additional arities.

zmaril commented 11 years ago

That all makes sense. Transactions seem to cover threading well enough, and lazy sequences are probably problematic no matter how we deal with them. I don't think Titanium uses any lazy sequences anywhere, so the developer would have to introduce them. A note in the documentation transactions would be a good idea then.