Rules / selectors

[plexus]

Part of my vision with Facai is to have a system of CSS-like selectors, to be able to specify certain things at a higher, more expressive level. I've had a version of this in an earlier iteration, and for now took much of it out again, although remnants are still there, e.g. the way that a :facai.build/path is kept of where we are in the nested structure we're generating. The sel and sel1 functions also use selectors.

This is an issue to explain a bit where this all comes from, and to discuss the API and implementation. Now that I've written all of this out I think I more or less know where I want to go with this, but I'd still appreciate any input. I think's it's also a good idea to explain my reasoning at this point, since this could become one of the "killer features" of Facai that sets it apart from equivalents in other languages like factory_bot.

Background

In terms of alternatives that exist in the Clojure world for Facai I think Specmonstah comes closest to filling the same need as Facai. I've used Specmonstah on some projects, and like many things about it, but it's not trivial to use. I've had a hard time remembering myself the special syntax for its "queries", and an even harder time getting colleagues to invest in learning and properly adopting it. I also think for many uses a much simpler approach, without all the graph wizardry with Loom, is just as good, or even better.

That said Specmonstah has one appealing feature, if a user has posts, and you ask it to create three posts, it will only create a single user and link it to all the posts. Currently with Facai if you want things to be shared then you need to be explicit: create the shared thing first, and then pass it in when creating other entities.

This is tedious when it's actually some deeply nested thing. E.g. post->profile->user->organization. You want to create a post, but want to pass in the organization. Now you need to explicitly specify all the intermediate objects.

And a related problem arises when trying to pull entities out of the end result. Facai has created a handful of different things for me, now give me two things of a certain type from that result so I can continue with them.

Paths and Selectors

While building a factory Facai keeps track of the "path" inside the structure. Whenever we descend into a map we add the key onto the path, whenver we recurse into another factory we add the factory name/type to the path.

So say we're building something like

{:post-id {:profile-id {:user-id {:organization-id {:org-name "Acme"}}}}}

The path at the lowest level of nesting is

[:post-id my-factories/post :profile-id my-factories/profile :user-id my-factories/user :organization-id my-factories/organization :org-name]

Selectors match or don't match a path. A selector matches a path, if the last element of the selector equals the last element of the path, and any previous elements also occur in the path, in the same order. E.g. we could match this path with

[:org-name]
[my-factories/organization :org-name]
[:profile-id :org-name]
;; etc

The special element :> means "direct descendant", similar to > in CSS, and can also be used as the first element to tie the selector to the "root"

[my-factories/organization :> :org-name] ;; matches
[my-factories/user :> :org-name] ;; does not match
[:> :org-name] ;; matches org-name in the top level object, but not in any nested object

:* is currently supported as a final element in a selector that matches anything, but not sure yet if that's really necessary, I perhaps added it to work around another shortcoming (see below, implementation). It's not useful elsewhere in the query since intermediate descendants automatically match unless direct descendent :> is used.

Use cases

Rules

When building an entity it's easy enough to pass in top-level values

(facai/build my-factories/post {:with {:title "Hello"}})

But what if you want to specify values of nested entities? You can call build separately for them, but with rules you can specify this at a higher level

(facai/build my-factories/post 
  {:with {:title "Hello"}
   :rules {[my-factories/organization :org-name] "Acme"}})

Now all org names are set to Acme, regardless of the nesting.

Unification

Say you have a multi-tenant system, so almost every record has a tenant-id. When generating a bunch of entities you want them to all have the same tenant-id. Unification to the rescue. This basically says: if you generate a second thing of the same type, then instead reuse the one you previously created.

(facai/build my-factories/post 
  {:with {:title "Hello"}
   :unify #{[my-factories/tenant]}}) ;; set of selectors

Selecting

This already kinda works with sel/sel1. When you get a facai build result (so with :facai.result/value and :facai.result/linked, you can grab any of the linked objects using a selector.

(let [result (facai/build my-factories/post)
      post   (facai/sel1 result [:>]) ;; or (facai/value result)
      user   (facai/sel1 result [my-factories/user])]
  ;; do stuff with post and user
  )

Implementation

There's currently an implementation of path-match? that checks if a selector matches a path, and it seems to be largely doing its job

https://github.com/lambdaisland/facai/blob/main/src/lambdaisland/facai/kernel.cljc#L24

The main thing I've run into so far is the distinction between map keys and factories.

Take this example

(f/defactory user
  {:name "Arne"})

(f/defactory post
  {:title "Things To Do"
   :author (user {:with {:name "Tobi"}})})

(f/build post)
;;=>
{:facai.result/value {:title "Things To Do", :author {:name "Tobi"}},
 :facai.result/linked {[lambdaisland.facai-test/post :author] {:name "Tobi"}},
 :facai.factory/id lambdaisland.facai-test/post}

I can (f/sel result [:author]), but not (f/sel result [user]). I think this really should add the factory to the path/stack already when building e.g. user:

{:facai.result/value {:title "Things To Do", :author {:name "Tobi"}},
 :facai.result/linked {[lambdaisland.facai-test/post :author lambdaisland.facai-test/user] {:name "Tobi"}},
 :facai.factory/id lambdaisland.facai-test/post}

Problem is that now I can only match on [user] (i.e. the factory) and not the map key :author. I think there path-match? needs to be smarter. If the last element of the selector is a keyword then it should match on the penultimate rather than the last element in the path.

This would also make it possible for us to already push the top-level factory onto the path, e.g. [facai-test/post], and still have that match the root (:>).

[alysbrooks]

Unification would save a lot of code in our work for Open Tech. I'm doing a lot of overriding to ensure the records are linked together. Rules, too—there's a lot of boilerplate to link records when there's a many-to-many relationship between them.

[alysbrooks]

Facai is essentially in maintenance mode while future development happens in the successor library Harvest. I think we're already reconsidering and rethinking rules and selectors in Harvest, so I don't think reoopening this issue on the Harvest repository is necessary.