Custom driver

[gap777]

Hi, I'm building a modular monolith Rails application. My controller, views and view-model (datagrid) are in one "package" (subsystem) and the models (ActiveRecord) are in a separate one. Rather than connecting my datagrid table to data via ActiveRecord, I need to make calls on an API class that returns arrays of value objects. The API methods support pagination, sorting, and filtering.

It seems like I can use Datagrid with my system efficiently if I can have my Grid use a custom ORM driver (that I implement).

What do you think? Is this possible?

I tried creating one:

class MyAPIDriver < Datagrid::Drivers::AbstractDriver
  def self.match?(scope)
      ...
  end

   # etc
end

but perhaps AbstractDriver isn't accessible outside of your Gem.

i want to avoid doing in-memory sort/filter/pagination for performance reasons.

Thoughts?

[bogdan]

Hey, I have a lot of thoughts on the subject. First, it is really easy to have some custom approach on using API endpoint as datagrid data source. Here is how it can be done using datagrid built-in tools:

class ApiGrid
  include Datagrid
  scope { [] } # Fake scope
  dynamic do
    scope do
      API.get("/api/v1/users", params: {page: page, search: search, country: country).map do |data|
        ApiUser.new(data) # in case you want to wrap data in an object
      end
    end
  end
  filter(:page, :integer, dummy: true)
  filter(:search, :string, dummy: true)
  filter(:country, :enum, select: :country_select, dummy: true)
  column(:id)
  column(:name)
  column(:email)

  def country_select
    ...
  end
end

More advanced way of doing that requires some special API interaction library that supports scopes chaining just like ActiveRecord. Example:

Employee.where(email: email).joins(:department).where(department: {name: name)).page(3)

All ORMs supported by Datagrid have this feature and this feature is a must if you want to build a real driver.

I am really open to write a Datagrid driver for a HTTP API library that supports such chaining and put it inside the gem itself. It doesn't cost much to support it and this is not the first time someone is asking for this feature.

Do you know any library you think is right for such a functionality? I just checked ActiveResource and it doesn't support scope chaining and delayed load

[gap777]

@bogdan Thanks for getting back to us on this! The dynamic scope approach you've proposed might do the trick. We're going to setup a time to spike this and we'll get back to you.

Our API calls are in-process, not HTTP... and we don't already have a chainable/fluent interface (but building one is interesting!). If we did have such a thing, how could a driver be written to use it? What do you think about making a feature to allow custom drivers?

Thanks again!

[bogdan]

I am pretty open to custom drivers. I just never considered them for real because it is very difficult to maintain an internal ORM. But if you really think in-house ORM is your pass, I am open to collaborate.

I just looked into the source code. The driver choose code is pretty flexible. https://github.com/bogdan/datagrid/blob/ac7cbfbd0f083e1c9f301baa0314d2ddfbe20502/lib/datagrid/drivers/abstract_driver.rb#L16

What it means is that you just need to inherit AbstractDriver class and implement all the abstract methods. I never tried that myself, but I don't see any reason why it wouldn't work by looking at source code.

There are total 18 of them by the number of raise NotImplementedError. Most of them are associated with relations chaining and data schema (like guessing type of certain model property).

I don't quite believe that implementing all 18 methods is a good idea for you because the majority of them are used only for particular features. If you don't use those feature, you are making additional job that you don't need. The problem is that it is difficult to track which methods are used for which features. So, if you don't implement all 18 methods, you can't inform your team which features specifically will not work and it can come up as a surprise anytime.

[bogdan]

No followup, closing.