Proposal to split d2 into d2-api and d2-schemas

[varl]

(.. or d2-models)

d2 could do with some further separation of concerns than it currently has. There are two significant design details in d2 which stop us from using it in every app we build:

1. It is designed as a singleton
2. The API cannot be used without instantiating the singleton

For simpler applications we don't need the schemas which is the brunt of the load d2 carries, we only want to use the d2.Api.

Creating an instance of d2 adds an overhead of multiple seconds before we can render the application, and in e.g. d2-ui we end up having to pass around the d2 object as a prop which circumvents some problems but introduces other problems.

Another consequence of this is that different applications have rolled their own Fetch API wrappers to do the boilerplate code for dealing with requests. Some even reach into ui/utils/api to try to re-use some of the code from in there instead of rolling a new wrapper.

I am thinking something along the lines of:

d2-api

Usage

Examples of what can be in d2-api:

import api, {
    datastore,
    system,
    filter,
    gis,
} from 'd2-api'

// helpers for specific api features, e.g. filters, datastore, geofeatures, system, etc.
filter([':all', ['displayName']])
datastore.set({ key: val })
datastore.get(key)
system.info()
gis.all()

// http verbs
api.post()
api.get()
api.patch()
api.delete()
api.put()

d2-schemas

Usage

Example:

import schemas from 'd2-schemas'

const orgUnits = await schemas.organisationUnit
    .filter()
    .on('level')
    .equals(2)
    .list()

d2.Analytics?

I'm not familiar with the Analytics portions and I am not sure if they fit in with either d2-schemas or d2-api. Is the d2.Analytics class a schema today?

Summary of proposal

• Create a d2-api repo and package
• Rename d2 repo and package to d2-schemas (or d2-models)
• Remove d2.Api from d2-schemas
• Let d2-schemas depend on d2-api
• Replace pure API usage with d2-api in other apps and libs

Thoughts? Comments?

[edoardo]

I can comment about d2.Analytics. No, it's not a schema. It implements an interface for the analytics APIs; see https://docs.dhis2.org/master/en/developer/html/webapi_analytics.html and https://docs.dhis2.org/master/en/developer/html/webapi_event_analytics.html. The old implementation is in the d2-analysis package, which we decided to phase out in favour of d2.Analytics.

[varl]

@edoardo cool, thanks for the input.

We're feeling more pressure to set up the d2-api and start using it now that non-metadata API endpoints are being removed from core, which is causing some our apps to break.

[ismay]

Thoughts? Comments?

So about d2-api, I'm wondering what the potential use cases would be. Just to clarify what it will be doing exactly. So from the above we have:

• part of it is to avoid boilerplate like setting default headers, url, options, etc.
• some convenient methods, like a .get, .post, etc. method
• some dhis2 specific helpers

The first two points seem like things that people commonly make fetch wrappers for, so I'm assuming that we'd be able to use some other library here, or build something very generic ourselves.

The last point is more difficult for me to imagine. It seems conceptually on a different level than the other two points. Maybe it's worthwhile to keep the boundaries clear between a generic fetch-wrapper, and dhis2 specific configuration and helpers. Otherwise I can imagine things happening like the helpers reaching into generic fetch-wrapper stuff over time, etc. What I'm trying to express here is that basically I'm a bit worried about the purpose and boundaries not being clear enough and subsequent feature creep.

I guess this all also depends on how the helpers would work exactly. For example, does filter return a ready to use querystring, or does it do something else?

[amcgee]

Possibly a naive question, but isn't d2-schemas just a specific type of d2-api request?

Tangential question: thoughts on d2js as the top-level library namespace? It's a tiny bit more descriptive (though not a ton) than the overloaded d2.

Personally I'm in favor of making things consolidated but modular. So even if we have separate groups of functionality for api and schema/model operations they should probably live in the same place (i.e. d2js/api and d2js/models if we must) but still be functionally isolated. So importing one means the other could conceivably be tree-shaken out of the bundle. That implies, of course, that there's no up-front performance hit to initialize and proliferate configuration (the former is bad, the latter I think is good). All the d2 initialization we do today should not be blocking renders, and in most cases it shouldn't be loading schemas at all, but it is good to pass configuration and a javascript api through the application for consistent subsequent API calls (through the new React context api in the React case, which probably calls for a d2js-react wrapper)

All of this is just food for thought, generally very on-board with splitting up API access and schema caching.

[varl]

@amcgee at the root of it, sure, all it does is HTTP requests. In its use it is higher-level than that, and d2-schemas also contain complex object classes (complete with class inheritance) which attempts to wrap a lot of logic inside each object. This creates some clashes between our preferred FP-style and the OO-style in d2-schemas.

Splitting it up internally is fine by me, and I'm OK with d2js as well.