GetDKAN / dkan

DKAN Open Data Portal
https://dkan.readthedocs.io/en/latest/index.html
GNU General Public License v2.0
365 stars 170 forks source link

Restrict DB query abstraction to the Datastore module #3823

Open dafeder opened 2 years ago

dafeder commented 2 years ago

We use AbstractDabaseQuery in several places throughout DKAN, but this is really just an additional database abstraction layer on top of Drupal's. We have no plans to ever implement DKAN as a standalone application outside of Drupal, so this adds an unnecessary amount of complexity to the codebase.

Currently, we have an abstract notion of a data query that is used in a few places around DKAN. We also have a concept of a Datastore Query, which is slightly different. So we first convert incoming datastore query objects to generic DKAN "queries", then run them through our abstracted storage classes to convert them to db queries that Drupal can actually execute.

To make this all more efficient and easy to work with:

  1. Collapse the above-mentioned code flow for the datastore. We should eliminate "QueryFactory" and "Abstract Database Table", and simply pass the incoming Datastore query directly to a modified version of the SelectFactory, cutting out the middlemen completely. (Will make a second pass at this issue and spec this out in more detail - @dafeder.)
  2. Refactor the Jobstore service to use the Drupal DB API directly, eliminating the DKAN storage classes from that part of the code.
  3. Refactor the Harvest service to eliminate the "Storage" layer and make Drupal DB API calls as needed.
  4. Refactor the ResourceMapper service to use Drupal DB API calls as needed

It may make sense to separate the storage logic somewhat in these cases and create an interface around them, but let's minimize the abstractions and add simple code that works.

paul-m commented 6 months ago

In progress. Internal: 16902

dafeder commented 4 months ago

Making a note to link this to recent work refactoring various things as Drupal entities which gets us closer to being able to do this.