[develop branch] Entities missing from searches

[gjost]

On the current ddrpublic develop branch at commit b0b61b9, fulltext searches for minidoka do not return any Entity objects. This is a showstopper - we can't proceed with #332 until this is fixed.

[gjost]

On master, a search "minidoka" returns 193 items, mostly Narrator records, plus the ddr-densho-10 and a few Entities. On the current develop branch the same search returns only the collection and narrators.

The two branches have the same version of densho-elastictools:

    (ddrpublic) ddr@densho101dev:/opt/ddr-public$ pip freeze | grep elastic
    elasticsearch==7.17.9
    elasticsearch-dsl==7.4.1
    elastictools @ git+https://github.com/denshoproject/densho-elastictools.git@be9ef406320c81e6128ea622abb33602d4d27d0d

    (ddrpublic) ddr@densho101dev:/opt/ddr-public$ pip freeze | grep elastic
    elasticsearch==7.17.9
    elasticsearch-dsl==7.4.1
    elastictools @ git+https://github.com/denshoproject/densho-elastictools.git@be9ef406320c81e6128ea622abb33602d4d27d0d

[gjost]

The only code differences are changes to models.py to support sorting organization objects by the newly-added sort field.

diff --git a/ddrpublic/ui/models.py b/ddrpublic/ui/models.py
index 96da5a2..380cab7 100644
--- a/ddrpublic/ui/models.py
+++ b/ddrpublic/ui/models.py
@@ -127,6 +127,7 @@ SEARCH_INCLUDE_FIELDS = [
     'persons',
     'rights',
     'topics',
+    'sort',
     # narrator fields
     'image_url',
     'display_name',
@@ -201,7 +202,7 @@ MODEL_PLURALS = {

 # TODO move to ddr-defs
 REPOSITORY_LIST_FIELDS = ['id', 'model', 'title', 'description', 'url',]
-ORGANIZATION_LIST_FIELDS = ['id', 'model', 'title', 'description', 'url',]
+ORGANIZATION_LIST_FIELDS = ['id', 'sort', 'model', 'title', 'description', 'url',]
 COLLECTION_LIST_FIELDS = ['id', 'model', 'title', 'description', 'signature_id',]
 ENTITY_LIST_FIELDS = ['id', 'model', 'title', 'description', 'signature_id',]
 FILE_LIST_FIELDS = ['id', 'model', 'title', 'description', 'access_rel','sort',]
@@ -899,13 +900,23 @@ class Repository(object):

     @staticmethod
     def children(oid, request, limit=DEFAULT_LIMIT, offset=0):
-        return _object_children(
+        results = _object_children(
             document=_object(request, oid),
             request=request,
             sort_fields=ORGANIZATION_LIST_SORT,
             limit=limit,
             offset=offset
         )
+        # manually sort organizations by sort field
+        # we cannot sort in Elasticsearch until Organization `sort` field
+        # is mapped as keyword field type.
+        objects = []
+        for keyword in sorted([org.sort for org in results.objects]):
+            for org in results.objects:
+                if org.sort == keyword:
+                    objects.append(org)
+        results.objects = objects
+        return results

 class Organization(object):

[gjost]

Fixed in ddrpublic commit 5e1130f, merged to master.