Installing package error: matching query does not exist.

[EnisBerk]

Describe the bug

I am trying to install pkg from her package to default arches installation. I only need graphs and collections. There is an import error for many of the items.

To Reproduce Steps to reproduce the behavior:

Using Python 3.7.10, and postgres (PostgreSQL) 12.7 (Ubuntu 12.7-1.pgdg20.04+1)

1) installed arches stable/5.2.x 2) install pkg with following command python manage.py packages -o load_package -s ~/arches_her/pkg -db

Logs

loading resource models and branches
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Site Visits.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Contacts.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Area.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Localities-Administrative Areas.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Location Data.json
2021-06-05 06:08:33,919 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Place of Departure.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Maritime Craft Production Phase.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Geometry.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Parent Asset.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Map Settings.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Curated Holding.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/BASIC_SEARCH_SETTINGS.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Communications.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Subjects and Periods.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Extent.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Images.json
2021-06-05 06:08:47,199 arches.app.utils.data_management.resource_graphs.importer ERROR    Widget matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 122, in import_graph
    default_config = Widget.objects.get(widgetid=card_x_node_x_widget["widget_id"]).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.Widget.DoesNotExist: Widget matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Application Area References.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Designation and Protection Assignment.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Additional Notes.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Settings System Defaults.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Activities.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Crash_Event.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Settings Saved Searches.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Descriptions.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Birth.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Type.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Components.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Archives.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Voyages.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Condition Survey.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/System Reference Numbers.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Outcomes.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Wreck_Event.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Timespan (edtf).json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Digital File(s).json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Radiocarbon Calendar Age Assignment.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Asset Dimensions.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Page References.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Resource Model Type.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Repository Storage location.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Copyright.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Audit Metadata.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Heritage Assets, Areas and Artefacts.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Proposal.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Timespan (date).json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Actors.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Bibliographic Source Citation.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Consultations.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Construction Phases.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Settings Map.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Pages.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Archive Material.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Scientific Date Assignment.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Flights.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Use Phase.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Latitude-Longitude References.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Addresses.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Phase Classification.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Names.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Aerial Photograph Source.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Death.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Consultation Dates.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Associated Archive Objects.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Settings Basic Search.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Location Descriptions.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Bibliographic Source Creation.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Archive Source Creation.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/External Cross References.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Mitigations.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Crew.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/National Grid References.json
2021-06-05 06:09:53,964 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Settings Time Search.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Nationalities.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Land Use Classification Assignment.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Named Location.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Publication.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Contact Details.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/References.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/branches/Spatial Metadata.json
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
Widget matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Place.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Application Area.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Person.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Heritage Area.json
2021-06-05 06:10:54,071 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Period.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Archive Source.json
2021-06-05 06:11:22,132 arches.app.utils.data_management.resource_graphs.importer ERROR    Widget matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 122, in import_graph
    default_config = Widget.objects.get(widgetid=card_x_node_x_widget["widget_id"]).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.Widget.DoesNotExist: Widget matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Historic Aircraft.json
2021-06-05 06:11:24,028 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Heritage Asset.json
2021-06-05 06:11:25,919 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Heritage Story.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Organization.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Bibliographic Source.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Artefact.json
2021-06-05 06:12:15,687 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Consultation.json
2021-06-05 06:12:16,710 arches.app.utils.data_management.resource_graphs.importer ERROR    Function matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 86, in import_graph
    graph = Graph(resource)
  File "/root/archesrepo/arches/app/models/graph.py", line 106, in __init__
    default_config = models.Function.objects.get(functionid=function["function_id"]).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.Function.DoesNotExist: Function matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Historic Landscape Characterization.json
2021-06-05 06:12:17,809 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Radiocarbon Date.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Digital Object.json
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Maritime Vessel.json
2021-06-05 06:12:32,919 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
/root/doheritage/digitalheritage37/_pkg_210605_060341/pkg/graphs/resource_models/Activity.json
2021-06-05 06:12:34,367 arches.app.utils.data_management.resource_graphs.importer ERROR    DDataType matching query does not exist.
Traceback (most recent call last):
  File "/root/archesrepo/arches/app/utils/data_management/resource_graphs/importer.py", line 94, in import_graph
    default_config = DDataType.objects.get(datatype=node.datatype).defaultconfig
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/manager.py", line 82, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/root/venvs/archesenv37/lib/python3.7/site-packages/django/db/models/query.py", line 408, in get
    self.model._meta.object_name
arches.app.models.models.DDataType.DoesNotExist: DDataType matching query does not exist.
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
Widget matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
Function matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************
********************************************************************************
DDataType matching query does not exist.
********************************************************************************

Expected behavior Installing graphs and collections from HER package succesfuly.

[mradamcox]

@EnisBerk can you confirm that you have run python manage.py migrate on this database before trying to load the package? If you have, then from the errors here I assume what's going on is that the graphs (either resource models or branches) rely on custom functions, widgets, and datatypes that should be loaded by the HER package but are not. Therefore, I think this ticket would be more fitting on the arches_her repo, instead of here on the core Arches repo.

[chiatt]

I believe @mradamcox is correct that this is an arches_her issue. I'll transfer this ticket.

[chiatt]

I haven't been able to reproduce this issue. @EnisBerk, please let me know if the issue reoccurs for you.