search

open-contracting / cove-ocds

OCDS Data Review Tool
https://ocds-data-review-tool.readthedocs.io
Other
2 stars 3 forks source link

JSON reference error #165

Closed duncandewhurst closed 1 year ago

duncandewhurst commented 3 years ago

I tried declaring the consolidated OCDS for PPPs extension with some data from ANI Colombia and the DRT returned the following error:

We have detected a JSON reference error in the schema. This may be due to some extension trying to resolve non-existing references.

Error message: Unresolvable JSON pointer: '/definitions/OrganizationReference'

I tested the DRT with the worked example data from OCDS for PPPs and it worked fine: https://standard.open-contracting.org/review/data/58d5c222-9c7d-4a88-a53a-44b9427eb1ed

cc @cecicasco

jpmckinney commented 3 years ago

Odd, it works fine with the libcoveocds CLI.

jpmckinney commented 3 years ago

I got a full traceback by disabling the handling of except jsonref.JsonRefError in libcove (below).

It's possible it has to do with caching (the libcoveocds CLI doesn't cache the jsonref requests, as far as I know). Not sure what else changes between cove-ocds and lib-cove-ocds.

Traceback (most recent call last):
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 252, in __subject__
    return self.cache
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
AttributeError: 'JsonRef' object has no attribute 'cache'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 217, in resolve_pointer
    document = document[part]
KeyError: 'OrganizationReference'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/django/core/handlers/exception.py", line 47, in inner
    response = get_response(request)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/django/core/handlers/base.py", line 181, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback
    return callback(request, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/cove/views.py", line 23, in wrapper
    return func(request, *args, **kwargs)
  File "./cove_ocds/views.py", line 244, in explore_ocds
    context = common_checks_ocds(context, upload_dir, json_data, schema_ocds)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcoveocds/common_checks.py", line 124, in common_checks_ocds
    upload_dir, json_data, schema_obj, schema_name, context, fields_regex=True, api=api, cache=cache
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcove/lib/common.py", line 488, in common_checks_context
    schema_fields = schema_obj.get_pkg_schema_fields()
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcove/lib/common.py", line 332, in get_pkg_schema_fields
    return set(schema_dict_fields_generator(self.get_pkg_schema_obj(deref=True)))
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcoveocds/schema.py", line 193, in get_pkg_schema_obj
    deref_schema_obj = self.get_schema_obj(deref=True)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcoveocds/schema.py", line 173, in get_schema_obj
    schema_obj = self.deref_schema(extended_schema_str)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcove/lib/common.py", line 316, in deref_schema
    return _deref_schema(schema_str, self.schema_host, cache=cache)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/libcove/lib/common.py", line 434, in _deref_schema
    repr(deref_obj)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 234, in __repr__
    return repr(self.__subject__)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 234, in __repr__
    return repr(self.__subject__)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 234, in __repr__
    return repr(self.__subject__)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 234, in __repr__
    return repr(self.__subject__)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 176, in __getattribute__
    return _oga(self, attr)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 254, in __subject__
    self.cache = super(LazyProxy, self).__subject__
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 240, in __subject__
    return self.callback()
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 174, in callback
    result = self.resolve_pointer(self.store[uri], fragment)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 219, in resolve_pointer
    self._error("Unresolvable JSON pointer: %r" % pointer, cause=e)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/proxytypes.py", line 134, in wrapper
    return method(self, *args, **kwargs)
  File "/home/cove/cove/.ve/lib/python3.6/site-packages/jsonref.py", line 229, in _error
    cause=cause,
jsonref.JsonRefError: Unresolvable JSON pointer: '/definitions/OrganizationReference'
Bjwebb commented 3 years ago

Looks like this is because we default to the 1.0 schema, which doesn't have this definition.

If I add "version": 1.1, to the data it works in the DRT.

duncandewhurst commented 2 years ago

Thanks @Bjwebb. Good spot. I created a follow up issue about consistently validating data with a missing or incorrect version against OCDS 1.1: https://github.com/open-contracting/lib-cove-ocds/issues/110. If we do that then this issue can be closed.

jpmckinney commented 1 year ago

I'll simply close this in favor of open-contracting/lib-cove-ocds#110.