goessebr commented 2 years ago

Admin section

I start the application and go to http://0.0.0.0:6543/admin

Following error appears:

Log:

2022-06-30 16:21:31,524 ERROR [atramhasis.views.exception_views][waitress-1] Could not guess RDF format for <rdflib.parser.StringInputSource object at 0x7f3dce69ba60> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.
Traceback (most recent call last):
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/graph.py", line 1258, in parse
    parser.parse(source, self, **args)  # type: ignore[call-arg]
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1913, in parse
    p.loadStream(stream)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 434, in loadStream
    return self.loadBuf(stream.read())  # Not ideal
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 440, in loadBuf
    self.feed(buf)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 466, in feed
    i = self.directiveOrStatement(s, j)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 488, in directiveOrStatement
    return self.checkDot(argstr, j)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1149, in checkDot
    self.BadSyntax(argstr, j, "expected '.' or '}' or ']' at end of statement")
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1646, in BadSyntax
    raise BadSyntax(self._thisDoc, self.lines, argstr, i, msg)
  File "<string>", line None
rdflib.plugins.parsers.notation3.BadSyntax: <no detail available>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/tweens.py", line 41, in excview_tween
    response = handler(request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/router.py", line 143, in handle_request
    response = _call_view(
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/view.py", line 674, in _call_view
    response = view_callable(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/config/views.py", line 170, in attr_view
    return view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/config/views.py", line 196, in predicate_wrapper
    return view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/viewderivers.py", line 427, in rendered_view
    result = view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/viewderivers.py", line 129, in _class_view
    response = getattr(inst, attr)()
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/atramhasis/views/crud.py", line 128, in get_conceptschemes
    return ProviderView(self.request).get_conceptschemes()
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid_skosprovider/views.py", line 135, in get_conceptschemes
    return [
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid_skosprovider/views.py", line 140, in <listcomp>
    'uri': p.concept_scheme.uri,
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/providers.py", line 71, in concept_scheme
    self._conceptscheme = self._get_concept_scheme()
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/providers.py", line 75, in _get_concept_scheme
    return conceptscheme_from_uri(
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/utils.py", line 43, in conceptscheme_from_uri
    graph = uri_to_graph('%s.rdf' % (conceptscheme_uri), session=s)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/utils.py", line 303, in uri_to_graph
    graph.parse(data=res.content)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/graph.py", line 1261, in parse
    raise ParserError(
rdflib.exceptions.ParserError: Could not guess RDF format for <rdflib.parser.StringInputSource object at 0x7f3dce69ba60> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.

Commenting the getty providers fixes the admin interface.

#    registry.register_provider(aat)
#    registry.register_provider(tgn)

We need to remove them or fix loading them. Fixing them might require a change in https://github.com/OnroerendErfgoed/skosprovider_getty

Downgrading rdflib to 5.0.0 fixes the issue also (current version is 6.1.1), we need to take a look at https://rdflib.readthedocs.io/en/stable/upgrade5to6.html

Public section

Directly adressing a getty thesauri raises an Exception as well. For exemple http://0.0.0.0:6543/conceptschemes/AAT

{
    "message": "unexpected server error"
}

2022-07-01 13:06:22,281 ERROR [atramhasis.views.exception_views][waitress-3] Could not guess RDF format for <rdflib.parser.StringInputSource object at 0x7fe56e295610> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.
Traceback (most recent call last):
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/graph.py", line 1258, in parse
    parser.parse(source, self, **args)  # type: ignore[call-arg]
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1913, in parse
    p.loadStream(stream)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 434, in loadStream
    return self.loadBuf(stream.read())  # Not ideal
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 440, in loadBuf
    self.feed(buf)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 466, in feed
    i = self.directiveOrStatement(s, j)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 488, in directiveOrStatement
    return self.checkDot(argstr, j)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1149, in checkDot
    self.BadSyntax(argstr, j, "expected '.' or '}' or ']' at end of statement")
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/plugins/parsers/notation3.py", line 1646, in BadSyntax
    raise BadSyntax(self._thisDoc, self.lines, argstr, i, msg)
  File "<string>", line None
rdflib.plugins.parsers.notation3.BadSyntax: <no detail available>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/tweens.py", line 41, in excview_tween
    response = handler(request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/router.py", line 143, in handle_request
    response = _call_view(
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/view.py", line 674, in _call_view
    response = view_callable(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/config/views.py", line 170, in attr_view
    return view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/config/views.py", line 196, in predicate_wrapper
    return view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/viewderivers.py", line 427, in rendered_view
    result = view(context, request)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/pyramid/viewderivers.py", line 113, in _class_requestonly_view
    response = getattr(inst, attr)()
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/atramhasis/audit.py", line 59, in advice
    return fn(parent_object, *args, **kw)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/atramhasis/views/views.py", line 118, in conceptscheme_view
    conceptscheme = provider.concept_scheme
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/providers.py", line 71, in concept_scheme
    self._conceptscheme = self._get_concept_scheme()
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/providers.py", line 75, in _get_concept_scheme
    return conceptscheme_from_uri(
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/utils.py", line 43, in conceptscheme_from_uri
    graph = uri_to_graph('%s.rdf' % (conceptscheme_uri), session=s)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/skosprovider_getty/utils.py", line 303, in uri_to_graph
    graph.parse(data=res.content)
  File "/home/goessebr/.virtualenvs/atramhasis_demo/lib/python3.8/site-packages/rdflib/graph.py", line 1261, in parse
    raise ParserError(
rdflib.exceptions.ParserError: Could not guess RDF format for <rdflib.parser.StringInputSource object at 0x7fe56e295610> from file extension so tried Turtle but failed.You can explicitly specify format using the format argument.

goessebr commented 2 years ago

Question: Do we want to keep these external providers in our demo?

koenedaele commented 2 years ago

Just for the record: yes, we do want these external providers in our demo as it demonstrates some of the very powerful features we have with loading and merging external concepts.

koenedaele commented 2 years ago

This works in skosprovider < 6 because the default RDF format in RDFlib is RDF/xml there. As of RDFlib 6, the default format is turtle, so https://github.com/OnroerendErfgoed/skosprovider_getty/blob/develop/skosprovider_getty/utils.py#L303 tries to parse RDF/xml as turtle, which does not work. Best solution would be to make the format explicit when parsing. It's also rather weird that we set the format to rdf outside of the uri_to_graph function. I would suggest passing in an actual uri (no extension) and letting the function add the extension. Right now we have a function that looks like it can be used with all kinds of formats, while actually it only functions with a .rdf file.

koenedaele commented 2 years ago

Basic solution, get skosprovider_getty working with RDFlib >= 6 first.

goessebr commented 1 month ago

The cookiecutter code was integrated in atramhasis since release 2.1.0: https://github.com/OnroerendErfgoed/atramhasis/releases/tag/2.1.0

Documentation can be found at:

Create your own custom project: https://atramhasis.readthedocs.io/en/latest/customisation.html#creating-your-own-project
Create a demo project: https://atramhasis.readthedocs.io/en/latest/demo.html#running-a-demo-site-with-cookiecutter

OnroerendErfgoed / atramhasis_demo_cookiecutter

Errors when trying to load external getty thesauri #10

Admin section

Public section