Failed to set up oxigraph endpoint

[ch-sander]

issue described here: https://github.com/oxigraph/oxigraph/issues/957

I fail to set up oxigraph as an endpoint for ResearchSpace

[aindlq]

@ch-sander you can separately configure query and update endpoint for default repository.

Essentially you need to create runtime-data/config/repositories/default.ttl:

@prefix rep: <http://www.openrdf.org/config/repository#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix mph: <http://www.researchspace.org/resource/system/repository#> .
@prefix sparqlr: <http://www.openrdf.org/config/repository/sparql#> .

[] a rep:Repository ;
  rep:repositoryID "default" ;
  rdfs:label "Default HTTP SPARQL Repository" ;
  rep:repositoryImpl [
    rep:repositoryType "researchspace:SPARQLRepository" ;
    sparqlr:query-endpoint <http://example.org/sparql> ; # Endpoint to send SPARQL queries to.
    sparqlr:update-endpoint <http://example.org/update> ; # Update endpoint (optional - only if different from the query endpoint)
    mph:quadMode true ; # Flag determining whether the requests for statements should retrieve triples <subj, pred, obj> or quads <subj, pred, obj, named_graph>
  ] .

it will overwrite repository specified with -Dconfig.environment.sparqlEndpoint.

Through this configuration you can also use repositories that requires basic auth, etc. See more details here - see more details here - https://documentation.researchspace.org/resource/Help:RepositoryManager

[ch-sander]

@aindlq Thanks! With this, I got somewhat further. I now can use oxigraph as default repository, I can query and update the default repository via the SPARQL endpoint (http://localhost:10214/sparql). However, both the UI file upload (http://localhost:10214/resource/Admin:DataImportExport) and curl at http://localhost:10214/rdf-graph-store prompt:

http://oxigraph:7878/query repository is readonly!

So, this means the definition in default.ttl

sparql:query-endpoint <http://oxigraph:7878/query>;
sparql:update-endpoint <http://oxigraph:7878/update>;

is not accounted for by the API, it seems. Is this a glitch of ResearchSpace or yet another misconfiguration of mine?

I really appreciate your help! And I think this might not really touch on @Tpt's oxigraph and https://github.com/oxigraph/oxigraph/issues/957 but is a ResearchSpace issue.

LOGS:

2024-08-11 09:01:09,829 DEBUG [qtp2035070981-20] org.researchspace.rest.endpoint.RDFGraphStoreEndpoint - No graph URI specified. Creating random URI.
2024-08-11 09:01:10,039 ERROR [qtp2035070981-20] org.researchspace.rest.endpoint.RDFGraphStoreEndpoint - Failed to create GRAPH "file://agg_graph.ttl-11-08-2024-11-01-09" : Can't commit transaction. http://oxigraph:7878/query repository is readonly!
2024-08-11 09:01:10,039 DEBUG [qtp2035070981-20] org.researchspace.rest.endpoint.RDFGraphStoreEndpoint - Details:
org.eclipse.rdf4j.repository.RepositoryException: Can't commit transaction. http://oxigraph:7878/query repository is readonly!

[ch-sander]

Also this setup

-Dconfig.environment.sparqlEndpoint=http://oxigraph:7878/update

doesn't do the trick!

[ch-sander]

Update!

I had the repository set read-only (mph:writable false)

Once I changed it to true, however, the app fails to load, leading to the 503 error for localhost:10214.

It seems to be due to templates or assets:

g17test-oxigraph-1       | Internal server error: The graph GRAPH <http://www.researchspace.org/ontology/PX_profession/context> does not exists
g17test-researchspace-1  | 2024-08-11 09:11:04 WARN  [main] (SPARQLProtocolSession.java:1192) - Server reports problem: null (enable debug logging for full details)
g17test-researchspace-1  | 2024-08-11 09:11:04.428:WARN:oejw.WebAppContext:main: Failed startup of context o.e.j.w.WebAppContext@5db6b9cd{/,file:///tmp/jetty/jetty-0_0_0_0-8080-ROOT_war-_-any-6157512089489128674.dir/webapp/,UNAVAILABLE}{/var/lib/jetty/webapps/ROOT.war}

log.txt

Is this maybe due to https://github.com/researchspace/researchspace-instance-configurations ?

Simply uploading the two trig files (quads) in the repo into my oxigraph didn't help. However, it created the mentioned PX-profession graph, but the next time the error will be

org.eclipse.rdf4j.repository.RepositoryException: The graph GRAPH http://www.researchspace.org/pattern/ancient_periods/P82b_end_of_the_end/context does not exists

PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>
PREFIX rdfs: <http://www.w3.org/2000/01/rdf-schema#>
SELECT DISTINCT ?graph
WHERE {
  GRAPH ?graph { ?s ?p ?o }
  FILTER(STRSTARTS(STR(?graph), "http://www.researchspace.org"))
}

returns 12 graphs:

http://www.researchspace.org/ontology/boundingBox/context
http://www.researchspace.org/ontology/PX_physical_description/context
http://www.researchspace.org/ontology/PX_has_video/context
http://www.researchspace.org/instances/fields/E22ManMadeObjectP108iwasproducedbyE12ProductionP4hastimespanE52TimeSpan/context
http://www.researchspace.org/ontology/PX_is_connected/context
http://www.researchspace.org/conservation/fieldDefinition/Structural%20condition/context
http://www.researchspace.org/ontology/viewport/context
http://www.researchspace.org/ontology/PX_has_main_representation/context
http://www.researchspace.org/resource/system/fieldDefinitionContainer/context
http://www.researchspace.org/ontology/PX_profession/context
http://www.researchspace.org/instances/fields/wasusedfor/context

http://www.researchspace.org/pattern/ancient_periods/P82b_end_of_the_end/context nowhere to be found.

As the app doesn't load, I cannot access the UI /resource/Platform:fieldDefinitionContainer?repository=assets. Anyways, maybe the assets repository should not be touched by the default repository in the first place? It seems that the patterns are loaded in the container somewhere and get shadowed or overruled by me custom default repository if set as writable.

[aindlq]

@ch-sander what researchspace docker image version are you using? Could you share your .env and docker-compose files

[ch-sander]

Of course!

docker-compose.yml

version: "2.2"
services:
  researchspace:
    image: ${RESEARCHSPACE_IMAGE}
    restart: unless-stopped
    mem_limit: ${RESEARCHSPACE_MEMORY}
    mem_reservation: ${RESEARCHSPACE_MEMORY}
    ports:
      - "10214:8080"
    environment:
      # 1. Connect researchspace instance to blazegraph instance, in this case we use internal hostname available only inside docker-compose create network.
      # 2. Set default logging profile to log4j2, other possible options are log4j2-debug.xml, log4j2-trace.xml
      # 3-5. Create storage needed for image upload functionality
      # 6-8. Create temp storage needed for image upload functionality
      # 7. Additional options provided in .env file
      PLATFORM_OPTS: >-
        -Dconfig.environment.sparqlEndpoint=http://oxigraph:7878/query
        -Dlog4j.configurationFile=classpath:org/researchspace/logging/log4j2-debug.xml
        -Dconfig.storage.images.type=nonVersionedFile
        -Dconfig.storage.images.mutable=true
        -Dconfig.storage.images.root=/images
        -Dconfig.storage.tmp.type=nonVersionedFile
        -Dconfig.storage.tmp.mutable=true
        -Dconfig.storage.tmp.root=/tmp-data
        ${RESEARCHSPACE_OPTS}
      JAVA_OPTS: ${RESEARCHSPACE_JAVA_OPTS}
    volumes:
       # mount folder where uploaded images should be stored 
       - ./researchspace/data/images:/images:rw

       # mount folder where temporary files should be stored
       - ./researchspace/data/tmp:/tmp-data:rw

      # mount folder where apps should be stored
       - ./researchspace/apps:/apps:rw

       # runtime data folder with config files and ad-hoc templates 
       - ./researchspace/runtime-data:/runtime-data
    depends_on:
      - oxigraph
      - digilib 
    logging:
      driver: json-file
      options:
        max-size: "200k"
        max-file: "10"

  oxigraph:
    image: oxigraph/oxigraph
    ports:
      - "7878:7878"
    volumes:
     - ./data_oxigraph:/data
    command: ["serve", "--location", "/data", "--bind", "0.0.0.0:7878"]

  # blazegraph:
  #   image: ${BLAZEGRAPH_IMAGE}
  #   restart: unless-stopped
  #   mem_limit: ${BLAZEGRAPH_MEMORY}
  #   mem_reservation: ${BLAZEGRAPH_MEMORY}
  #   ports:
  #     - "10215:8080"
  #   environment:
  #     JAVA_OPTS: ${BLAZEGRAPH_JAVA_OPTS}
  #   volumes:
  #    - ./blazegraph/:/blazegraph-data
  #   logging:
  #    driver: json-file
  #    options:
  #     max-size: "200k"
  #     max-file: "10"

  digilib:
    image: ${DIGILIB_IMAGE}
    restart: unless-stopped
    mem_limit: ${DIGILIB_MEMORY}
    mem_reservation: ${DIGILIB_MEMORY}
    environment:
      BASEDIR_LIST: /var/lib/images
      IIIF_IMAGE_BASE_URL: "http://localhost:10214/proxy/IIIF/"
    volumes:
      - ./researchspace/data/images/file:/var/lib/images:rw
    logging:
      driver: json-file
      options:
        max-size: "200k"
        max-file: "10"

.env

# (required) your project name, to have nice container names in the docker-compose setup.
# see https://docs.docker.com/compose/reference/envvars/#compose_project_name
COMPOSE_PROJECT_NAME=g17test

# (required) memory limits for docker containers

# Total amount of memory available for containers
RESEARCHSPACE_MEMORY=1g
BLAZEGRAPH_MEMORY=1g
DIGILIB_MEMORY=500m

# (optional) additional ReseearchSpace runtime options.
# By default increases jetty upload limit to ~100mb
RESEARCHSPACE_OPTS=-Dorg.eclipse.jetty.server.Request.maxFormContentSize=1000000

# (optional) additional JVM opts for various applications
# WARNING! Don't set Xmx here, use memory variables above. We use -XX:+UseContainerSupport, see https://merikan.com/2019/04/jvm-in-a-container/ for more details about memory management of Java apps inside docker.
RESEARCHSPACE_JAVA_OPTS=
BLAZEGRAPH_JAVA_OPTS=
DIGILIB_JAVA_OPTS=

# docker images
RESEARCHSPACE_IMAGE=researchspace/platform-ci:latest
BLAZEGRAPH_IMAGE=researchspace/blazegraph:latest
DIGILIB_IMAGE=robcast/digilib:latest

default.ttl

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix sparql: <http://www.openrdf.org/config/repository/sparql#> .
@prefix rep: <http://www.openrdf.org/config/repository#> .
@prefix sail: <http://www.openrdf.org/config/sail#> .
@prefix sr: <http://www.openrdf.org/config/repository/sail#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix mph: <http://www.researchspace.org/resource/system/repository#> .
@prefix ephedra: <http://www.researchspace.org/resource/system/ephedra#> .
@prefix fedsail: <http://www.openrdf.org/config/sail/federation#> .

[] a rep:Repository;
  rep:repositoryID "default";
  rdfs:label "Default G17 HTTP SPARQL Repository";
  rep:repositoryImpl [
      rep:repositoryType "researchspace:SPARQLRepository";
      sparql:query-endpoint <http://oxigraph:7878/query>;
      sparql:update-endpoint <http://oxigraph:7878/update>;
      mph:quadMode true;
      mph:writable true
    ] .

assets.ttl

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix proxy: <http://www.openrdf.org/config/repository/proxy#> .
@prefix rep: <http://www.openrdf.org/config/repository#> .
@prefix sail: <http://www.openrdf.org/config/sail#> .
@prefix sr: <http://www.openrdf.org/config/repository/sail#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix mph: <http://www.researchspace.org/resource/system/repository#> .
@prefix ephedra: <http://www.researchspace.org/resource/system/ephedra#> .
@prefix fedsail: <http://www.openrdf.org/config/sail/federation#> .
@prefix sparqlr: <http://www.openrdf.org/config/repository/sparql#> .

[] a rep:Repository;
  rep:repositoryID "assets";
  rdfs:label "assetsrepository for platform.";
  rep:repositoryImpl [
      rep:repositoryType "openrdf:ProxyRepository";
      proxy:proxiedID "default"
    ] .

[ch-sander]

@aindlq any idea what's being the issue here (or how to skip the mandatory check of all assets)?

[aindlq]

@ch-sander please try this image researchspace/platform-ci:oxigraph-fix, oxigraph doesn't like CLEAR GRAPH when graph doesn't exist, I add PR to RS with a config option that should fix that - https://github.com/researchspace/researchspace/pull/349

with this default.ttl:

@prefix xsd: <http://www.w3.org/2001/XMLSchema#> .
@prefix sparql: <http://www.openrdf.org/config/repository/sparql#> .
@prefix rep: <http://www.openrdf.org/config/repository#> .
@prefix sail: <http://www.openrdf.org/config/sail#> .
@prefix sr: <http://www.openrdf.org/config/repository/sail#> .
@prefix rdfs: <http://www.w3.org/2000/01/rdf-schema#> .
@prefix mph: <http://www.researchspace.org/resource/system/repository#> .
@prefix ephedra: <http://www.researchspace.org/resource/system/ephedra#> .
@prefix fedsail: <http://www.openrdf.org/config/sail/federation#> .

[] a rep:Repository;
    rep:repositoryID "default";
    rdfs:label "Default G17 HTTP SPARQL Repository";
    rep:repositoryImpl [
        rep:repositoryType "researchspace:SPARQLRepository";
    sparql:query-endpoint <http://oxigraph:7878/query>;
    sparql:update-endpoint <http://oxigraph:7878/update>;
    mph:quadMode true;
    mph:writable true ;
    mph:silentMode true ;

    ] .

But looks like oxigraph doesn't like default ontologies and some other config data that RS is loading, So you also need to disable default config loading, but keep in mind that these will make most of default RS templates useless. Add runtime-data/config/global.prop:

repositoriesLDPLoad=,
forceLDPLoadFromStorages=,

It should help, at least locally I can then go to http://localhost:10214/resource/Start.

Here is a SPARQL query that is generated by RD4J that oxigraph doesn't like - https://gist.githubusercontent.com/aindlq/22390b0312d049a8c8b896c7df3abf64/raw/559a0c33870243467fc6c1befe335b0867749cef/ontologies

[Tpt]

Oxigraph developper here:

Thank you for investigating!

The bahavior of making CLEAR fail in case the graph does not exist is suggested by the SPARQL UPDATE specification:

If the store records the existence of empty graphs, then the SPARQL 1.1 Update service, by default, SHOULD return failure if the specified graph does not exist. If SILENT is present, the result of the operation will always be success.

About the error with the INSERT DATA request, it's because Oxigraph server limits SPARQL bodies to 1MB. This MR should increase the limit to 128MB and return a better error message: https://github.com/oxigraph/oxigraph/pull/969

[ch-sander]

Thank you so much @aindlq @Tpt !

I can confirm it works (no error, did not test the assets and templates) but only with :latest docker image. However, mph:silentMode true seems to be ignored in the default.ttl (does not show in the UI but set in the file)

With the oxigraph-fix tag, I get:

g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (GuiceServletConfig.java:76) - Main platform servlet context initialized.
g17_rs_oxigraph-researchspace-1  |
g17_rs_oxigraph-researchspace-1  | *************************************************************************************
g17_rs_oxigraph-researchspace-1  | * Main platform servlet context initialized. Press CTRL+C to terminate the process. *
g17_rs_oxigraph-researchspace-1  | *************************************************************************************
g17_rs_oxigraph-researchspace-1  |
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "default"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (RepositoryManager.java:349) - Repository with id "default" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "vocabularies"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (RepositoryManager.java:349) - Repository with id "vocabularies" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:49 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "tna-discovery-record-details"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "tna-discovery-record-details" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "vam"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "vam" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "tna-discovery-children"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "tna-discovery-children" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "osm-nominatim-search"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "osm-nominatim-search" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "configurations"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "configurations" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "vam-object"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "vam-object" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "met-collection-search"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "met-collection-search" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "tna-discovery-search-record"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "tna-discovery-search-record" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "tna-discovery-range-search"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "tna-discovery-range-search" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "assets"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "assets" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "system"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "system" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "tests"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "tests" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "met-object-details"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "met-object-details" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "wikidata-entity"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "wikidata-entity" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "ontologies"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "ontologies" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:323) - Trying to initialize repository with id "ephedra"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:349) - Repository with id "ephedra" successfully initialized
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 DEBUG [main] (RepositoryManager.java:194) - Registering RepositoryManager shutdown hook
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "vocabularies".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "vocabularies" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "configurations".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "configurations" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "default".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "default" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "assets".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "assets" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "system".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "system" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:298) - Testing connection for repository "ontologies".
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:50 INFO  [main] (RepositoryManager.java:303) - Connection to repository "ontologies" has been established successfully.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:132) - Loading LDP assets...
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:165) - Skipping loading LDP assets into the "vocabularies" repository: the repository is not listed in "repositoriesLDPLoad" property in "global.prop"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:165) - Skipping loading LDP assets into the "system" repository: the repository is not listed in "repositoriesLDPLoad" property in "global.prop"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:165) - Skipping loading LDP assets into the "configurations" repository: the repository is not listed in "repositoriesLDPLoad" property in "global.prop"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:165) - Skipping loading LDP assets into the "ontologies" repository: the repository is not listed in "repositoriesLDPLoad" property in "global.prop"
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (LDPAssetsLoader.java:170) - All LDP assets loading finished
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 WARN  [main] (SPARQLProtocolSession.java:1192) - Server reports problem: null (enable debug logging for full details)
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 ERROR [main] (LDPAssetsLoader.java:214) - Failed to run knowledge patterns generator....
g17_rs_oxigraph-researchspace-1  |
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 ERROR [main] (LDPAssetsLoader.java:215) - error at 1:97: expected one of Prefix not found, ['%'], ['-' | '0' ..= '9' | '\u{00B7}' | '\u{0300}' ..= '\u{036F}' | '\u{203F}' ..=
g17_rs_oxigraph-researchspace-1  | '\u{2040}'], ['.'], [':'], ['A' ..= 'Z' | 'a' ..= 'z' | '\u{00C0}' ..= '\u{00D6}' | '\u{00D8}' ..=
g17_rs_oxigraph-researchspace-1  | '\u{00F6}' | '\u{00F8}' ..= '\u{02FF}' | '\u{0370}' ..= '\u{037D}' |
g17_rs_oxigraph-researchspace-1  | '\u{037F}' ..= '\u{1FFF}' | '\u{200C}' ..= '\u{200D}' | '\u{2070}' ..=
g17_rs_oxigraph-researchspace-1  | '\u{218F}' | '\u{2C00}' ..= '\u{2FEF}' | '\u{3001}' ..= '\u{D7FF}' |
g17_rs_oxigraph-researchspace-1  | '\u{F900}' ..= '\u{FDCF}' | '\u{FDF0}' ..= '\u{FFFD}'], ['\\'], ['_']
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (MDCFilter.java:99) - Init MDC Filter.
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (PlatformPluginManager.java:143) - Detected the following REST Extensions: []
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:51 INFO  [main] (RestApplicationExtension.java:49) - Trying to register the following RestExtensions: []
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:52.030:INFO:oejsh.ContextHandler:main: Started o.e.j.w.WebAppContext@7e5afaa6{/,file:///tmp/jetty/jetty-0_0_0_0-8080-ROOT_war-_-any-5478918572924569408/webapp/,AVAILABLE}{/var/lib/jetty/webapps/ROOT.war}
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:52.045:INFO:oejs.AbstractConnector:main: Started ServerConnector@89465d9{HTTP/1.1, (http/1.1)}{0.0.0.0:8080}
g17_rs_oxigraph-researchspace-1  | 2024-08-21 07:51:52.046:INFO:oejs.Server:main: Started @13654ms

The login UI loads but then prompts Platform initialization failed: Error: Server Error Please contact the system administrator.

I understand, once https://github.com/oxigraph/oxigraph/pull/969 is available in the latest oxigraph image, the setting in runtime-data/config/global.prop can be deleted, correct?

[ch-sander]

Update

as of https://github.com/oxigraph/oxigraph/pull/969#event-13965074024 the new oxigraph built can ingest the assets. The error above however remains...

[Tpt]

g17_rs_oxigraph-researchspace-1 | 2024-08-21 07:51:51 ERROR [main] (LDPAssetsLoader.java:215) - error at 1:97: expected one of Prefix not found, ['%'], ['-' | '0' ..= '9' | '\u{00B7}' | '\u{0300}' ..= '\u{036F}' | '\u{203F}' ..= g17_rs_oxigraph-researchspace-1 | '\u{2040}'], ['.'], [':'], ['A' ..= 'Z' | 'a' ..= 'z' | '\u{00C0}' ..= '\u{00D6}' | '\u{00D8}' ..= g17_rs_oxigraph-researchspace-1 | '\u{00F6}' | '\u{00F8}' ..= '\u{02FF}' | '\u{0370}' ..= '\u{037D}' | g17_rs_oxigraph-researchspace-1 | '\u{037F}' ..= '\u{1FFF}' | '\u{200C}' ..= '\u{200D}' | '\u{2070}' ..= g17_rs_oxigraph-researchspace-1 | '\u{218F}' | '\u{2C00}' ..= '\u{2FEF}' | '\u{3001}' ..= '\u{D7FF}' | g17_rsoxigraph-researchspace-1 | '\u{F900}' ..= '\u{FDCF}' | '\u{FDF0}' ..= '\u{FFFD}'], ['\'], ['']

This error looks like a SPARQL parsing error from Oxigraph. It would be great to figure out what the actual payload is.

[aindlq]

@ch-sander To see what queries are send to the sparql endpoint you can enable trace logging for RDF4J.

Modify your docker-compose file to point to custom log4j config file:

      PLATFORM_OPTS: >-
        -Dlog4j.configurationFile=file:/runtime-data/log4j2-trace.xml

And add the actual config file to the researchspace/runtime-data folder:

<Configuration xmlns:xi="http://www.w3.org/2001/XInclude">
<Appenders>

    <Console name="STDOUT" target="SYSTEM_OUT">
      <PatternLayout pattern="%d{yyyy-MM-dd HH:mm:ss} %-5p [%t] (%F:%L) - %m%n"/>
    </Console>
    </Appenders>

  <Loggers>

    <Logger name="org.researchspace" level="trace" additivity="false">
      <AppenderRef ref="STDOUT"/>
    </Logger>

    <Logger name="org.eclipse" level="trace" additivity="false">
      <AppenderRef ref="STDOUT"/>
    </Logger>

    <!-- additional platform specific loggers that can be switched on individually (INFO) -->
    <Logger name="platform.logging" level="OFF" additivity="false">
      <AppenderRef ref="STDOUT"/>
    </Logger>
    <Logger name="platform.logging.ephedra.QueryPlanLogger" level="INFO" additivity="true" />

    <!-- https://issues.apache.org/jira/browse/BEANUTILS-477 -->
    <Logger name="org.apache.commons.beanutils.FluentPropertyBeanIntrospector" level="error" additivity="false">
      <AppenderRef ref="STDOUT"/>
    </Logger>

    <Root level="warn">
      <AppenderRef ref="STDOUT"/>
    </Root>

  </Loggers>

</Configuration>

[ch-sander]

g17_rs_oxigraph-researchspace-1  | 2024-08-22 09:25:25 TRACE [main] (LDPAssetsLoader.java:195) - SELECT ?ontology (count(?prop) as ?propCount) (count(?kp) as ?kpCount) {?ontology a owl:Ontology . BIND((IF((STRENDS(STR(?ontology),"/")),STR(?ontology),CONCAT(STR(?ontology),"/"))) as ?ontologyURI)BIND(IRI(CONCAT(STR(?ontologyURI),"context")) as ?ontologyContext){graph ?ontologyContext {  ?prop a ?owlType .   FILTER (?owlType IN (owl:DatatypeProperty,owl:ObjectProperty))   FILTER(CONTAINS(STR(?prop),REPLACE(STR(?ontologyContext),"/context",""))) }}UNION {   ?kp a <http://www.researchspace.org/resource/system/fields/Field> .  ?kp a ?owlType .  VALUES ?owlType {owl:ObjectProperty owl:DatatypeProperty owl:AnnotationProperty rdf:Property}}}group by ?ontology
g17_rs_oxigraph-researchspace-1  | 2024-08-22 09:25:25 WARN  [main] (SPARQLProtocolSession.java:1192) - Server reports problem: null (enable debug logging for full details)
g17_rs_oxigraph-researchspace-1  | 2024-08-22 09:25:25 DEBUG [main] (SPARQLProtocolSession.java:1193) - full error message: error at 1:97: expected one of Prefix not found, ['%'], ['-' | '0' ..= '9' | '\u{00B7}' | '\u{0300}'..='\u{036F}' |
g17_rs_oxigraph-researchspace-1  | '\u{203F}'..='\u{2040}'], ['.'], [':'], ['A' ..= 'Z' | 'a' ..= 'z' | '\u{00C0}'..='\u{00D6}' | '\u{00D8}'..='\u{00F6}'
g17_rs_oxigraph-researchspace-1  | | '\u{00F8}'..='\u{02FF}' | '\u{0370}'..='\u{037D}' | '\u{037F}'..='\u{1FFF}'
g17_rs_oxigraph-researchspace-1  | | '\u{200C}'..='\u{200D}' | '\u{2070}'..='\u{218F}' | '\u{2C00}'..='\u{2FEF}'
g17_rs_oxigraph-researchspace-1  | | '\u{3001}'..='\u{D7FF}' | '\u{F900}'..='\u{FDCF}' | '\u{FDF0}'..='\u{FFFD}'], ['\\'], ['_'] 
g17_rs_oxigraph-researchspace-1  | 2024-08-22 09:25:25 ERROR [main] (LDPAssetsLoader.java:214) - Failed to run knowledge patterns generator....
g17_rs_oxigraph-researchspace-1  |
g17_rs_oxigraph-researchspace-1  | 2024-08-22 09:25:25 ERROR [main] (LDPAssetsLoader.java:215) - error at 1:97: expected one of Prefix not found, ['%'], ['-' | '0' ..= '9' | '\u{00B7}' | '\u{0300}'..='\u{036F}' |
g17_rs_oxigraph-researchspace-1  | '\u{203F}'..='\u{2040}'], ['.'], [':'], ['A' ..= 'Z' | 'a' ..= 'z' | '\u{00C0}'..='\u{00D6}' | '\u{00D8}'..='\u{00F6}'
g17_rs_oxigraph-researchspace-1  | | '\u{00F8}'..='\u{02FF}' | '\u{0370}'..='\u{037D}' | '\u{037F}'..='\u{1FFF}'
g17_rs_oxigraph-researchspace-1  | | '\u{200C}'..='\u{200D}' | '\u{2070}'..='\u{218F}' | '\u{2C00}'..='\u{2FEF}'
g17_rs_oxigraph-researchspace-1  | | '\u{3001}'..='\u{D7FF}' | '\u{F900}'..='\u{FDCF}' | '\u{FDF0}'..='\u{FFFD}'], ['\\'], ['_'] 

So, the query is SELECT ?ontology (count(?prop) as ?propCount) (count(?kp) as ?kpCount) {?ontology a owl:Ontology . BIND((IF((STRENDS(STR(?ontology),"/")),STR(?ontology),CONCAT(STR(?ontology),"/"))) as ?ontologyURI)BIND(IRI(CONCAT(STR(?ontologyURI),"context")) as ?ontologyContext){graph ?ontologyContext { ?prop a ?owlType . FILTER (?owlType IN (owl:DatatypeProperty,owl:ObjectProperty)) FILTER(CONTAINS(STR(?prop),REPLACE(STR(?ontologyContext),"/context",""))) }}UNION { ?kp a <http://www.researchspace.org/resource/system/fields/Field> . ?kp a ?owlType . VALUES ?owlType {owl:ObjectProperty owl:DatatypeProperty owl:AnnotationProperty rdf:Property}}}group by ?ontology

I get the same parsing error also in the YASGUI.

[aindlq]

@ch-sander @Tpt that is RS bug, there is some builtin in query that is send without proper prefixes for owl and rdf namespaces. I guess many databases just have these ones predefined in the engine.

[ch-sander]

Yes, this works:

PREFIX owl: <http://www.w3.org/2002/07/owl#>
PREFIX rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>

SELECT ?ontology (COUNT(?prop) AS ?propCount) (COUNT(?kp) AS ?kpCount)
WHERE {
  ?ontology a owl:Ontology .

  BIND(
    IF(STRSTARTS(STR(?ontology), "/"), 
       STR(?ontology), 
       CONCAT(STR(?ontology), "/")) 
    AS ?ontologyURI
  )

  BIND(IRI(CONCAT(STR(?ontologyURI), "context")) AS ?ontologyContext)

  {
    GRAPH ?ontologyContext {
      ?prop a ?owlType .
      FILTER (?owlType IN (owl:DatatypeProperty, owl:ObjectProperty))
      FILTER(CONTAINS(STR(?prop), REPLACE(STR(?ontologyContext), "/context", "")))
    }
  }
  UNION 
  {
    ?kp a <http://www.researchspace.org/resource/system/fields/Field> .
    ?kp a ?owlType .
    VALUES ?owlType {owl:ObjectProperty owl:DatatypeProperty owl:AnnotationProperty rdf:Property}
  }
}
GROUP BY ?ontology

How to fix?

---

I don't mind this query to fail but if this is the cause for the UI prompt (Platform initialization failed: Error: Server Error Please contact the system administrator. ) then it would be great to skip or fix this query.

[aindlq]

pull the latest researchspace/platform-ci:oxigraph-fix and try again

[ch-sander]

it seems this did the trick, thnx @aindlq !

[aindlq]

@ch-sander looks like there are still some issues.

@Tpt looks like that if data is in the graph <X>, it is not visible if I just query SELECT * { ?s ?p ?o } only through SELECT * { GRAPH <X> {?s ?p ?o} }

[ch-sander]

As the UI for this RS oxigraph-fix built looks kind of different from the latest built, are there any important differences?

[ch-sander]

@Tpt looks like that if data is in the graph <X>, it is not visible if I just query SELECT * { ?s ?p ?o } only through SELECT * { GRAPH <X> {?s ?p ?o} }

maybe this requires use_default_graph_as_union=true?

[aindlq]

@Tpt I see that there is union-default-graph, is there a way to set this a default for all requests. I tried to use http://oxigraph:7878/query?union-default-graph but it obviously fail because when RF4J generates queries for commands it sometimes sets default-graph-uri, so I get an error default-graph-uri or named-graph-uri and union-default-graph should not be set at the same time

[aindlq]

As the UI for this RS oxigraph-fix built looks kind of different from the latest built, are there any important differences?

oxigraph-fix is based on the latest dev branch. It is better to ask this question @doldman or @dianat, I know that there were many changes to default template/configuration and overall many UI changes.

[ch-sander]

@Tpt I see that there is union-default-graph, is there a way to set this a default for all requests. I tried to use http://oxigraph:7878/query?union-default-graph but it obviously fail because when RF4J generates queries for commands it sometimes sets default-graph-uri, so I get an error default-graph-uri or named-graph-uri and union-default-graph should not be set at the same time

maybe @Tpt would allow to hardcode the param here: https://github.com/oxigraph/oxigraph/blob/591a13115c359095ae2b91244df905a6f530b6e7/cli/src/main.rs#L1060

--> https://github.com/oxigraph/oxigraph/issues/973

[dianat]

Regarding the latest dev branch (rs-new-design), it is a considerable bit of development focused on creating a ResearchSpace that is useable from the moment one downloads it, without any particular customisations. It helps domain experts to use it without having to dive into programming and it helps developers to start from a UI that already has many of ResearchSpace's functionality already in place. It is still a starting point for developers to build on.

We have not tested with setups like oxigraph, so your work is very interesting and useful, thank you. As @aindlq mentioned if not using RS KPs and ontologies, many of the templates won't work. As this is the case with a data-driven UI. It would be great if you shared the context you are choosing to work with RS, beyond the technical use of oxygraph and RS together.

[ch-sander]

Regarding the latest dev branch (rs-new-design), it is a considerable bit of development focused on creating a ResearchSpace that is useable from the moment one downloads it, without any particular customisations. It helps domain experts to use it without having to dive into programming and it helps developers to start from a UI that already has many of ResearchSpace's functionality already in place. It is still a starting point for developers to build on.

We have not tested with setups like oxigraph, so your work is very interesting and useful, thank you. As @aindlq mentioned if not using RS KPs and ontologies, many of the templates won't work. As this is the case with a data-driven UI. It would be great if you shared the context you are choosing to work with RS, beyond the technical use of oxygraph and RS together.

I'd love to, but maybe in a different place (as for a more lengthy reply)? I tried https://researchspace.org/contact-us/ but that failed.

[ch-sander]

with command: ["serve", "--location", "/data", "--bind", "0.0.0.0:7878", "--union-default-graph"] oxigraph now should solve the problem -- big thanks to @Tpt ! Could you @aindlq test the cases you mentioned or let me know what to do to actually cause the error?

[aindlq]

@ch-sander I still get:

default-graph-uri or named-graph-uri and union-default-graph should not be set at the same time

[ch-sander]

Assuming you pulled the latest build the named-graph-uri flag then does not overrule the global union-default-graph flag? Is that right @Tpt ?

The SPARQL 1.1 specification does not define strict rules on the interaction between named-graph-uri and union-default-graph flags? If not then these are implementation-specific features rather than part of the standard SPARQL language itself. Logically, if a specific named graph is explicitly provided via the named-graph-uri flag, it seems reasonable to expect that it should take precedence over a more general setting like union-default-graph.

[Tpt]

@ch-sander Thank you for raising that! Indeed, I forgot this case. https://github.com/oxigraph/oxigraph/pull/989 should fix the error following the approach you suggested.

[ch-sander]

I'm glad not having missed the point. Thanks!

[ch-sander]

Fixed through https://github.com/oxigraph/oxigraph/pull/989 by @Tpt . @aindlq , could you test again ;) ?

[aindlq]

@ch-sander I haven't properly tested it, but from the quick look it seems to be working now with ghcr.io/oxigraph/oxigraph:main.

thanks @Tpt!

I'm closing the issue, if you run into other issues, please create a ticket in the main RS github repo, for better visibility. Don't many people check this repo.

[ch-sander]

Thanks @aindlq ! You might consider moving this issue to the main repo. I opened it here as I thought it was due to the docker setup (which it wasn't).

Maybe a docker-compose installation with oxigraph here in this repo could be one outcome of this? https://github.com/DHI-Roma/researchspace-docker-compose-oxigraph/tree/master/custom as a (dev) start.

[aindlq]

@ch-sander that is a good idea, PR is welcome.