search

International-Data-Spaces-Association / metadata-broker-open-core

This is the repository of the open-core reference implementation of the IDS Metadata Broker.
Apache License 2.0
10 stars 17 forks source link

Resource not indexed #108

Open sebplorenz opened 2 years ago

sebplorenz commented 2 years ago
  1. De-register a connector that has resources -> The connector and its resources are removed from the UI
  2. Register the connector again -> The connector is visible in the UI, but not the resources
  3. Register one of the resources -> Resource is not visible! Expected: Resource is visible

Connector logs (latest entry on top):

  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.323 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Finished adding to the Resources Index (1 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.322 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.ElasticsearchIndexingConnectorupdateResource - IGNORED indexResource
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.321 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Adding Resource to the Resources Index. URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.320 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Finished adding to the Connector Index (27 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.293 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.ElasticsearchIndexingupdate - Updating indexing of https://broker.test.mobilitydataspace.io/connectors/346699066 .
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.291 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Adding Connector to the Connector Index. URI: https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:51 | brokert2_broker-core_1  04-05-2022 12:29:51.290 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Retrieved the Reduced Connector (377 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.914 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadegetReducedConnector - Getting reduced Connectorhttps://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.913 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Retrieving Reduced Connector. URI: https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.911 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Added to the TripleStore (470 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.883 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadechangePassivationOfGraph - Changing passivation of graph https://broker.test.mobilitydataspace.io/connectors/346699066. Is now active: true
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.439 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Adding Resource to the TripleStore. URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.426 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadegetActiveGraphs - SPARQL query to fetch active graphs complete
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.420 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadegetActiveGraphs - Running SPARQL query to the Fuseki Server to fetch active graphs
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.418 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Rewrote resource. New URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.412 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Checking if connector is active in triple store. ID: https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:50 | brokert2_broker-core_1  04-05-2022 12:29:50.410 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Serialized Resource (778 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310/-32584152
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.630 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Serialize rewritten Resource
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.628 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Catalog found (23 ms). URI: https://broker.test.mobilitydataspace.io/connectors/346699066/-1814306310
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.617 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadegetActiveGraphs - SPARQL query to fetch active graphs complete
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.606 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.index.common.persistence.RepositoryFacadegetActiveGraphs - Running SPARQL query to the Fuseki Server to fetch active graphs
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.603 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Fetching catalog of connector
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.602 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Connector URI did not start with our component catalog URI: https://broker.test.mobilitydataspace.io/connectors/
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.601 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Rewrote connectorUri to https://broker.test.mobilitydataspace.io/connectors/346699066
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.599 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.broker.core.common.persistence.ResourcePersistenceAndIndexingupdated - Update request received. Connector URI: https://ids.dwd-connector.test.mobilitydataspace.io with resource https://connector:8080/api/offers/1f0d52d3-dd5d-4132-8271-b0ee1ee26b36
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.598 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.component.core.logging.MessageLoggerlogMessage - Payload hash: -1249707079
  | 2022-05-04 14:29:49 | brokert2_broker-core_1 affectedResource: https://connector:8080/api/offers/1f0d52d3-dd5d-4132-8271-b0ee1ee26b36
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.596 [http-nio-8080-exec-2] INFO  de.fraunhofer.iais.eis.ids.component.core.logging.MessageLoggerlogMessage - ResourceUpdateMessage received from https://ids.dwd-connector.test.mobilitydataspace.io (called by de.fraunhofer.iais.eis.ids.component.core.logging.MessageLogger)
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.595 [http-nio-8080-exec-2] WARN  de.fraunhofer.iais.eis.ids.connector.commons.broker.SameOriginResourceMapValidationStrategyvalidate - SameOriginValidation not yet implemented for Resources
  | 2022-05-04 14:29:49 | brokert2_broker-core_1  04-05-2022 12:29:49.593 [http-nio-8080-exec-2] WARN  de.fraunhofer.iais.eis.ids.component.core.MessageDispatcher$2verifySecurityToken - No security token verifier set. Use setSecurityTokenVerifier method.

In the beginning there is "IGNORED indexResource". Maybe it didn't index the resource and therefore it is not visible?!

timwirtz86 commented 2 years ago

@sebplorenz thanks for the post. Can you please look whether you find

timwirtz86 commented 2 years ago

@sebplorenz told us in a private communication that "MobiDs Indexing enabled" is part of the logs, so the correct indexing is set.

We figured out that the functionality wasn't implemented yet!