Feature: Index Ontologies metadata and content & Agents

[syphax-bouazzouni]

Prerequisites

• 126

  Changes

• Index submission and ontologies metadata on save (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/ff61cce63b69ed94138c45bc9135f29a551b7cf0)
• Index agents metadata (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/8b607acdf383a26c38b9898aedc4ef8c7265c74e)
• Add ontology and agent metadata indexation tests (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/aa39d724c26c7fad30866c5a343c040d029f094a)
• Make agent, name, acronym, email, and identifiers searchable (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/aaadaaf16c6559423f3816305ea28a5a67b9ba9c)
• Unindex ontology submission when archived (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/fae7bd466a86d7cc1e6a7ec5182152a9062df6cf)
• Update embedded ontology to all the fields and update submission in save (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/c0aef51a1c81108e2fa69c5bfc3dcee8f7d12dff)
• Rename ontology unindex to unindex_all_data to prevent conflicts (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/4773e7f90a1afba33f3d84d4b05105826465d8a9)
• Implement index all ontology content (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/6afb1d1ea62237d503f740ebc08ee5f6d6dfedbd)
• Add parallel processing the index_all_data step (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/ae7046d662b4050b8973466a0f326f399862e790)
• Clear indexed data after ontology delete (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/527515dbbabc6c9c7af57429d7e76336e76660a7)
• Optimize indexing all data in Virtuoso and GraphDb by pre-fetching and sorting all ids by slice in parallel (https://github.com/ontoportal-lirmm/ontologies_linked_data/pull/130/commits/f9b6a0fd1f8239cff116aa44722aa707942004c3)
  • Indexing all the content of INRAETHES (18k IDs / 150k triples), before optimization:
    • fs ⇒ 15.224490000051446s
    • ag ⇒ 19.238805999979377s
    • vo ⇒ 42.95274499990046s
    • gb ⇒ 33.52821200003382s
  • Indexing all the content of INRAETHES (18k IDs / 150K triples), after optimization:
    • fs ⇒ 15.369778999942355s
    • ag ⇒ 17.367580000078306s
    • vo ⇒ 16.564614000031725s
    • gb ⇒ 15.431716999970376s

[codecov[bot]]

Codecov Report

Attention: Patch coverage is 91.13300% with 18 lines in your changes are missing coverage. Please review.

Project coverage is 80.59%. Comparing base (cab072e) to head (46ecccb).

Files	Patch %	Lines
...tologies_linked_data/models/ontology_submission.rb	76.62%	18 Missing :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## development #130 +/- ## =============================================== + Coverage 80.33% 80.59% +0.25% =============================================== Files 84 84 Lines 6190 6296 +106 =============================================== + Hits 4973 5074 +101 - Misses 1217 1222 +5 ``` | [Flag](https://app.codecov.io/gh/ontoportal-lirmm/ontologies_linked_data/pull/130/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ontoportal-lirmm) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/ontoportal-lirmm/ontologies_linked_data/pull/130/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ontoportal-lirmm) | `80.59% <91.13%> (+0.25%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=ontoportal-lirmm#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.