ETL runtime stats more granular

opensemanticsearch / open-semantic-etl

Python based Open Source ETL tools for file crawling, document processing (text extraction, OCR), content analysis (Entity Extraction & Named Entity Recognition) & data enrichment (annotation) pipelines & ingestor to Solr or Elastic search index & linked data graph database

https://opensemanticsearch.org/etl

GNU General Public License v3.0

254 stars 69 forks source link

ETL runtime stats more granular #114

Closed opensemanticsearch closed 4 years ago

opensemanticsearch commented 4 years ago

Make ETL runtime stats more granular / for each plugin (analog to runtime stats of text extraction by Tika), so we can easier analyze stats for each plugin / tool / enrichment / enhancement stage.

Mandalka commented 4 years ago

Implemented separated runtime stat for each plugin to integer fields etl_pluginname_time_millis_i which are shown in UI in preview tab "Document processing (ETL)"