Performance Improvements

[heywhy]

Overview

Currently, building an inverted index happens immediately after the documents have been indexed. But this comes with a problem, as the extracted terms from documents increase the computation time increases so the best way to handle this is to defer the computation to a scheduler.

In this PR are contained an immediate and async scheduler, as the names imply, the immediate scheduler handles its tasks immediately while the async scheduler defers the task.

TODO

• [x] Add error handling
• [x] Provide the ability to configure what action should take place when a document conflict occurs
• [x] Update affected tests
• [x] Expose a function to calculate an index documents size
• [x] GitHub Actions are all passing

[coveralls]

Pull Request Test Coverage Report for Build c38b588545e85c0501f3259b986eb704206ae9cb-PR-18

• 42 of 65 (64.62%) changed or added relevant lines in 8 files are covered.
• 2 unchanged lines in 1 file lost coverage.
• Overall coverage decreased (-3.3%) to 88.504%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
lib/elasticlunr/storage/disk.ex	1	2	50.0%
lib/elasticlunr/logger.ex	0	2	0.0%
lib/elasticlunr/core/field.ex	18	22	81.82%
lib/elasticlunr/scheduler/async.ex	0	16	0.0%
<!--	Total:	42	65	64.62%	-->

Files with Coverage Reduction	New Missed Lines	%
lib/elasticlunr/protocol_implementations.ex	2	90.48%
<!--	Total:	2		-->

Totals
Change from base Build d21451ca1395a3bd381f1a1d9e1647610c03239d:	-3.3%
Covered Lines:	562
Relevant Lines:	635

---

💛 - Coveralls

[codecov[bot]]

Codecov Report

Merging #18 (1b29fdf) into master (0fcb8de) will decrease coverage by 0.46%. The diff coverage is 90.00%.

@@            Coverage Diff             @@
##           master      #18      +/-   ##
==========================================
- Coverage   91.80%   91.33%   -0.47%     
==========================================
  Files          27       31       +4     
  Lines         622      658      +36     
==========================================
+ Hits          571      601      +30     
- Misses         51       57       +6     

Impacted Files	Coverage Δ
lib/elasticlunr/application.ex	83.33% <ø> (ø)
lib/elasticlunr/logger.ex	50.00% <50.00%> (ø)
lib/elasticlunr/storage/disk.ex	93.54% <50.00%> (ø)
lib/elasticlunr/core/field.ex	95.90% <81.81%> (-3.25%)	:arrow_down:
lib/elasticlunr/scheduler/async.ex	94.73% <94.73%> (ø)
lib/elasticlunr/core/index.ex	77.50% <100.00%> (+1.50%)	:arrow_up:
lib/elasticlunr/deserializer.ex	90.00% <100.00%> (ø)
lib/elasticlunr/protocol_implementations.ex	91.30% <100.00%> (+0.39%)	:arrow_up:
lib/elasticlunr/scheduler.ex	100.00% <100.00%> (ø)
lib/elasticlunr/scheduler/immediate.ex	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 0fcb8de...1b29fdf. Read the comment docs.