ElasticSearch plugin stores duplicate operation history data (1.11.x) in ElasticSearch database

[abitmore]

Description

Logically, an operation history object (1.11.x) can be linked to multiple account history objects (2.9.x). When stored in the in-memory object database through the account history plugin, they are of 2 types and stored separately. But in ES, they are stored together, and the data is indexed by the account history ID. This means that every document (entry) in the ES database contains an operation history object and an account history object. Therefore, the complete data of an operation history object may exist in multiple documents, that is, duplicates.

On the one hand, this mode uses a lot of disk spaces.

On the other hand, this mode makes it easy to search for data because it avoids join-queries.

It is a trade-off.

I think this is worth discussing.

Impacts Describe which portion(s) of BitShares Core may be impacted by this bug. Please tick at least one box.

• [ ] API (the application programming interface)
• [ ] Build (the build process or something prior to compiled code)
• [ ] CLI (the command line wallet)
• [ ] Deployment (the deployment process after building such as Docker, Travis, etc.)
• [ ] DEX (the Decentralized EXchange, market engine, etc.)
• [ ] P2P (the peer-to-peer network for transaction/block propagation)
• [x] Performance (system or user efficiency, etc.)
• [ ] Protocol (the blockchain logic, consensus, validation, etc.)
• [ ] Security (the security of system or user data, etc.)
• [x] UX (the User Experience)
• [ ] Other (please add below)

CORE TEAM TASK LIST

• [ ] Evaluate / Prioritize Bug Report
• [ ] Refine User Stories / Requirements
• [ ] Define Test Cases
• [ ] Design / Develop Solution
• [ ] Perform QA/Testing
• [ ] Update Documentation

[abitmore]

ES is not a rational database. It is by design.