search

SecureSECO / searchSECO-miner

v2 miner implementation for SearchSECO
https://docs.secureseco.org/searchseco-integration
MIT License
3 stars 3 forks source link

Blocking database communication #45

Open slingerbv opened 2 months ago

slingerbv commented 2 months ago

The ”Sending: BODY” functionality involves transmitting data to the database as a blocking process, halting the entire application until completion.

Steps to Reproduce

  1. Run the SearchSECO miner.
  2. Observe the behavior during the ”Sending: BODY” process.

Expected Behavior: The application should continue processing while data is being sent to the database. Actual Behavior: The entire application halts until the data transfer is complete.

Proposed Solutions

  1. Open a continuous stream with a stream listener on the server, allowing data to be sent incrementally.
  2. Temporarily store data in a fast, in-memory database such as Redis, and use a separate worker to send the data to the server.