search

De117 / scanner

MIT License
0 stars 0 forks source link

Scanner architecture #5

Open sgros opened 8 years ago

sgros commented 8 years ago

Scanner should be refactored so that it consists of:

  1. Input thread that takes IP address/hostname and feeds it into input queue.
  2. Thread pool that consists of N workers (configuration option) that wait on input queue and process each data item from the queue. When the processing is finished, results are pushed into output queue.
  3. Output thread takes data from output queue and sends it to a database.
De117 commented 7 years ago

This was done in commit 59418a96286758cc9cdb05a33213efed132fd486. However, I've since thrown out the input/output threads in commit dbadb3dd3e91e47a7a900f6044ab542da13093fc. The queues are still present, but this greatly simplifies the required synchronization between threads. The memory doesn't seem to be a problem, so this really seems like the best solution at the moment.