search

theangryangel / logstash-output-jdbc

JDBC output for Logstash
MIT License
256 stars 101 forks source link

Data insert is very slow : 11 records per minute #111

Closed rkhapre closed 6 years ago

rkhapre commented 6 years ago

Hi I am inserting records in Oracle DB, but the insert is very slow. The pipeline completes the overall operation with input and filter ( as i see in command), but when it comes to insert records in DB, this is very slow.

I tries to insert 300,000 records it took me somewhere 9 hrs.

Any suggestions what all things i need to take care in jdbc output to improve this

Thanks

theangryangel commented 6 years ago

This is entirely based on your environment.

It could be anything from;

  1. Your input is slow
  2. Your save query is badly written or slow
  3. Your database server is slow
  4. The box running logstash is undersized
  5. Your connectivity to the Oracle server is exceptionally slow

Honestly, I don't really see how I can help here.

I can tell you I don't have any Oracle servers. I don't use Oracle. I don't have anywhere near this slow performance in my environment (Microsoft SQL server).