search

etianen / django-watson

Full-text multi-table search application for Django. Easy to install and use, with good performance.
BSD 3-Clause "New" or "Revised" License
1.21k stars 129 forks source link

5 GB database buildwatson command took hour and end with fatal error #247

Closed Pulkit-Sharma closed 4 years ago

Pulkit-Sharma commented 6 years ago

I am trying to index a table with 3 million rows. When I run manage.py buildwatson, it takes around one hour and then without any data creation it ends with error "DB under recovery". Its a postgres DB with 64GB RAM and 16CPU. CPU usage stays around 15% through out.

What could be the issues? Buildwatson is made for small datasets?

etianen commented 6 years ago

Try running buildwatson with the --non-atomic flag.

It'll still take a long time to run, but your transaction won't overflow and explode.

On 3 June 2018 at 07:04, Pulkit notifications@github.com wrote:

I am trying to index a table with 3 million rows. When I run manage.py buildwatson, it takes around one hour and then without any data creation it ends with error "DB under recovery". Its a postgres DB with 64GB RAM and 16CPU. CPU usage stays around 15% through out.

What could be the issues? Buildwatson is made for small datasets?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/etianen/django-watson/issues/247, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJFCL6HH-Ol08EKxcxxLt_xf3em2l76ks5t43yAgaJpZM4UX9bP .