search

oracle / opengrok

OpenGrok is a fast and usable source code search and cross reference engine, written in Java
http://oracle.github.io/opengrok/
Other
4.33k stars 746 forks source link

Make history cache generation quicker #3169

Open ChristopheBordieu opened 4 years ago

ChristopheBordieu commented 4 years ago

Hi,

Generating history cache for some of our Git repositories takes very long time... Using OpenGrok 1.3.16 / Openjdk 11.0.7 / Tomcat 9.0.36 on our docker image.

What are the various way OpenGrok could offer to speed up this runtime?

My branch containing that work was really trying to massively speed up history, since FreeBSD history takes f-o-r-e-v-e-r to manage by OpenGrok since OpenGrok's data model copies commit contents (dates + commit message) to every affected file separately — and FreeBSD commits and merges very often touch thousands of files. The branch is still a work-in-progress and idle for a year, but maybe I can extract the octopus handling from it, which would solve your issue too.

Originally posted by @idodeclare in https://github.com/oracle/opengrok/issues/3166#issuecomment-643512596

vladak commented 4 years ago

Do you mean initial history cache generation or incremental ?

ChristopheBordieu commented 4 years ago

Initial history cache. When done, when reindexing, it is ok.