Per project indexing missing history when checkout is in subdir

[fstefanov]

Git clone in project dir

mkdir src/test
cd src/test
git clone https://github.com/oracle/opengrok.git

Add project

opengrok-projadm --base /home/ubuntu/bin/opengrok --add test
Adding project test
Added project test with repositories: ['/test/opengrok']
Refreshing configuration

Indexing

curl -s -X GET http://localhost:8080/source/api/v1/configuration -o fresh_config.xml
opengrok-indexer -a /home/ubuntu/bin/opengrok/dist/lib/opengrok.jar -- \
    -S test \
    --depth 20 \
    -c /usr/local/bin/ctags \
    -U 'http://localhost:8080/source' \
    -R fresh_config.xml \
    -H test \
    test

INFO: Indexer version 1.7.11 (939758a8e1acfde1d650a15cc94ec5653ebb4ce2) running on Java 11.0.11
Jun 23, 2021 1:33:01 PM org.opengrok.indexer.util.CtagsUtil validate
INFO: Using ctags: Universal Ctags 5.9.0(845a3ab0), Copyright (C) 2015 Universal Ctags Team
Universal Ctags is derived from Exuberant Ctags.
INFO: done invalidating 1 repositories (took 514 ms)
INFO: Done scanning for repositories, found 1 repositories (took 332 ms)
INFO: Creating historycache for 1 repositories
INFO: Done historycache for /home/ubuntu/bin/opengrok/src/test/opengrok (took 10.6 seconds)

opengrok-projadm -b ~/bin/opengrok -r
Refreshing configuration

History is missing for all files

Error: File not found!
The requested resource is not available. 

But is available under /home/ubuntu/bin/opengrok/data/historycache/test/opengrok If i checkout opengrok.git directly under src/ dir then history is present. This is problematic for projects like AOSP tree I have similar issue some of the projects (not all) are missing history.

[fstefanov]

Sometimes is not able to discover gits inside project dir opengrok-projadm --base /home/ubuntu/bin/opengrok --add test Adding project test Refreshing configuration

It is missing Added project test with repositories: ['/test/opengrok'] curl gets empty list

curl http://localhost:8080/source/api/v1/projects/test/repositories
[]

[fstefanov]

In catalina.out i see many NoSuchFileException

Caused by: java.nio.file.NoSuchFileException: /opt/tomcat/webapps/source/WEB-INF/lib/chronicle-core-2.20.80.jar
Caused by: java.nio.file.NoSuchFileException: /opt/tomcat/webapps/source/WEB-INF/lib/annotations-19.0.0.jar
Caused by: java.nio.file.NoSuchFileException: /opt/tomcat/webapps/source/WEB-INF/lib/javax.el-3.0.0.jar

find /opt/tomcat/webapps/source/ -name 'chronicle-core*.jar' -type f
/opt/tomcat/webapps/source/WEB-INF/lib/chronicle-core-2.20.122.jar

find /opt/tomcat/webapps/source/ -name 'javax*.jar' -type f
/opt/tomcat/webapps/source/WEB-INF/lib/javax.activation-api-1.2.0.jar

find /opt/tomcat/webapps/source/ -name 'annotations*.jar' -type f

Just updated to - 1.7.12

[ahornace]

In catalina.out i see many NoSuchFileException

These are not related. It's Tomcat's way of checking dependencies. We are using newer versions of some transitive dependencies and Tomcat complains. https://github.com/oracle/opengrok/issues/3390

[fstefanov]

It is misbehaving in a strange way. I was adding, indexing and removing single project for testing multiple times. And now all my other projects( 4 AOSP branches 100GB+ each ) lost the History button on the UI. It is running on ARM64.

[fstefanov]

Deleted historycache dir and ran indexer over again with version 1.7.12. This time history seems correct.