make sub-repositories collapsible in UI

[malferns]

Using opengrok-1.1-rc18 and have several projects in Git indexed in OpenGrok. I added a new project in SVN which uses a manifest file to checkout several sub-projects with different SVN URLs. This creates a cluttered main page with entries such as: ProjectA ProjectA/x/y ProjectA/x/z .... I'd prefer to have one entry for ProjectA. Modified the OpenGrok script to add this line: OPENGROK_SCAN_DEPTH=1 However, after the re-index, this scan_depth does not seem to kick in and I still see all the entries.

Also, tried to use '--depth' directly on the command line, but had no success.

/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx8192m -Dorg.opensolaris.opengrok.history.Subversion=/usr/bin/svn -Dorg.opensolaris.opengrok.history.git=/usr/bin/git -Djava.util.logging.config.file=/var/opengrok/logging.properties -jar /opt/Opengrok/opengrok-1.1-rc18/bin/../lib/opengrok.jar -r on -H --renamedHistory off --depth 1 -c /usr/bin/ctags-exuberant --leadingWildCards on -U localhost:2424 -W /var/opengrok/etc/configuration.xml -S -P -s /opt/opengrok/src -d /opt/opengrok/data

[vladak]

Could you try to replicate this with the latest OpenGrok version ?

[vladak]

The downsides of using --depth:

• the history for the sub-repositories will not be indexed, therefore not searchable
• it will apply to all projects

[vladak]

Trying with 1.3.0, I cannot reproduce this behaviour:

mkdir /var/opengrk/src.svntest
mkdir /tmp/data
cd /var/opengrok/src.svntest
svn checkout http://svn.apache.org/repos/asf/apr/apr/branches/1.3.x apr-1.3
cd apr-1.3
svn checkout http://svn.apache.org/repos/asf/apr/apr-util/branches/1.3.x apr-util-1.3
java -cp 'distribution/target/dist/*' org.opengrok.indexer.index.Indexer \
    -s /var/opengrok/src.svntest -d /tmp/data -P -S -W /tmp/config.xml 

Once the indexer is done, the configuration contains just a single repository - the apr-1.3. This is because SubversionRepository does not override the supportsSubRepositories() method and hence it will return false, therefore the search in HistoryGuru#addRepositories() does not descend: https://github.com/oracle/opengrok/blob/dc6f2658402bf73a306933d83e37de8f8423c42e/opengrok-indexer/src/main/java/org/opengrok/indexer/history/HistoryGuru.java#L443-L454

[malferns]

In your command line, you did not use the '--depth' option, so why did it skip the apr-util-1.3 repository?

[vladak]

As I said, the SubversionRepository is marked as it does not support sub-repositories so the scan stops. This went into effect with 32c956e34142fbaaa023717eb63d1d856b9c1e86

So, the scan depth does not really matter for Subversion repositories.

[malferns]

On a new VM, I installed opengrok-1.3.0 and ran this command:

/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx8192m -jar /opt/opengrok/bin/opengrok.jar -U http://localhost:8080/source -W /var/opengrok/etc/configuration.xml -S -P -s /opt/opengrok/src -d /opt/opengrok/data

I'm still seeing the same behavior with the Subversion sub-projects showing up in the main page.

ProjectA ProjectA/x/y ProjectA/x/z

[vladak]

Could you post the contents of configuration.xml ?

Dne st 14. 8. 2019 2:54 uživatel malferns notifications@github.com napsal:

On a new VM, I installed opengrok-1.3.0 and ran this command:

/usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Xmx8192m -jar /opt/opengrok/bin/opengrok.jar -U http://localhost:8080/source -W /var/opengrok/etc/configuration.xml -S -P -s /opt/opengrok/src -d /opt/opengrok/data

I'm still seeing the same behavior with the Subversion sub-projects showing up in the main page.

ProjectA ProjectA/x/y ProjectA/x/z

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/oracle/opengrok/issues/2884?email_source=notifications&email_token=AAWMMDDRAUGQSVPDNJJQSXTQENJUPA5CNFSM4IH7HAP2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4HMPPA#issuecomment-521062332, or mute the thread https://github.com/notifications/unsubscribe-auth/AAWMMDGZ3DKMXIESXJRUGK3QENJUPANCNFSM4IH7HAPQ .

[malferns]

I've masked the proprietary entries in the configuration.xml and attached it.

configuration.xml.txt

[vladak]

Okay, so there are these repositories:

      /ProjectXXX/bar/aaa/bbb
      /ProjectXXX/bar/ccc
      /ProjectXXX/bar/ddd
      /ProjectXXX/bar/eee
      /ProjectXXX/bar/fff
      /ProjectXXX/bar/ggg
      /ProjectXXX/bar/hhh
      /ProjectXXX/bar/iii
      /ProjectXXX/bar/jjj
      /ProjectXXX/bar/kkk
      /ProjectXXX/bar/lll
      /ProjectXXX/bar/mmm
      /ProjectXXX/bar/nnn
      /ProjectXXX/bar/ooo
      /ProjectXXX/bar/ppp
      /ProjectXXX/bar/qqq
      /ProjectXXX/bar/shared/rrr
      /ProjectXXX/bar/sss
      /ProjectXXX/bar/ttt
      /ProjectXXX/bar/uuu
      /ProjectXXX/bar/vvv
      /ProjectXXX/bar/www
      /ProjectXXX/bar/xxx
      /ProjectXXX/bar/yyy
      /ProjectXXX/bar/zzz
      /ProjectXXX/bar/aaa1
      /ProjectXXX/bar/bbb1
      /ProjectXXX/bar/ccc1
      /ProjectXXX/bar/ddd1
      /ProjectXXX/bar/eee1
      /ProjectXXX/bar/fff1
      /ProjectXXX/src
      /ProjectXXX/deps/ggg1

There are no nested repositories as far as I can tell so the above described process for skipping sub-repositories does not kick in.

Since the issue reported is primarily about the UI, I think the repositories of a project can be made collapsible and that would solve it so reopening as such.

[vladak]

anyhow, for the --depth option to work in this case, I had to use value of 0. I modified the previous example a bit to match the reported use case:

mkdir /var/opengrk/src.svntest
mkdir /tmp/data
cd /var/opengrok/src.svntest
mkdir projectA
cd projectA
svn checkout http://svn.apache.org/repos/asf/apr/apr/branches/1.3.x apr-1.3
svn checkout http://svn.apache.org/repos/asf/apr/apr-util/branches/1.3.x apr-util-1.3
java -cp 'distribution/target/dist/*' org.opengrok.indexer.index.Indexer \
    -s /var/opengrok/src.svntest -d /tmp/data -P -S -W /tmp/config.xml --depth 0

[malferns]

Is there a fix for the above mentioned problem?

[vladak]

Is there a fix for the above mentioned problem?

I don't think so.