oracle / opengrok

OpenGrok is a fast and usable source code search and cross reference engine, written in Java
http://oracle.github.io/opengrok/
Other
4.36k stars 747 forks source link

PerforceRepository very noisy looking at non-perforce repositories #2808

Closed cross closed 5 years ago

cross commented 5 years ago

I have a mix of Perforce and git projects/repositories in my source root. Once I have set up all of the correct parameters in the trees containing the perforce repositories, the p4 commands that run at the start of indexing produce no errors on those files. However, it seems to scan the entirety of the git repository project I have, and reports errors for each of them. Example:

09:28:49 INFO: Scanning for repositories...
[...]
09:29:32 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/thing1/user: */... - must create client 'genuser:opengrok' to access local files.

09:29:33 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/thing1/user: * - must create client 'genuser:opengrok' to access local files.

09:29:33 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/thing1/icmp: */... - must create client 'genuser:opengrok' to access local files.

09:29:33 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/thing1/icmp: * - must create client 'genuser:opengrok' to access local files.

09:29:34 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/thing1/test: */... - must create client 'genuser:opengrok' to access local files.

09:29:34 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/thing1/test: * - must create client 'genuser:opengrok' to access local files.
[...]
09:30:21 INFO: Done scanning for repositories, found 3 repositories (took 0:01:32)

These go on for all of the directories in projectA, I believe. Is there any way we can get OpenGrok to know that once it's identified it as a git repository, it doesn't try to run the more aggressive PerforceRepository detector on it?

vladak commented 5 years ago

HistoryGuru#addRepositories() calls RepositoryFactory.getRepository() once for each directory. The latrer calls isRepositoryFor() for given directory (which in case of Perforce triggers the execution of the p4 command) and traverses the list of repositories: https://github.com/oracle/opengrok/blob/4ba0ec70577f64551a572196c2c69297ab7e79dc/opengrok-indexer/src/main/java/org/opengrok/indexer/history/RepositoryFactory.java#L47-L62 Git is listed before Perforce in that list so if addRepositories() checked only Git in subdirectories of already detected Git repository, the call to other repositories' isRepositoryFor() would not happen. So, I believe this is a dup of #1461.

cross commented 5 years ago

The fix applied in #1461 does not resolve this problem. I can now see, indexing with 1.2.16:

22:56:47 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src: */... - must create client 'vmhostname' to access local files.

22:56:47 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src: * - must create client 'vmhostname' to access local files.

22:56:58 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/subdir1/tests: */... - must create client 'vmhostname' to access local files.

22:56:58 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/subdir1/tests: * - must create client 'vmhostname' to access local files.

22:56:59 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/subdir1/tests/instance: */... - must create client 'vmhostname' to access local files.

22:56:59 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/subdir1/tests/instance: * - must create client 'vmhostname' to access local files.
[...]
23:01:23 WARNING: Non-zero exit status 1 from command [p4, dirs, *] in directory /disk/src/projectA/subdir4/scripts: */... - must create client 'vmhostname' to access local files.

23:01:24 WARNING: Non-zero exit status 1 from command [p4, files, *] in directory /disk/src/projectA/subdir4/scripts: * - must create client 'vmhostname' to access local files.

23:01:26 INFO: Done scanning for repositories, found 13 repositories (took 0:04:39)

I don't seem to be seeing this error for /disk/src/projectA, or even immediate subdirs of that (/disk/src/projectA/subdir1 or /disk/src/projectA/subdir2) But, am still seeing it for many (all?) directories at levels 2 levels below the top of the git repository projectA.

@vladak let me know if you understand this, or have ideas. Thanks.

vladak commented 5 years ago

I forgot to propagate the type in the non-repo case. PR #2827 should address this.

cross commented 5 years ago

Thanks. I can confirm that I no longer see these messages from within the git repository in 1.2.17.