org.eclipse.jgit.errors.MissingObjectException: Missing unknown ba07cec424cee7979d5d8354e52c4237a4e572b7

[dkultasev]

I am trying to setup verigreen application. I've downloaded war file with hook jar files from the: https://github.com/Verigreen/verigreen/releases

My configuration is following: Bitbucket - runs on 7990 port VG on Tomcat - runs on 8081 port (Java 7.45) Jenkins on Tomcat (different than for VG) - runs on 8080 port (Java 8+) OS - Windows 10 64BIT (Virtual machine)

1. My bitbucket repository (master) is with "Public access" enabled.
2. I've downloaded "External Pre Receive Hook" plugin in Bitbucket and the configuration is following:
   • Executeable: "C:\Users\kultasev\tmp\veri_hook\vg_hook.bat"
   • Parameters: "master"
3. Tomcat settings:
   • CATALINA_BASE: "C:\Users\kultasev\tmp\apache-tomcat-7.0.81"
   • CATALINA_HOME: "C:\Users\kultasev\tmp\apache-tomcat-7.0.81"
   • CATALINA_TMPDIR: "C:\Users\kultasev\tmp\apache-tomcat-7.0.81\temp"
   • JRE_HOME: "C:\Program Files\Java\jdk1.7.0_45"
   • CLASSPATH: "C:\Users\kultasev\tmp\apache-tomcat-7.0.81\bin\bootstrap.jar;C:\Users\kultasev\tmp\apache-tomcat-7.0.81\bin\tomcat-juli.jar"
4. Tomcat setenv.bat:
   • set "VG_HOME=C:\Users\kultasev\tmp\veri_conf"
   • set JPDA_OPTS=-agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=y
   • exit /b 0
5. config.properties (Located at C:\Users\kultasev\tmp\veri_conf\config.properties)

   
   ###############################################################################
   # Copyright 2015 Hewlett-Packard Development Company, L.P.
   #
   # Licensed under the Apache License, Version 2.0 (the "License");
   # you may not use this file except in compliance with the License.
   # You may obtain a copy of the License at
   #
   #        http://www.apache.org/licenses/LICENSE-2.0
   #
   # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   # See the License for the specific language governing permissions and limitations under the License.
   ###############################################################################

Jenkins Configuration

jenkins.url=http://localhost:8080/ jenkins.user=dimka jenkins.password=da9085a7bd334a187d1cf9d12e10ccce ##############Changes in this field will be effected after save (no restart is needed)################

Warning! Do not change the jobName parameter while there are commits being verified in the system!

jenkins.jobName=btbckt_test jenkins.branchParamName=SCM_BRANCH

The maximum number of seconds Verigreen waits for a verification build to complete.

jenkins.timeoutInSeconds=3600

Git configuration

git.repositoryLocation=C:\Users\kultasev\tmp\tmp_source\.git ##############Changes in this field will be effected after save (no restart is needed)################ git.protectedBranches=refs/heads/master ##############Changes in this field will be effected after save (no restart is needed)################ git.permittedUsers=

JGitOperator configuration

These sets Verigreen's user.name and user.email git configuration parameters.

commited.by.collector=VeriGreen email.address=somemail@timepayment.com

Mail Configuration

The SMTP server Verigreen will use to send its emails.

SMTP authentication is not currently supported.

mail.server=localhost

Collector Display Name

collectors=VeriGreen

The URL of the Verigreen collector.

This is the address that need be configured in the Verigreen git-hook configuration file, hook.properties.

Add / after the port number, if deployed as war.

This parameter is required for Verigreen to work.

collector.address=http://localhost:8081/verigreen/rest

Verigreen Internal Configuration

CacheCleanerJob and BranchCleanerJob configuration.

0 => Remove all but Running commits.

daysThreashold=3

How many hours to wait before Verigreen runs its cache and branch cleaning jobs.

repeatHourlyForever=24

JobScheduler configuration

The number of seconds after which Verigreen wakes up to process any undone items in the commit item list.

repeatSecondlyForever=10

The number of iterations to wait before attempting to re-trigger a new build for a verification.

timeout.counter=4

The number of Jenkins build re-trigger attempts before it is setting the commit as Trigger_Failed.

default_count=3

Queue Manager:

The maximum number of commits Verigreen will run verification for at the same time.

Set this to no more than the number of concurrent verification builds that Jenkins can run simultaneously.

number.commits=5

The number of attempts Verigreen makes to verify the configured Jenkins Job exists.

job.retry.counter=3

Force push password hash

hash.password=c21e299d9ec8db9718e162aca10689317d94f34f

##############Changes in this field will be effected after save (no restart is needed)################

enable/disable retry of Failed commits

full.push=disabled

Additional parameters that be be sent to Jenkins.

Set value to 'true' (without quotes) to send the key/value pair using below key name.

Alternatively, set the value to the Jenkins String Parameter name.

jenkinsparam.mode=params jenkinsparam.longid=false jenkinsparam.parent=false jenkinsparam.protected=false jenkinsparam.commitid=false jenkinsparam.committer=false

FAQ Page Configuration

Enable offline FAQ page

In case the Internet connection is limited or restricted this parameter should be set to true in order to have access to the FAQ page

If there are no restrictions this parameter should be set to false, the FAQ page being accessed directly on the github page

Set to true and provide below offline URLs and files as needed.

faq.offline=true faq.offline.git=faqpage.html#q-i-got-a-git_failure-status-what-does-it-mean faq.offline.timeout=faqpage.html#q-i-got-a-time_out-status-what-does-it-mean

These properties should not be changed

faq.online.git=https://github.com/Verigreen/verigreen/wiki/Verigreen-FAQ#q-i-got-a-git_failure-status-what-does-it-mean faq.online.trigger=https://github.com/Verigreen/verigreen/wiki/Verigreen-FAQ#q-i-got-a-trigger_failed-status-what-does-it-mean faq.online.timeout=https://github.com/Verigreen/verigreen/wiki/Verigreen-FAQ#q-i-got-a-time_out-status-what-does-it-mean

faq.offline.git=faqpage.html#q-i-got-a-git_failure-status-what-does-it-mean faq.offline.trigger=faqpage.html#q-i-got-a-trigger_failed-status-what-does-it-mean faq.offline.timeout=faqpage.html#q-i-got-a-time_out-status-what-does-it-mean

6. git-hook.jar (+ all related jars), hook.properties, vg_hook.bat are located at C:\Users\kultasev\tmp\veri_hook
7. vg_hook.bat

@ECHO OFF REM # REM # Copyright 2015 Hewlett-Packard Development Company, L.P. REM # REM # Licensed under the Apache License, Version 2.0 (the "License"); REM # you may not use this file except in compliance with the License. REM # You may obtain a copy of the License at REM # REM # http://www.apache.org/licenses/LICENSE-2.0 REM # REM # Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. REM # See the License for the specific language governing permissions and limitations under the License. REM # REM # Sample git pre-receive shell script to execute the Verigreen hook. REM # Please use this carefully to avoid any issues with git hook triggering REM # can and should be adapted for use with/for other hooks. REM # REM # Parameters: Accepts one required parameter - the repository name which is protected REM # SETLOCAL enabledelayedexpansion SET "DEBUG_FILE=C:\Users\kultasev\tmp\vg.txt" REM # REM # Required parameters - Set these as needed REM # REM The IF conditions are used to set the value only if the variable in undefined REM Remove the IF part and use only the SET part in case variable need be overwritten

REM Set the path to the hook.properties file REM modify as needed. echo %username% > "C:\Users\kultasev\tmp\vg2.txt" SET "VG_HOOK=C:\Users\kultasev\tmp\veri_hook" REM ECHO VG_HOOK: %VG_HOOK%

REM Set the path to the git-hook.jar file REM modify as needed. SET "VG_PATH=C:\Users\kultasev\tmp\veri_hook" REM ECHO VG_PATH: %VG_PATH%

REM This sets the JAVA_HOME for use. REM modify as needed. SET "JAVA_HOME=C:\Program Files\Java\jdk1.7.0_45"

REM # REM # Main REM #

REM Check number of arguments specified on run REM First parameter passed to this script is the repository name IF "%1" == "" ( ECHO "Error: Script must be run with one (1) argument!" EXIT /B 2 )

REM Read STDIN (oldrev newrev ref) for /F "tokens=*" %%a in ('more') do ( SET LINE=%%a REM echo #LINE: %LINE% )

REM debug ECHO VG HOOK: %VG_HOOK% > %DEBUG_FILE% ECHO Vector: %LINE% >> %DEBUG_FILE% ECHO JAVA_HOME: %JAVA_HOME% >> %DEBUG_FILE%

REM Note: !!! Each hook call must exit if its result is not 0 !!! echo CALL: "%JAVA_HOME%\bin\java.exe" -jar "%VG_PATH%\git-hook.jar" %1 %LINE% REM >> %DEBUG_FILE% call "%JAVA_HOME%\bin\java.exe" -jar -Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=n,address=5005 "%VG_PATH%\git-hook.jar" %1 %LINE% 2>"C:\Users\kultasev\tmp\vg_err.txt"

ECHO ERRORLEVEL: %ERRORLEVEL% >> %DEBUG_FILE% EXIT /B %ERRORLEVEL%

8. hook.properties
`collector.address=http://localhost:8081/verigreen/rest`

My local repo is synced with the remote master.

When I try to push I am getting following errors:
**this is what I get in UI (git):**

> repository: Remote error: Repository not found
> The requested repository does not exist, or you do not have permission to
> access it.

**This is what I get in the log file:**

> VG HOOK: C:\Users\kultasev\tmp\veri_hook 
> Vector: b37c01369c8fdd661a28031baee072cd4cf3051a ba07cec424cee7979d5d8354e52c4237a4e572b7 refs/heads/master 
> JAVA_HOME: C:\Program Files\Java\jdk1.7.0_45 
> ERRORLEVEL: 1

**This is what I get in the code:**

> Exception in thread "main" java.lang.RuntimeException: Failed retrieving commit data [ba07cec424cee7979d5d8354e52c4237a4e572b7]
> at com.verigreen.jgit.JGitOperator.getRevCommit(JGitOperator.java:122)
> at com.verigreen.hook.BranchOperator.processBranch(BranchOperator.java:54)
> at com.verigreen.hook.EntryPoint.main(EntryPoint.java:38)
> Caused by: org.eclipse.jgit.errors.MissingObjectException: Missing unknown ba07cec424cee7979d5d8354e52c4237a4e572b7
> at org.eclipse.jgit.internal.storage.file.WindowCursor.open(WindowCursor.java:148)
> at org.eclipse.jgit.lib.ObjectReader.open(ObjectReader.java:229)
> at org.eclipse.jgit.revwalk.RevWalk.parseAny(RevWalk.java:809)
> at org.eclipse.jgit.revwalk.RevWalk.parseCommit(RevWalk.java:722)
> at com.verigreen.jgit.JGitOperator.getRevCommit(JGitOperator.java:120)
> ... 2 more

I tried to find the place where is it failing and it is:

class: verigreen/git-hook/src/com/verigreen/common/spring/GitHookApi.java
method: public RevCommit getRevCommit(String commitId) 
line (120): ret = walk.parseCommit(_repo.resolve(commitId));

error message:

> org.eclipse.jgit.errors.MissingObjectException: Missing unknown ba07cec424cee7979d5d8354e52c4237a4e572b7

[soninob]

@edagan can you take a look? why aren't we able to retrieve the commit?

[eitansch]

Hi,

The UI response you're getting is from Git, not from VG ("repository: Remote error: Repository not found The requested repository does not exist, or you do not have permission to access it.")

If you disable VG, can you push to the remote BitBucket? If so, it means VG can't access BitBucket (which user does it use? does it have the access to BB? FW issues).

[dkultasev]

I can push if I disable VG. All the applications are ran under the same user (my user) as I start tomcat manually.

[dkultasev]

Anyway, as I wrote that the repository is with Public access

[flaresolar]

Hi,

All indications show, IMO, that VG cannot find its work repository.

@dkultasev, in the config.properties file of VG, could you please try replacing: git.repositoryLocation=C:\\Users\\kultasev\\tmp\\tmp_source\\.git

with: git.repositoryLocation=C:/Users/kultasev/tmp/tmp_source/.git

Then, restart Verigreen.

If this doesn't help, msg me and we delve into this further.

[dkultasev]

Opening repositories: C:\Users\kultasev\tmp\tmp\tmp The following errors were reported during push: refs/heads/master, pre-receive hook declined

[flaresolar]

Looks like VG finds its work repo now.

• Any more lines to the push failure message?
• Is there any data in the VG web page?
• What is your Jenkins version? @soninob - To my knowledge VG only supports Jenkins 1.x, though there was some code submitted to support Jenkins 2.x.

[soninob]

@flaresolar - it is to earlier for jenkins. the hook should first communicate with the collector. only after notification that we need to run VG, we are triggering Jenkins after dealing with the branches and merges. @dkultasev if you have more details on the error, it will help, 10x.

[dkultasev]

https://content.screencast.com/users/sdks/folders/Jing/media/2c9891eb-9a09-4b69-89e3-09d0d5cc3ae2/2017-09-28_1039.png

What else can I provide?

There is no lines in UI.

[soninob]

I'm getting 404 on the link :-)

[dkultasev]

Jenkins 2,73,1

[dkultasev]

https://content.screencast.com/users/sdks/folders/Jing/media/2c9891eb-9a09-4b69-89e3-09d0d5cc3ae2/2017-09-28_1039.png

[dkultasev]

Hyperlink is not working somehow. Copy paste URL in the browser

[flaresolar]

Getting 404 on both link.

No lines in UI means that the hook is not communicating with VG. Lets check the hooks debug file: C:\Users\kultasev\tmp\vg.txt @dkultasev could you please share its contents?

[dkultasev]

VG HOOK: C:\Users\kultasev\tmp\veri_hook
Vector: ba07cec424cee7979d5d8354e52c4237a4e572b7 e07a3ccf5a3939ea8a9d6d759eeb2d293dd6f165 refs/heads/master
JAVA_HOME: C:\Program Files\Java\jdk1.7.0_45
ERRORLEVEL: 1

[dkultasev]

@flaresolar copy paste the url to the browser. I do not know why, but hyperlink is not working

[flaresolar]

Looks like the repo cannot find or pull the new commit.

Is the repo in both below folders is the same? Why was this changed? Due to the "_" character?

• C:\Users\kultasev\tmp\tmp\tmp
• C:/Users/kultasev/tmp/tmp_source/.git

[dkultasev]

C:\Users\kultasev\tmp\tmp\tmp - is the folder where the bitbucket puts the data C:/Users/kultasev/tmp/tmp_source/.git - is the cloned folder from bitbucket

[flaresolar]

Thanks, understood.

Please check BitBucket for VG's temporary branch for this repository. If I recall correctly, VG's temporary branches begin with "vg_", and should be shown in near the top of BitBucket's Branches for this repository, in BitBucket's UI.

If the temp. branch does not exist on BitBucket, then the hook failed to create it. If it the temp. branch exists, then it seems the work repository at tmp_source could not pull it.

[dkultasev]

I do not have such branches

[flaresolar]

@dkultasev one more thing - Could you please show me the hook config in BitBucket?

[dkultasev]

sure

[soninob]

@dkultasev not sure I understood: C:\Users\kultasev\tmp\tmp\tmp - is the folder where the bitbucket gets the data C:/Users/kultasev/tmp/tmp_source/.git - is the cloned folder from bitbucket

as we talked earlier, the second path is your VG working repository (and it is importent that you will not push from it. I guess you meant the first path is where you did the push - if so, the error you got is related not to VG but to your repositroty from where you push the commits:

(Opening repositories: C:\Users\kultasev\tmp\tmp\tmp The following errors were reported during push: refs/heads/master, pre-receive hook )

[flaresolar]

this is getting a tad hard to debug like this. @dkultasev We can try a short Skype session. If possible, PM me and I'll give you my skype handle.

[flaresolar]

One more clarification, please - Is BitBucket on the same machine as well? otherwise, the URL in hook.properties is unreachable from BitBucket

Sorry, just looked at the screenshot again. all on same machine

[dkultasev]

Everything is the on the same machine (localhost), on the same physical and logical drive

about git paths: I cloned repository to the new location, and used this path in the VG config --> C:\Users\kultasev\tmp2\tmp.git All the changes and the place where I am trying pushes are located at -->C:\Users\kultasev\tmp\tmp_source.git

In the config file I've tried following values: C:\Users\kultasev\tmp2\tmp.git C:\Users\kultasev\tmp2\tmp\.git C:/Users/kultasev/tmp2/tmp/.git

[dkultasev]

@flaresolar I am on $kype --> dmitrij[replace_with_dot]kultasev

[dkultasev]

thanks for the @flaresolar as we made 1 step further. We've found that the problem is with the directory where the batch file is executed from. We were able to run the hook jar file manually after we changed directory to the repository data. In my case it was "C:\Atlassian\ApplicationData\Bitbucket\shared\data\repositories\1" however we still couldn't achieve how to make it working automatically

[dkultasev]

I've tried to merge pull request made by @shivarajugowda. The only difference I've made is that I am using the latest jGit version instead of the one that was provided in that PR.

[soninob]

hey @dkultasev - what is the status of this one?

[flaresolar]

This has been reproduced on both Windows and Linux. Trying to determine which version(s) introduced a change that broke the git-hook.jar Also looking into the hook's code.

[soninob]

@flaresolar - any chance you tried with the old version of jgit?

[flaresolar]

I am working with the 2.5.6 release version files, so yes, using old jGit version. Looks like the hook is no longer able to read the commit info for the new push. I can read the info correctly within the .sh file which calls the hook, so it's not a git or bitbucket issue.

[soninob]

Hello @dkultasev, it's been a while :-) Can you tell if you managed to solve this issue? Can you suggest which version of Bitbucket did you use? I suspect that Atlassian did some changes in their latest versions... I checked the hook code and when we are creating the jgitoperator instance we are giving it as a repo path the value '.'
Also, you mentioned that you managed to run it, from the path C:\Atlassian\ApplicationData\Bitbucket\shared\data\repositories\1, I test it and still received that same result. Maybe you did anything else?

[dkultasev]

Hi @soninob. We don't need that anymore, but for the record the Bitbucket is of v4.1.0