hook.pl fails, can't obtain diff

[GoogleCodeExporter]

I've been experimenting with hook.pl, learning how it works, prior to 
integrating into my subversion post_commit hook.  So far I have not been able 
to successfully add commit information to the bugzilla database.  (The sync.pl 
tool does work though).

I'm using subversion v1.6.6 and bugzilla v5.6.2

Initially I invoked hook.pl as follows:
perl hook.pl --project="trunk" --bug=2 --revision=15510 --repo 
svn+ssh:/iris/home/svn/repo/upe_sandbox3/ --bugzilla=http://iris/bugzilla2 
--login=XXX --pass=YYY 

(BTW, for anyone new to this, the repo you must specify here must already be 
entered as a valid VCS URL in the VCS section of the Bugzilla administration.)

This resulted in the error:
Bugzilla Error: (32000) An error occurred while attempting to fetch commit 
15510 from the "trunk" project in the 
"svn+ssh://iris/home/svn/repo/upe_sandbox3/" repository. Sometimes this means 
that you specified an invalid repository or project. The error was: Network 
connection closed unexpectedly: at lib/VCI/VCS/Svn/Repository.pm line 12

The error states the network connection was closed unexpectedly so either this 
was truely a network problem or some sort of failure at the far end of the 
network connection that caused the connection to drop prematurely.  In order to 
eliminate network issues, I repeated the test but instead used a file URL for 
the repository as follows:

perl hook.pl --project="trunk" --bug=2 --revision=15510 --repo 
file:///home/svn/repo/upe_sandbox3/ --bugzilla=http://iris/bugzilla2 
--login=XXX --pass=YYY 

Now the error is:
Bugzilla Error: (-32000) TypeError in method 'svn_client_diff', argument 2 of 
type 'char const *'

Original issue reported on code.google.com by kro...@gmail.com on 23 Sep 2010 at 9:28

[GoogleCodeExporter]

Hmmm. I'm having trouble reproducing this right now, but I know that I've seen 
it before.

If you apply the attached patch to VCI (not to the VCS extension, but to the 
actual VCI library itself) does that fix it?

Original comment by avatrax...@gmail.com on 2 Oct 2010 at 1:00

• Added labels: Priority-High
• Removed labels: Priority-Medium

Attachments:

• vci-int.diff

[GoogleCodeExporter]

This did not fix it.  I tested by making the patch to lib/VCI/VCS/Svn/Commit.pm 
in my bugzilla directory, re-running checksetup.pl and re-running the hook 
script.  I repeated the test a second time but also commented out the if so 
that the debug immediately above it would execute.  I observed what looked like 
good path and rev variables printed out in my httpd error log.  Did I apply the 
patch and test correctly?

Isn't the error complaining about argument 2?  The patch seems to change 
arguments 3 and 5.

Original comment by kro...@gmail.com on 4 Oct 2010 at 5:29

[GoogleCodeExporter]

Well, my guess was that the error was counting arguments starting from 0. In 
actual fact, I have no idea what the error is talking about and I can't find 
anybody else who does, either. The value being passed is in fact a valid 
string, as you pointed out.

Original comment by avatrax...@gmail.com on 4 Oct 2010 at 11:32

[GoogleCodeExporter]

Could this be an incompatibility with my version of subversion, 1.6.6?

Original comment by kro...@gmail.com on 6 Oct 2010 at 2:38

[GoogleCodeExporter]

No, I don't think so, although you could try a later version and see. I'm using 
1.6.9 here and I *think* that I saw a similar error, but I'm not certain.

Original comment by avatrax...@gmail.com on 7 Oct 2010 at 12:34

[GoogleCodeExporter]

I just attempted to install bugzilla-vcs on a different server to see if my 
problem is server specific.  This time my bugzilla-vcs installation went a bit 
differently.
During the install of the original system, checksetup.pl never complained of 
any missing perl modules.  On this new system, checksetup.pl states "Svn: 
missing perl module(s) (see above)".  Up above it lists:
Alien-SVN: /usr/bin/perl install-module.pl SVN::Core

This module install fails due to it not finding files EXTERN.h, perl.h, and 
XSUB.h even though they are on my system at 
/usr/lib/perl5/5.10.0/i386-linux-thread-multi/
(see attached build results, line 2816).  As a result, I don't have 
bugzilla-vcs working at all on this new system.

I'm wondering if the subversion interface in my original system is something 
incompatible with bugzilla-vcs or the VCI library.  On that system, I see a 
package installed called perl-SVN-Simple.  Do you think bugzilla-vcs is using 
that package in lieu of the Svn::Core perl module and could that be the 
problem?  If so, I still haven't figured out how to build the Svn::Core module 
successfully.

Original comment by kro...@gmail.com on 12 Oct 2010 at 4:59

Attachments:

• svncore.txt

[GoogleCodeExporter]

In comment 6 I meant to say the missing .h files are in 
/usr/lib/perl5/5.10.0/i386-linux-thread-multi/CORE/

I also tried installing SVN::Core with cpan and I have the same build error.

Original comment by kro...@gmail.com on 12 Oct 2010 at 6:20

[GoogleCodeExporter]

You should be able to build Alien::SVN if you install the subversion-devel 
package.

You can also just install "subversion-perl" to get the required libraries 
installed.

Is the subversion client version different than the subversion server version?

SVN::Simple is definitely not being used.

Original comment by avatrax...@gmail.com on 12 Oct 2010 at 9:11

[GoogleCodeExporter]

By installing SVN::Simple, it installed subversion-perl as a dependency, thus 
that's why I thought it was relevant.  I didn't realize that until I went back 
and reviewed what I did.

Are you saying subversion-perl is enough and I need not bother with Alien::SVN?

subversion-perl and subversion is version 1.6.6-1.fc11 on the machine that 
doesn't work.  I will now test on my fc12 machine which uses version 
1.6.9-1.fc12 and see how that does.

Original comment by kro...@gmail.com on 12 Oct 2010 at 9:53

[GoogleCodeExporter]

I have the same TypeError now on a completely different system.  This one is 
fedora 12.  It's a fresh bugzilla and VCS install.  subversion-perl and 
subversion is version 1.6.9-1.fc12.

Original comment by kro...@gmail.com on 12 Oct 2010 at 11:17

[GoogleCodeExporter]

Okay. Would it be possible to get a copy of your repo, by any chance? It would 
be nice to be able to reproduce this consistently on a repo that I can have a 
local copy of.

Original comment by avatrax...@gmail.com on 13 Oct 2010 at 6:30

[GoogleCodeExporter]

I can't give you the repo but I'll repeat my testing with a fresh repository 
and get back to you.

Original comment by kro...@gmail.com on 13 Oct 2010 at 2:54

[GoogleCodeExporter]

I'm sorry to say but the testing I reported in comment 10 is tainted.  I didn't 
realize I was pointing the hook.pl --bugzilla argument to the bugzilla on my 
original problem system.  I assume by using the old bugzilla, I was also using 
the subversion libraries on the original system too.

I've corrected my mistake with the parameters and now can't get the hook to run 
for another reason on the new system.  Here's how it fails:

perl extensions/VCS/hook.pl --bug=2 --revision=300 
--repo=file:///home/svn/repo/sky --project=trunk --login=me@company.com 
--pass=hotdog --bugzilla=http://differentmachine.company.com/bugzilla/

mismatched tag at line 108, column 4, byte 2949:
    <link rel="shortcut icon" href="images/favicon.ico" >
  </head>
===^

 at /usr/lib/perl5/vendor_perl/5.10.0/RPC/XML/Client.pm line 351 at extensions/VCS/hook.pl line 44.

This appears to be a failure when attempting to access the bugzilla's web 
services.  I'm not sure how to proceed to resolve this error.

Original comment by kro...@gmail.com on 13 Oct 2010 at 9:25

[GoogleCodeExporter]

On the original problem machine, I tested with a brand new svn repository and 
it still fails with the TypeError.

I've attached a tar of the repository

Original comment by kro...@gmail.com on 13 Oct 2010 at 9:52

Attachments:

• svn.tgz

[GoogleCodeExporter]

The WebServices error probably means that XML-RPC isn't set up in your 
Bugzilla--you'll want to run checksetup.pl and install the necessary optional 
modules.

Thanks for the repo! I should be able to troubleshoot with that.

Original comment by avatrax...@gmail.com on 13 Oct 2010 at 9:54

[GoogleCodeExporter]

I now have a completely new installation set up for testing.
The WebServices error went away after installing RPMs for perl-SOAP-Lite and 
perl-Test-Taint.  I had previously installed perl-XML-RPC.  Perhaps your web 
page can give some help in this area.

At this point I've corrected by testing errors and the test results I specified 
in comment 10 do hold true.

So to summarize, 2 different installs, different svn repositories, different 
svn versions and I still have the TypeError problem.

I'm attaching another file that describes the environment of the 2nd machine.  
It is perl.txt which is a listing of all the perl RPMs I'm using on this Fedora 
12 machine.  I satisfied most of the perl module dependencies by using yum to 
load the packages I needed.

I also installed the VCI package from CPAN via the install-module.pl script.

Just in case it's useful I'm also enclosing the output from checksetup.pl

Original comment by kro...@gmail.com on 13 Oct 2010 at 10:39

Attachments:

• perl.txt
• checksetup.txt

[GoogleCodeExporter]

Okay, oddly enough, I can't reproduce this with your svn.tgz, like this:

perl hook.pl --bug=1 --revision=2 --project=trunk 
--repo=file:///var/www/html/vcsint/svn/garbage/ --login=XXX --pass=YYY 
--bugzilla=http://localhost/vcsint/

I'm also on Fedora 12, using all the standard Fedora 12 packages.

Original comment by avatrax...@gmail.com on 15 Oct 2010 at 10:48

[GoogleCodeExporter]

So then help me understand how this works.  Does the hook.pl script issue a 
request to the Bugzilla instance via the web services and then does the 
subversion diff get generated in the context of Bugzilla running, i.e., 
Bugzilla when processing the request actually runs the code to do the diff. 

If so, this might be a difference in Bugzilla config.  The only thing I can 
think I'm doing that's probably less common is setting PROJECT before I run 
checksetup.pl.

Assuming my understanding of how this executes is correct and you have the same 
F12 libraries as me, it seems the only thing left as a variable is Bugzilla 
configuration related.  I'll have to ponder this a bit.

Original comment by kro...@gmail.com on 16 Oct 2010 at 5:10

[GoogleCodeExporter]

Yeah, you're exactly right about how it works. Yeah, I did notice that you're 
using PROJECT. I don't know if that would affect things. There might also be 
something about your Apache configuration--are you running under mod_perl or 
suexec? I didn't test those two situations.

Original comment by avatrax...@gmail.com on 16 Oct 2010 at 5:20

[GoogleCodeExporter]

I set up my system with the svn at the same path you're using and I used the 
same parameters to hook (except for the bugzilla params) and I get the output:

Bugzilla Error: (32000) There is no commit with the id "2" for "trunk" in the 
"file:///var/www/html/vcsint/svn/garbage/" repository.

trunk wasn't added until revision 2 so I don't see how it worked for you.

If I change revision from 2 to 3, I'm back to the TypeError again.

In the bugzilla parameters, vcs_repos is set to Svn 
file:///var/www/html/vcsint/svn/garbage/ and vcs_web is blank, and vcs_path is 
the default path (I didn't change it).  I don't see any documentation for 
vcs_path.  Is it's setting crucial for this to work?

Did you look at my checksetup.txt file to see if we have the same perl modules 
set up for bugzilla?

Original comment by kro...@gmail.com on 16 Oct 2010 at 5:39

[GoogleCodeExporter]

I reconfigured my bugzilla not to use PROJECT.  The problem persisted.
I then reconfigured bugzilla to use mod_perl instead of mod_cgi.  The bugzilla 
instance is working, but now the hook script gives a different error:

Bugzilla Error: (Client) Application failed during request deserialization: 
no element found at line 1, column 0, byte -1 at 
/usr/lib/perl5/vendor_perl/5.10.0/i386-linux-thread-multi/XML/Parser.pm line 187

Parser.pm has the code:

    170   my $expat = new XML::Parser::Expat(@expat_options, @_);
    171   my %handlers = %{$self->{Handlers}};
    172   my $init = delete $handlers{Init};
    173   my $final = delete $handlers{Final};
    174 
    175   $expat->setHandlers(%handlers);
    176 
    177   if ($self->{Base}) {
    178     $expat->base($self->{Base});
    179   }
    180 
    181   &$init($expat)
    182     if defined($init);
    183 
    184   my @result = ();
    185   my $result;
    186   eval {
    187     $result = $expat->parse($arg);
    188   };
    189   my $err = $@;
    190   if ($err) {
    191     $expat->release;
    192     die $err;
    193   }

Original comment by kro...@gmail.com on 16 Oct 2010 at 3:42

[GoogleCodeExporter]

After googling around I got the idea to use bz_webservice_demo.pl to see what 
would happen.  It didn't seem to work.  Output is attached.  Perhaps this is 
the root of the problem now.

My latest checksetup.pl output is attached too.

Original comment by kro...@gmail.com on 16 Oct 2010 at 5:09

Attachments:

• otpt.txt
• checksetup.txt

[GoogleCodeExporter]

To avoid missing a perl module, I know did perl install-module.pl -all and 
every possible perl module is installed.  The problems with hook and 
bz_webservice_demo persist.

Original comment by kro...@gmail.com on 16 Oct 2010 at 8:01

[GoogleCodeExporter]

There is a problem with SOAP::Lite's recent versions under mod_perl. You need 
an older version of mod_perl.

Original comment by avatrax...@gmail.com on 16 Oct 2010 at 9:43

[GoogleCodeExporter]

Oops! I mean you need an older version of SOAP::Lite!

Original comment by avatrax...@gmail.com on 16 Oct 2010 at 11:51

[GoogleCodeExporter]

I have version 0.710.10-1.fc12.  Can you suggest what version I should go back 
to?

Original comment by kro...@gmail.com on 17 Oct 2010 at 12:11

[GoogleCodeExporter]

I think I'm going to need some help on how I would downgrade SOAP::Lite.
I tried removing perl-SOAP-Lite with yum and I replaced it with perl 
install-module.pl SOAP::Lite but that installs an even newer version (.712).
I tried a few 710-06 rpms I found, but they won't install because of dependency 
issues.

How do you recommend I do this downgrade?  

Original comment by kro...@gmail.com on 17 Oct 2010 at 5:40

[GoogleCodeExporter]

In the past day I did find an older perl-SOAP-Lite RPM, version 
0.710-08.2.fc11.  Once I installed that I was back to the TypeError again.  So 
in trying to figure out what might be different between my system and yours, I 
made an assumption you weren't running the exact 3.6.2 version of bugzilla.  In 
fact you might be running 3.7.3 or trunk code.  I then decided to try a 3.7.3 
upgrade.  Believe it or not, once I did that, the hook.pl script works!  It 
works unless you repeat it's use with the same bug id and then it crashes:

Bugzilla Error: (-32000) DBD::mysql::db do failed: Duplicate entry 
'3-2-file:///var/www/html/vcsint/svn/garbage/-trunk' for key 
'vcs_commit_revision_idx' [for Statement "INSERT INTO vcs_commit (revno, 
creator, project, bug_id, author, message, revision, repo, type, commit_time) 
VALUES (?,?,?,?,?,?,?,?,?,?)"] at /usr/share/bugzilla-3.7.3/Bugzilla/Object.pm 
line 508

So this is significant progress.  But since I want to use hook.pl on a 
production system that's running 3.6.2 and an upgrade to 3.7.3 is strongly 
discouraged on the bugzilla download web page what can I do?

Is there some code in 3.7.3 I or you can backport to a 3.6.2 system to make 
this work?
Is 4.0 really on schedule and will it come out this year.  Supposedly the 
release candidate was due in September.  If its release is imminent, waiting 
for it is a viable solution.

I suspect my switching to mod_perl wasn't really a factor in making this work.  
I'll revert back to mod_cgi and verify it still works.

Original comment by kro...@gmail.com on 18 Oct 2010 at 5:11

[GoogleCodeExporter]

Ah ha, yes, I'm testing against trunk. I'll test against 3.6.

Original comment by avatrax...@gmail.com on 18 Oct 2010 at 5:16

[GoogleCodeExporter]

I switched back to mod_cgi using 3.7.3 and hook.pl works that way too.  
Therefore this hook.pl issue seems to be related to something in bugzilla 3.6.2.

Original comment by kro...@gmail.com on 18 Oct 2010 at 5:35

[GoogleCodeExporter]

Do you think this is a bugzilla bug?  Should I enter a bug against 3.6.2?

Original comment by kro...@gmail.com on 20 Oct 2010 at 10:02

[GoogleCodeExporter]

No, it's related to the extension. I just have to have some time to debug it.

Original comment by avatrax...@gmail.com on 20 Oct 2010 at 10:48

[GoogleCodeExporter]

Considering hook.pl appeared to work in version 3.7.3, would it be reasonable 
to assume that if I upgrade to bugzilla 4.0rc1 or bugzilla 4.0, that hook.pl 
would work?  If so, I'll do that.

Original comment by kro...@gmail.com on 29 Nov 2010 at 8:31

[GoogleCodeExporter]

Yes, if it worked in 3.7.3, it should work in 4.0rc1. I still want to fix the 
plugin to work with 3.6, though.

Original comment by avatrax...@gmail.com on 30 Nov 2010 at 1:00

[GoogleCodeExporter]

Has this been fixed? I am not able to get this working in 3.6.3.

Bugzilla Error: (Client) Application failed during request deserialization:
no element found at line 1, column 0, byte -1 at 
/usr/lib64/perl5/vendor_perl/5.8.8/x86_64-linux-thread-multi/XML/Parser.pm line 
187

Original comment by shi...@gmail.com on 2 Dec 2010 at 12:58

[GoogleCodeExporter]

It has not yet been fixed. However, your error is unrelated to this one--the 
problem you are experiencing is a bug in SOAP::Lite, and the solution is to 
downgrade to an earlier version of SOAP::Lite.

Original comment by avatrax...@gmail.com on 2 Dec 2010 at 3:31

[GoogleCodeExporter]

In Commit.pm
sub run_create_validators {
...
    my @file_rows;
    foreach my $file (@{ $commit->as_diff->files }) {

'commit->as_diff->files' have caused error on my environment;
Bugzilla Error: (-32000) TypeError in method 'svn_client_diff', argument 2 of 
type 'char const *'

So I have commented out entire foreach loop.
Personally it is ok because how many files or which files are commited can be 
checked by using repository browser such as WevSVN.

Original comment by kohir...@gmail.com on 17 Feb 2011 at 12:45

[GoogleCodeExporter]

I'm just confirming that the hook.pl script is working with bugzilla 4.0 for 
me, but I have some issues with the script that I'll address in another issue.

Original comment by kro...@gmail.com on 21 Feb 2011 at 3:41

[GoogleCodeExporter]

I'm having this same issue, bugzilla 4.0.2:

mismatched tag at line 91, column 4, byte 2904:
    <link rel="shortcut icon" href="images/favicon.ico" >
  </head>
===^

 at /usr/share/perl5/RPC/XML/Client.pm line 404 at /data/bugzilla/extensions/VCS/hook.pl line 44.

Original comment by tomma...@gmail.com on 26 Aug 2011 at 6:08