search

bioperl / bioperl-live-redmine

Legacy tickets migrated from the OBF Redmine issue tracker: http://redmine.open-bio.org
0 stars 0 forks source link

bioperl-ext: make test fails using io_lib v. 1.9 or higher #13

Open cjfields opened 8 years ago

cjfields commented 8 years ago

Author Name: Martin Mokrejs (Martin Mokrejs) Original Redmine Issue: 2074, https://redmine.open-bio.org/issues/2074 Original Date: 2006-08-12 Original Assignee: Bioperl Guts

I tried to compile current bioperl-ext from CVS against io_lib-1.9.0 and surprisingly succeeded. ;-) But the make test fails with:

$ make test make[1]: Entering directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Leaving directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Entering directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ make[1]: Leaving directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-MExtUtils::Command::MM” “-e” “test_harness(0, ‘blib/lib’, ‘blib/arch’)” t/*.t t/basic….ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.03 cusr + 0.02 csys = 0.05 CPU) make[1]: Entering directory `/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-I../blib/lib” “-I../blib/arch” test.pl 1..9

  1. Running under perl version 5.008008 for linux
  2. Current time local: Sat Aug 12 20:01:43 2006
  3. Current time GMT: Sat Aug 12 18:01:43 2006
  4. Using Test.pm version 1.25 ok 1 ok 2 ok 3 ok 4 ProteinSW Matrix calculation: [ 0] Cells 0% ok 5 ok 6 ok 7 ok 8

——————- EXCEPTION ——————- MSG: Algorithm must be either 1 or 2 STACK Bio::Tools::dpAlign::new /usr/lib/perl5/vendor_perl/5.8.8/Bio/Tools/dpAlign.pm:286 STACK toplevel test.pl:96

make[1]: *\ [test_dynamic] Error 255 make[1]: Leaving directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Entering directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-MExtUtils::Command::MM” “-e” “test_harness(0, ‘../blib/lib’, ‘../blib/arch’)” test.pl test….dubious Test returned status 0 (wstat 8, 0x8) DIED. FAILED tests 3-94 Failed 92/94 tests, 2.13% okay Failed Test Stat Wstat Total Fail List of Failed ———————————————————————————————————————- test.pl 0 8 94 184 3-94 Failed 1/1 test scripts. 92/94 subtests failed. Files=1, Tests=94, 0 wallclock secs ( 0.28 cusr + 0.04 csys = 0.32 CPU) Failed 1/1 test programs. 92/94 subtests failed. make[1]: *** [test_dynamic] Error 255 make[1]: Leaving directory `/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ $

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2006-08-28T00:51:01Z

The bioperl-ext README states that you may have to move os.h for io_lib to the INC directory (as well as config.h if needed) and that modifications to the files may need to be made.

I managed to get bioperl-ext working with io_lib 1.90 with 2 tests failing. When running tests for bioperl-live I got several similar errors involving Inline, but following the Inline suggestion got them to work fine (though some extra errors stayed around, annoyingly).

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2006-08-29T12:42:05Z

Upon further inspection, I found that v 1.9 and v 1.10 of io_lib do not work properly with bioperl-ext. Currently, you must use v 1.8.11/1.8.12. Furthermore, make sure to modify io_lib’s os.h :

include

to

include “config.h”

and copy both os.h and config.h to the include directory (normally /usr/local/include/io_lib). The tests may not work otherwise. This last bit is stated in the README.

I’ve added a bit to the docs about using the older io_lib version now and committed it to CVS. I’ll also notify Aaron Mackey about this to see what he says. However, I don’t see this getting fixed anytime soon to use the updated io_lib, so I’m changing this to LATER.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Martin Mokrejs Original Date: 2006-08-30T06:12:58Z

Hi Chris, I have updated the cvs versions and installed io_lib-1.8.12b and installed os.h and config.h into /usr/include/io_lib/ where the other headers reside as well.

I have edited the os.h file but that did not help either (read I had never problems to compile bioperl-ext). The problem is only with execution of scripts using that perl module.

$ make test make[1]: Entering directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Leaving directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Entering directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ make[1]: Leaving directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-MExtUtils::Command::MM” “-e” “test_harness(0, ‘blib/lib’, ‘blib/arch’)” t/*.t t/basic….ok All tests successful. Files=1, Tests=1, 0 wallclock secs ( 0.02 cusr + 0.01 csys = 0.03 CPU) make[1]: Entering directory `/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-I../blib/lib” “-I../blib/arch” test.pl 1..9

  1. Running under perl version 5.008008 for linux
  2. Current time local: Wed Aug 30 11:44:43 2006
  3. Current time GMT: Wed Aug 30 09:44:43 2006
  4. Using Test.pm version 1.25 ok 1 ok 2 ok 3 ok 4 ProteinSW Matrix calculation: [ 0] Cells 0% ok 5 ok 6 ok 7 ok 8

——————- EXCEPTION ——————- MSG: Algorithm must be either 1 or 2 STACK Bio::Tools::dpAlign::new /usr/lib/perl5/vendor_perl/5.8.8/Bio/Tools/dpAlign.pm:286 STACK toplevel test.pl:96

make[1]: *\ [test_dynamic] Error 255 make[1]: Leaving directory /home/mmokrejs/proj/bioperl/bioperl-ext/Bio/Ext/Align’ make[1]: Entering directory/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-MExtUtils::Command::MM” “-e” “test_harness(0, ‘../blib/lib’, ‘../blib/arch’)” test.pl test….ok 1/94# Test 7 got: “0” (test.pl at line 110 TODO**)

  1. Expected: (We don’t have the ability to write files for abi format)
  2. test.pl line 110 is: ok(0, undef, “We don’t have the ability to write files for $format format”) for 1..7;
  3. Test 8 got: “0” (test.pl at line 110 fail #2 TODO)
  4. Expected: (We don’t have the ability to write files for abi format)
  5. Test 9 got: “0” (test.pl at line 110 fail #3 TODO)
  6. Expected: (We don’t have the ability to write files for abi format)
  7. Test 10 got: “0” (test.pl at line 110 fail #4 TODO)
  8. Expected: (We don’t have the ability to write files for abi format)
  9. Test 11 got: “0” (test.pl at line 110 fail #5 TODO)
  10. Expected: (We don’t have the ability to write files for abi format)
  11. Test 12 got: “0” (test.pl at line 110 fail #6 TODO)
  12. Expected: (We don’t have the ability to write files for abi format)
  13. Test 13 got: “0” (test.pl at line 110 fail #7 TODO)
  14. Expected: (We don’t have the ability to write files for abi format)
  15. Test 14 got: “0” (test.pl at line 62 TODO)
  16. Expected: (Still missing test files for alf format)
  17. test.pl line 62 is: ok(0, undef, “Still missing test files for $format format”) for (1..$formatlooptests);
  18. Test 15 got: “0” (test.pl at line 62 fail #2 TODO)
  19. Expected: (Still missing test files for alf format)
  20. Test 16 got: “0” (test.pl at line 62 fail #3 TODO)
  21. Expected: (Still missing test files for alf format)
  22. Test 17 got: “0” (test.pl at line 62 fail #4 TODO)
  23. Expected: (Still missing test files for alf format)
  24. Test 18 got: “0” (test.pl at line 62 fail #5 TODO)
  25. Expected: (Still missing test files for alf format)
  26. Test 19 got: “0” (test.pl at line 62 fail #6 TODO)
  27. Expected: (Still missing test files for alf format)
  28. Test 20 got: “0” (test.pl at line 62 fail #7 TODO)
  29. Expected: (Still missing test files for alf format)
  30. Test 21 got: “0” (test.pl at line 62 fail #8 TODO)
  31. Expected: (Still missing test files for alf format)
  32. Test 22 got: “0” (test.pl at line 62 fail #9 TODO)
  33. Expected: (Still missing test files for alf format)
  34. Test 23 got: “0” (test.pl at line 62 fail #10 TODO)
  35. Expected: (Still missing test files for alf format)
  36. Test 24 got: “0” (test.pl at line 62 fail #11 TODO)
  37. Expected: (Still missing test files for alf format)
  38. Test 25 got: “0” (test.pl at line 62 fail #12 TODO)
  39. Expected: (Still missing test files for alf format)
  40. Test 31 got: “0” (test.pl at line 107 TODO)
  41. Expected: (Can’t write valid ctf files until we have a trace object)
  42. test.pl line 107 is: ok(0, undef, “Can’t write valid ctf files until we have a trace object”) for 1..7;
  43. Test 32 got: “0” (test.pl at line 107 fail #2 TODO)
  44. Expected: (Can’t write valid ctf files until we have a trace object)
  45. Test 33 got: “0” (test.pl at line 107 fail #3 TODO)
  46. Expected: (Can’t write valid ctf files until we have a trace object)
  47. Test 34 got: “0” (test.pl at line 107 fail #4 TODO)
  48. Expected: (Can’t write valid ctf files until we have a trace object)
  49. Test 35 got: “0” (test.pl at line 107 fail #5 TODO)
  50. Expected: (Can’t write valid ctf files until we have a trace object)
  51. Test 36 got: “0” (test.pl at line 107 fail #6 TODO)
  52. Expected: (Can’t write valid ctf files until we have a trace object)
  53. Test 37 got: “0” (test.pl at line 107 fail #7 TODO)
  54. Expected: (Can’t write valid ctf files until we have a trace object) test….ok All tests successful. Files=1, Tests=94, 3 wallclock secs ( 2.44 cusr + 0.06 csys = 2.50 CPU) make[1]: Leaving directory `/home/mmokrejs/proj/bioperl/bioperl-ext/Bio/SeqIO/staden’ $

All test successfull? Really?

External programs (perl-live’s make test actually) die with:

t/abi……………………The extension ‘Bio::SeqIO::staden::read’ is not properly installed in path: ‘/usr/lib/perl5/site_perl/5.8.8’

If this is a CPAN/distributed module, you may need to reinstall it on your system.

To allow Inline to compile the module in a temporary cache, simply remove the Inline config option ‘VERSION=’ from the Bio::SeqIO::staden::read module.

at t/abi.t line 0 INIT failed—call queue aborted, line 1.

When I remove the VERSION line from read.pm theyr work but what does this mean?

perl-live$ make test PERL_DL_NONLAZY=1 /usr/bin/perl5.8.8 “-MExtUtils::Command::MM” “-e” “test_harness(0, ‘blib/lib’, ‘blib/arch’)” t/*.t t/AAChange……………….ok t/AAReverseMutate…………ok t/abi……………………ok t/ace……………………ok t/AlignIO………………..ok 71/156One or more DATA sections were not processed by Inline.

t/AlignIO………………..ok

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2006-08-30T12:15:04Z

The Inline::C installation seems to complain on tests semi-randomly but it is harmless (yet annoying). Yeah I noticed it too on Mac OS X and I’m not sure yet what causes it. The abi.t and ztr.t bioperl-live tests (and a few others) are the key tests to check for sequence format parsing. and it looks like they pass.

The bioperl-ext tests, though noisy, also seem to pass for the SeqIO extensions, but there was some problem with the Bio::Ext::Align ones. Not sure what’s going on there but you can try installing it directly from the Bio/Ext/Align directory using ‘perl Makefile.PL; make; make test; make install’.

There was mention of trouble with the VERSION statement in Bio::SeqIO::staden::read from the mail lists. Inline usually picks that up and makes the correct recommendation, so it’s really a matter of reading the error messages. This is something we should add to the docs as well. Maybe we should just rewrite the docs completely, though. Seems they are a bit out-of-date.

So you know, you can also enter the individual directories to install subsets of the bioperl-ext extensions. Note there are Makefile.PL for each extension (Bio::SeqIO::staden::read, Bio::Ext::Align, etc).

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2006-08-30T13:52:14Z

As the major bugs here are addressed (use io_lib 1.8.11/12, change VERSION, os.h, etc), I am changing this to an enhancement request for a few things:

1) Better, up-to-date INSTALL docs for bioperl-ext. I’ll add what I can but I don’t use it that much. 2) If possible, updating bioperl-ext to use newer versions of io_lib. 3) Deal with Inline warnings that pop up during tests

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Bailey Original Date: 2007-07-11T16:04:01Z

I’ve been playing around with the Bio::SeqIO::staden::read code, and there are a couple of other issues with it. I’ve recently upgraded my curl libraries, which for some reason, seems to be messing inline compilation of the io_lib code. (compiling the raw c code by itself doesn’t add in any curl symbols, doing it via inline::C does). Several of the other problems mentioned in here seem to be also related to inline::C rather than the io_lib or the actuall code in the .pm file.

Based on this i propose that the staden::read package gets rewriten as a plain old h2xs style module, rather than a inline::C module. If people think that this would be a good idea, i’d be happy to do the work. This would also give an oppertunity to update the tests for bioperl-ext, to meet the standards now being used for the other components of bioperl, which i also don’t mind taking a look at. This would also give an opertunity to fold in the enhancements i’ve already made (see bug 2329), asuming others are happy for that.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2007-07-12T11:31:10Z

(In reply to comment #6)

I’ve been playing around with the Bio::SeqIO::staden::read code, and there are a couple of other issues with it. I’ve recently upgraded my curl libraries, which for some reason, seems to be messing inline compilation of the io_lib code. (compiling the raw c code by itself doesn’t add in any curl symbols, doing it via inline::C does). Several of the other problems mentioned in here seem to be also related to inline::C rather than the io_lib or the actuall code in the .pm file.

Based on this i propose that the staden::read package gets rewriten as a plain old h2xs style module, rather than a inline::C module. If people think that this would be a good idea, i’d be happy to do the work. This would also give an oppertunity to update the tests for bioperl-ext, to meet the standards now being used for the other components of bioperl, which i also don’t mind taking a look at. This would also give an opertunity to fold in the enhancements i’ve already made (see bug 2329), asuming others are happy for that.

That would be great! Feel free to hack away; bioperl-ext needs some work, so it’s an open field.

You can add code to this report as an attachment. If you plan on continuing maintenance of the ext XS modules we can discuss a dev account down the road once initial changes are checked in.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T05:41:18Z

Created an attachment (id=782) Changes Inline::C to just an xs module that installs into arch tree

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T05:42:59Z

(In reply to comment #8)

Created an attachment (id=782) [details] Changes Inline::C to just an xs module that installs into arch tree

I thought that would eat my comments… and it did. Attached is a version unchanged except … It doesn’t use Inline::C, it’s a standard xs module and the module gets installed into the architecture dependent tree ( I was having trouble installing the Inline version into a multi-architecture perl tree ). The Makefile.PL will use stadens io_lib-configure if it can find it, then try the environment variables, then try the wild guessing from previous Makefile.PL The include “read.h” is changed to include “io_lib/Read.h” as I didn’t see a version of staden that’s set up differently. This passes all tests with io_lib-1.11.0b3 and io_lib-1.8.11 ( after copying stadens config.h, os.h to stadens INSTALLDIR/include/io_lib for the older version ). But we haven’t put it into production yet so maybe it will have problems with the newer staden libraries.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T13:47:01Z

Created an attachment (id=786) cleaned up read.pm, removed unix paths from Makefile.PL

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2007-10-23T15:56:20Z

I managed to get everything passing in bioperl-live using Joel’s new XS version and the older io_lib 1.8; for some reason I can’t get newer io_lib versions to build on my MacBook.

I’ll go ahead and commit these changes (thanks tremendously, Joel!). I’ll leave this open as a reminder it needs further testing.

Note this dramatically affects bug 2329, which relies on Inline::C code, so if we plan on fixing that bug we’ll need to change it over as well. However I think moving to strict XS is for the best, as we seem to have problems with the Inline’d version.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T16:27:09Z

I looked at rolling the changes from bug 2329 into this, it was very simple but I don’t know what the test was testing, or what the new function was actually doing and don’t have time at the moment to figure it out.

below are patches to this with the read.pm read.xs changes from Chris Bailey

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T16:27:34Z

Created an attachment (id=790) patches read.pm code from bug 2329

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T16:29:04Z

Created an attachment (id=791) patches read.xs with code from bug 2329

oh, these aren’t really patches are they…

cjfields commented 8 years ago

Original Redmine Comment Author Name: Joel Martin Original Date: 2007-10-23T16:41:10Z

Created an attachment (id=792) patches read.xs with actual code from bug 2329…

last attachment was mistake ( was to keep the compiler quiet ), this has vanilla code from bug 2329.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Bailey Original Date: 2007-10-23T16:47:34Z

(In reply to comment #11)

I managed to get everything passing in bioperl-live using Joel’s new XS version and the older io_lib 1.8; for some reason I can’t get newer io_lib versions to build on my MacBook.

Nice one Joel. I must apologise for my lack of action on the action on rewriting the module myself. I’ve recently changed jobs and all our new servers are 64-bit where io_lib has real issues (may be related to the issues that chris is having. If i can get io_lib running on our servers i’ll be giving the new SeqIO::Staden package a try

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2007-10-23T17:54:41Z

There was no XSUB declared in the last patch for read_staden_graph. I added a quick one in (the args are exactly the same as for staden_read_trace(), so it is a copy-paste job). So far everything works, though I did get a warning on make:

read.xs: In function ‘staden_read_graph’: read.xs:200: warning: passing argument 2 of ‘Perl_newRV’ from incompatible pointer type read.xs:201: warning: passing argument 2 of ‘Perl_newRV’ from incompatible pointer type read.xs:202: warning: passing argument 2 of ‘Perl_newRV’ from incompatible pointer type read.xs:203: warning: passing argument 2 of ‘Perl_newRV’ from incompatible pointer type read.xs:204: warning: passing argument 2 of ‘Perl_newRV’ from incompatible pointer type Running Mkbootstrap for Bio::SeqIO::staden::read ()

Running the test script in bug 2329 works, though we might want to delve into the above warning some more.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2007-11-25T13:55:48Z

Committed Joel’s code. I’ll leave this open in order to trace the read.xs compilation warnings; they seem to be harmless for the time being.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Chris Fields Original Date: 2008-02-12T10:45:30Z

See bug 2069 and the reopened bug 2329 for comments on bioperl-ext.

cjfields commented 8 years ago

Original Redmine Comment Author Name: Martin Mokrejs Original Date: 2009-01-01T20:30:03Z

Will be bioperl-1.6 compatible with staden-1.7.0 (July 5, 2006) and io_lib-1.11.6 (December 10, 2008)? ;-)