Use of uninitialized value $esearch_result in pattern match (m//)

[ckeuschn]

Hi,

I successfully installed SACCHARIS and additional software. I tried to follow the example from your publication to test the pipeline and got an error message after the cazy_extract.pl script.

user$ cazy_extract.pl -f GH42 -g characterized Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 462. Use of uninitialized value $count in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 465. Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 469. Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 472. Use of uninitialized value $key in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 476. Use of uninitialized value $web in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 476. Use of uninitialized value $efetch_result in substitution (s///) at /usr/local/Cellar/SACCHARIS/cazyextract.pl line 482. Use of uninitialized value $ in split at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 165.

apparently $esearch_result is not assigned with a value (if I understand correctly), so does that mean the URL submission to the Entrez API fails to get any sequences? Maybe you could help me to solve that problem?

Thanks!

[DallasThomas]

It does look like an error with the query to NCBI. I will see if I can replicate it. Unfortunately I am out of office until the 26th

Dallas

Original Message From: ckeuschn Sent: Wednesday, March 14, 2018 8:35 AM To: DallasThomas/SACCHARIS Reply To: DallasThomas/SACCHARIS Cc: Subscribed Subject: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

Hi,

I successfully installed SACCHARIS and additional software. I tried to follow the example from your publication to test the pipeline and got an error message after the cazy_extract.pl script.

user$ cazy_extract.pl -f GH42 -g characterized Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 462. Use of uninitialized value $count in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 465. Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 469. Use of uninitialized value $esearch_result in pattern match (m//) at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 472. Use of uninitialized value $key in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 476. Use of uninitialized value $web in concatenation (.) or string at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 476. Use of uninitialized value $efetch_result in substitution (s///) at /usr/local/Cellar/SACCHARIS/cazyextract.pl line 482. Use of uninitialized value $ in split at /usr/local/Cellar/SACCHARIS/cazy_extract.pl line 165.

apparently $esearch_result is not assigned with a value (if I understand correctly), so does that mean the URL submission to the Entrez API fails to get any sequences? Maybe you could help me to solve that problem?

Thanks!

-- You are receiving this because you are subscribed to this thread. Reply to this email directly or view it on GitHub: https://github.com/DallasThomas/SACCHARIS/issues/1

[ckeuschn]

Thanks for your help

I was thinking that it might also be that the $id_list object has no assigned value due to some problem during pulling ids from the cazy database earlier in the code

Christoph

[DallasThomas]

This has happened before when there was a change on cazy's website. It is possible that happened as well. One thing to check is the URL created in get cazy script matches the URL of the GH family page you are testing.

From: ckeuschn Sent: Thursday, March 15, 2018 1:05 AM To: DallasThomas/SACCHARIS Reply To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

Thanks for your help

I was thinking that it might also be that the $id_list object has no assigned value due to some problem during pulling ids from the cazy database earlier in the code

Christoph

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-373292960, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADqs45QEB-hbpYprwwQ2Ax5s4F72FNIUks5teiC9gaJpZM4SqqOx.

[ckeuschn]

The URL which is created seems to be fine, at least the code still matches the structure of the URL of cazy.org

I attached a file with the output of the Saccharis.pl script, maybe this helps to clarify things.

Thanks again Saccharis_error.txt

[DallasThomas]

I have an idea what it might be unfortunately I will not be able to confirm this until I am back next Monday morning. I apologize for this issue and delay.

Dallas

From: ckeuschn Sent: Monday, March 19, 2018 5:16 AM To: DallasThomas/SACCHARIS Reply To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

The URL which is created seems to be fine, at least the code still matches the structure of the URL of cazy.org

I attached a file with the output of the Saccharis.pl script, maybe this helps to clarify things.

Thanks again Saccharis_error.txthttps://github.com/DallasThomas/SACCHARIS/files/1824882/Saccharis_error.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-374192091, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADqs40JW83SylciB4tSYuETOxK59zY2-ks5tf6FygaJpZM4SqqOx.

[DallasThomas]

Hello Christoph,

Sorry for the delay with me being gone. I am back now and I am looking into the problem.

To help me further can you confirm that you modified line 236 of the TagParser.pm from the install of HTML::TagParser with the following:

--> Old Line: bless $self, $package; --> New Line: bless $self, ref($package) || $package;

Also with your call to Saccharis – when you call with more than one group please confirm they are enclosed in single quotes

                    Eg.                               Saccharis.pl –f GH42 –g ‘characterized, all’

Just trying to cross off known issues.

Thanks Dallas

From: ckeuschn [mailto:notifications@github.com] Sent: March-19-18 6:16 AM To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

The URL which is created seems to be fine, at least the code still matches the structure of the URL of cazy.org

I attached a file with the output of the Saccharis.pl script, maybe this helps to clarify things.

Thanks again Saccharis_error.txthttps://github.com/DallasThomas/SACCHARIS/files/1824882/Saccharis_error.txt

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-374192091, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADqs40JW83SylciB4tSYuETOxK59zY2-ks5tf6FygaJpZM4SqqOx.

[ckeuschn]

Hi Thomas,

Yes I changed line 236 in the TagParser.pm as you said and I get the same error messages even when I just enter one attribute in the -g option.

I will be traveling the next two weeks, but I will try to answer as soon as possible during that time.... Thanks Ch

[DallasThomas]

Hey,

Thanks for the information. Was just wondering as I am having no difficulties running that same call on my system, so am trying to figure out what is different between my set-up and yours.

Could you send me a bit more information on your system, Linux distribution and version, version of perl along with the versions of TagParser and Simple. Want to try and create the same environment on a virtual box to test this out.

Also when you just try and run cazy_extract.pl without any arguments, is it throwing any errors at all?

Thanks Dallas

From: ckeuschn [mailto:notifications@github.com] Sent: March-26-18 12:32 PM To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

Hi Thomas,

Yes I changed line 236 in the TagParser.pm as you said and I get the same error messages even when I just enter one attribute in the -g option.

I will be traveling the next two weeks, but I will try to answer as soon as possible during that time.... Thanks Ch

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-376266546, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADqs44fpD2wQhAqLo1r74WASgN0IQZnlks5tiTQTgaJpZM4SqqOx.

[DallasThomas]

Just in case, you can get perl module versions via the following:

perl –MHTML::TagParser –le ‘print $HTML::TagParser::VERSION’

From: ckeuschn [mailto:notifications@github.com] Sent: March-26-18 12:32 PM To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

Hi Thomas,

Yes I changed line 236 in the TagParser.pm as you said and I get the same error messages even when I just enter one attribute in the -g option.

I will be traveling the next two weeks, but I will try to answer as soon as possible during that time.... Thanks Ch

— You are receiving this because you commented. Reply to this email directly, view it on GitHubhttps://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-376266546, or mute the threadhttps://github.com/notifications/unsubscribe-auth/ADqs44fpD2wQhAqLo1r74WASgN0IQZnlks5tiTQTgaJpZM4SqqOx.

[cmorganl]

Hi Dallas and Christoph,

I hope you don't mind me piggy-backing on this issue but I'm encountering a similar error probably stemming from the same problem. While I wasn't able to reproduce Christoph's error with perl cazy_extract.pl -f GH42 -g characterized, I was able to get similar errors with: perl cazy_extract.pl --family GH109 --group all. I'm running RedHat with perl v5.16.3 built for x86_64-linux-thread-multi and HTML::TagParser 0.20. I hope this helps.

Additionally, the command perl cazy_extract.pl --family GH3 --group all resulted in:

Use of uninitialized value $num in multiplication (*) at cazy_extract.pl line 259.

and an empty FASTA file. Any help would be appreciated!

Thanks, Connor

[DallasThomas]

Hello Connor,

Thanks for the input. I have uploaded a new version of cazy_extract.pl that fixes the $num error. This is common and has to do with style changes that happen on the cazy website every once and a while.

As for the error with GH109 I am still tracking that one. NCBI esearch generally allows a query of 500 sequences. I have a feeling this has changed and would totally explain the errors.

Christoph - are you getting the errors with GH42 characterized as well as all when run separately?

If what I suspect is happening based on an update from NCBI and the use of their esearch tools. I will be in contact with them to determine a work around. I will keep everyone posted.

I apologize for this.

Dallas

[cmorganl]

Hi Dallas

Thanks for the quick push! It looks like it solved the problem when searching for all sequences, but when I choose a taxonomic group the $num error persists. Here's the command I used:

perl cazy_extract.pl --family GH3 --group bacteria

Thanks! Connor

[DallasThomas]

Hello Connor,

Similar issue to before - updates have been made to cazy_extract.pl - please keep me informed if you find more of these.

I did email NCBI and am just waiting to hear back from them.

[ckeuschn]

Hey Thomas,

cazy_extract.pl without arguments gives no errors. My system is actually Mac OS X El Capitan and I run perl v5.26.1. and HTML::TagParser 0.20 and LWP::Simple 6.33. I might try to run the scripts also from a virtual Linux distribution when I am back to see if that does the trick. Thanks Ch

[DallasThomas]

That is one platform I have not tested yet. I will test it on my Mac at home as well.

Dallas

Original Message From: ckeuschn Sent: Tuesday, March 27, 2018 12:51 AM To: DallasThomas/SACCHARIS Reply To: DallasThomas/SACCHARIS Cc: Thomas, Dallas; Comment Subject: Re: [DallasThomas/SACCHARIS] Use of uninitialized value $esearch_result in pattern match (m//) (#1)

Hey Thomas,

cazy_extract.pl without arguments gives no errors. My system is actually Mac OS X El Capitan and I run perl v5.26.1. and HTML::TagParser 0.20 and LWP::Simple 6.33. I might try to run the scripts also from a virtual Linux distribution when I am back to see if that does the trick. Thanks Ch

-- You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/DallasThomas/SACCHARIS/issues/1#issuecomment-376416454

[DallasThomas]

Connor,

I have just uploaded a new version of cazy_extract.pl that deals more efficiently with the previous $num errors and should cover other potential cases.

Dallas

[cmorganl]

Hi Dallas,

I don't think this new version is working properly. It printed some variables then exited.

$ perl cazy_extract.pl --family GH3 --group characterized
$VAR1 = '100';
$VAR1 = 3;

Connor

[DallasThomas]

Ha, whoops I am an idiot - forgot to remove my debugging stuff. Just removed it - this version will work. In your version the problem lines are around line 255 - the two print Dumper lines and the exit. You can either download this version or just remove those lines from yours. Sorry about that.

[cmorganl]

Haha, no worries. I figured that was what happened... I'm also guilty of this.

Thanks again!

[DallasThomas]

Hey Christoph and Connor,

There was a change with NCBI and a further change coming from NCBI in the next couple of months. I have fixed the first issue in the current version so there should be no more issues as first reported by Christoph.

The second change will be implemented in a new version I should have up in a couple of days. NCBI is starting to experience increased draw on their servers, to prevent over subscription they are placing some checks in place to prevent this. The current implementation will be all users will need to submit a unique name for their program (the name is just for their tracking so can be whatever you want) and email address. Users will also have to create an account with NCBI which will give them a unique API key. Program names, email and API keys will be required for the e-utilities.

How this will effect the cazy_extract.pl script is in the new version once downloaded all users will have to first register Name and email with Cazy by sending an email to eutilities@ncbi.nlm.nih.gov - users will also have to register an account with NCBI and get their API key from the settings page. Program name registered should have no spaces. The user will then have to modify cazy_extract.pl - by first un-commenting the variable declarations for email, name and API key and then entering in single quotes those values.

This version as I said should be up by the end of the week. I will send another comment once it is available.

Thanks Dallas

[DallasThomas]

Sorry for the delay.

The modified version with the NCBI registration has been uploaded. Please see the section of the help page for NCBI registration. It outlines what is needed from NCBI and where cazy_extract.pl will need modified by the user.

Christoph - have you had any luck yet?

[cmorganl]

Hey Dallas,

After the fresh git pull, I completed the 3 steps to set up an account to access NCBI E-utils. After removing an exit call from line 41, everything works perfectly now.

Thanks a bunch! Connor

[DallasThomas]

Thanks for the heads up on the exit call. My bad.