Null pointer exception during import from searchgui

[mdondrup]

I am trying to import the results from a search into PeptideShaker 2.2.7 on Linux. Every time, I get a Null pointer exception at or after the 'Estimating PSM scores' task.

Wed Nov 30 15:17:43 CET 2022            - 83.6% peptide length less than 8 or greater than 30.
Wed Nov 30 15:17:43 CET 2022            - 16.3% peptide presenting high mass or isotopic deviation.
Wed Nov 30 15:17:43 CET 2022        Parsing Cement1.t.xml.gz.
Wed Nov 30 15:17:43 CET 2022        Checking spectra for Cement1.t.xml.gz.
Wed Nov 30 15:17:43 CET 2022        Importing PSMs from Cement1.t.xml.gz
Wed Nov 30 15:17:43 CET 2022        227 identified spectra (4.4%) did not present a valid peptide.
Wed Nov 30 15:17:43 CET 2022        965 of the best scoring peptides were excluded by the import filters:
Wed Nov 30 15:17:43 CET 2022            - 76.0% peptide length less than 8 or greater than 30.
Wed Nov 30 15:17:43 CET 2022            - 23.3% peptide presenting high mass or isotopic deviation.
Wed Nov 30 15:17:43 CET 2022        Parsing Cement1.tags.gz.
Wed Nov 30 15:17:44 CET 2022        Checking spectra for Cement1.tags.gz.
Wed Nov 30 15:17:44 CET 2022        Mapping tags to peptides.
Wed Nov 30 15:18:02 CET 2022        Importing PSMs from Cement1.tags.gz
Wed Nov 30 15:18:02 CET 2022        File import completed. 183309 first hits imported (833087 total) from 44552 spectra.
Wed Nov 30 15:18:02 CET 2022        [113191 first hits passed the initial filtering]
Wed Nov 30 15:18:02 CET 2022        Estimating PSM scores.
Wed Nov 30 15:18:02 CET 2022        An error occurred: null
Wed Nov 30 15:18:02 CET 2022        Please contact the developers (https://github.com/compomics/peptide-shaker/issues).

Wed Nov 30 15:18:02 CET 2022        Importing Data Canceled!

PeptiShaker.log

java.lang.NullPointerException
        at com.compomics.util.db.object.ObjectsDB.loadFromDB(ObjectsDB.java:342)
        at com.compomics.util.db.object.ObjectsDB.retrieveObject(ObjectsDB.java:477)
        at com.compomics.util.experiment.identification.Identification.retrieveObject(Identification.java:243)
        at com.compomics.util.experiment.identification.Identification.getSpectrumMatch(Identification.java:258)
        at eu.isas.peptideshaker.fileimport.PsmImportRunnable.importPsm(PsmImportRunnable.java:223)
        at eu.isas.peptideshaker.fileimport.PsmImportRunnable.run(PsmImportRunnable.java:178)
        at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.NullPointerException
        at com.compomics.util.db.object.ObjectsDB.getClassObjectIDs(ObjectsDB.java:264)
        at com.compomics.util.db.object.ObjectsDB.getClassObjectIDs(ObjectsDB.java:232)
        at com.compomics.util.db.object.ObjectsDB.getNumber(ObjectsDB.java:497)
        at com.compomics.util.experiment.identification.Identification.getSpectrumIdentificationSize(Identification.java:119)
        at eu.isas.peptideshaker.scoring.psm_scoring.PsmScorer.estimateIntermediateScores(PsmScorer.java:119)
        at eu.isas.peptideshaker.PeptideShaker.createProject(PeptideShaker.java:314)
        at eu.isas.peptideshaker.gui.NewDialog$20.run(NewDialog.java:748)
        at java.base/java.lang.Thread.run(Thread.java:834)

[hbarsnes]

I'm afraid this is a known backend issue (due to a third-party library) to which the only known solution is to increase the amount of memory provided to PeptideShaker.

Another option you can try is to use only one thread/core. This will however dramatically slow down the processing. But could be worth it if it allows you to complete the processing. You can change this via the Processing option at the bottom of the New Project dialog.

[mdondrup]

Ok, thank you. I had already set the memory limit to ~1TB, so I am trying to run the import on only one core. Is it possible to run the import process headless such that it can complete even if I log out? (I am now running the gui via ssh/Xorg)

[hbarsnes]

Both SearchGUI and PeptideShaker can be run from the command line. Please see https://github.com/compomics/searchgui/wiki/SearchCLI and https://github.com/compomics/peptide-shaker/wiki/PeptideShakerCLI.

[mdondrup]

Hi again, I have now run the import process from the command line with 1 thread. nohup java -Xms128M -Xmx1000000M -cp /Home/ii/michaeld/searchgui/PeptideShaker-2.2.17/PeptideShaker-2.2.17.jar eu.isas.peptideshaker.cmd.PeptideShakerCLI -reference Cement1 -identification_files searchguiresults/searchgui_out.zip -fasta_file Lsalmonis.pep.all.clean_concatenated_target_decoy.fasta -spectrum_files Raw/Cement1.mzml -threads 1

Unfortunately, it still fails with a nullpointer exception. Is there anything else I could do? Or if I will be unable to load my data, could the results from SearchGUI be inspected with a different program?

java.lang.NullPointerException
    at com.compomics.util.db.object.ObjectsDB.loadFromDB(ObjectsDB.java:342)
    at com.compomics.util.db.object.ObjectsDB.retrieveObject(ObjectsDB.java:477)
    at com.compomics.util.experiment.identification.Identification.retrieveObject(Identification.java:243)
    at com.compomics.util.experiment.identification.Identification.getSpectrumMatch(Identification.java:258)
    at eu.isas.peptideshaker.fileimport.PsmImportRunnable.importPsm(PsmImportRunnable.java:223)
    at eu.isas.peptideshaker.fileimport.PsmImportRunnable.run(PsmImportRunnable.java:178)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.NullPointerException
    at com.compomics.util.db.object.ObjectsDB.loadFromDB(ObjectsDB.java:342)
    at com.compomics.util.db.object.ObjectsDB.retrieveObject(ObjectsDB.java:477)
    at com.compomics.util.experiment.identification.Identification.retrieveObject(Identification.java:243)
    at com.compomics.util.experiment.identification.Identification.getSpectrumMatch(Identification.java:258)
    at eu.isas.peptideshaker.fileimport.PsmImportRunnable.importPsm(PsmImportRunnable.java:223)
    at eu.isas.peptideshaker.fileimport.PsmImportRunnable.run(PsmImportRunnable.java:178)
    at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
    at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
    at java.base/java.lang.Thread.run(Thread.java:834)
java.lang.NullPointerException
    at com.compomics.util.db.object.ObjectsDB.getClassObjectIDs(ObjectsDB.java:264)
    at com.compomics.util.db.object.ObjectsDB.getClassObjectIDs(ObjectsDB.java:232)
    at com.compomics.util.db.object.ObjectsDB.getNumber(ObjectsDB.java:497)
    at com.compomics.util.experiment.identification.Identification.getSpectrumIdentificationSize(Identification.java:119)
    at eu.isas.peptideshaker.scoring.psm_scoring.PsmScorer.estimateIntermediateScores(PsmScorer.java:119)
    at eu.isas.peptideshaker.PeptideShaker.createProject(PeptideShaker.java:314)
    at eu.isas.peptideshaker.cmd.PeptideShakerCLI.createProject(PeptideShakerCLI.java:1263)
    at eu.isas.peptideshaker.cmd.PeptideShakerCLI.call(PeptideShakerCLI.java:243)
    at eu.isas.peptideshaker.cmd.PeptideShakerCLI.main(PeptideShakerCLI.java:1467)

[hbarsnes]

Unfortunately, it still fails with a nullpointer exception. Is there anything else I could do?

You could perhaps try only searching with a single search eninge?

Or if I will be unable to load my data, could the results from SearchGUI be inspected with a different program?

Not directly no, but there are viewers for some of the output formats. For example, it should be possible to look at the mzIdentML outputs in, for example, PRIDE Inspector.

I will try to get the time to look at the null pointer next week. Perhaps I can simply replace the external library with standard Java code to fix the problem. The issue is that we've never had a consistent way of testing this before, but perhaps with your current setup that would be possible. I'll keep you posted.

[mdondrup]

On Dec 1, 2022, at 4:38 PM, Harald Barsnes @.***> wrote:

Unfortunately, it still fails with a nullpointer exception. Is there anything else I could do?

You could perhaps try only searching with a single search eninge?

I have tried with MS-Amanda and OMSSA individually, both imports fail with the same Null pointer exception.

Or if I will be unable to load my data, could the results from SearchGUI be inspected with a different program?

Not directly no, but there are viewers for some of the output formats. For example, it should be possible to look at the mzIdentML outputs in, for example, PRIDE Inspector.

I will try to get the time to look at the null pointer next week. Perhaps I can simply replace the external library with standard Java code to fix the problem. The issue is that we've never had a consistent way of testing this before, but perhaps with your current setup that would be possible. I'll keep you posted.

I will gladly help with testing, please let me know when I can check out a new version. Possibly, I could also help with or at least take a look at the code. Which library is the culprit?

Just to make sure it has nothing to do with my database file, now the Fasta file looks like this:

generic|EMLSAP00000012095|Protein enhancer of sevenless 2B MEAVAKHDFSATAEDELSFRKGQILKVLNIEDDKNWYRAESGGKEGLIPSNYIDMKPHDW YYSRMTRATAEKLLSNKHEGAFVIRVSESSPGDFSLSVKCGDGVQHFKVLRDGQGKFFLW VVKFNSLNELVEYHHSASVSRSQDIKLKEIICEEFLVQALYDFIPQEVGELEFRRGDVIN VTDKADRHWWAGELGNKRGYFPAKYRFEVFIKYKYLGIMFIHIFEI generic|EMLSAP00000012089|Hypothetical protein MKHYQRNMIKGNLREPSFPPDSARYQFLSSSIFSTFKICPFRKLNSSSAVAEKSCLATAS ISDQQNSITPNQELELDGYQKLFPIGIRSDGLLEPANTLSPLPLSKNPDSVRVCADWIKS IETLKTITRRMSYLHKTSVSCGEIGFYS generic|EMLSAP00000012090|hypothetical protein [Daphnia magna] MKKNLGTLSIQKTHGGIESLSNNSPERESKEVILFDRIASAIFGKEDGLAIGRDKNYLAK KKFPSKTSMKQESTTFEGISNSNEQQYPVSGTAKDIVQCEIEMMRVRVIFDRPFRGLIFS

...

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.

[hbarsnes]

I have tried with MS-Amanda and OMSSA individually, both imports fail with the same Null pointer exception.

Ok, thanks for testing. I don't think it is search engine specific. More related to the amount of data. Hence the idea was that using fewer search engines would help. But I guess not...

I will gladly help with testing, please let me know when I can check out a new version. Possibly, I could also help with or at least take a look at the code. Which library is the culprit?

As mentioned above, the issue only occurs when there is lots of data (well, at least relative to the amount of memory provided) and we have to start putting our objects into the underlying SQLite database.

However, I just looked closer at your stack trace and it seems to be difference from the ones we usually see when experiencing similar issues. In your case the null pointer happens at the following two lines:

PreparedStatement pstmt = connection.prepareStatement(sqlQuery);

PreparedStatement pstmt = connection.prepareStatement("SELECT class, data FROM data WHERE id = ?;");

Both seemingly indicating that the connection itself is null. Meaning that it has most likely not been established in the first place:

connection = DriverManager.getConnection("jdbc:sqlite:" + dbFile.getAbsolutePath());

But if this fails an exception should have been thrown...

You can find the rest of the class here: https://github.com/compomics/compomics-utilities/blob/master/src/main/java/com/compomics/util/db/object/ObjectsDB.java. See the establishConnection method on line 853. And the method where it is supposed to be called at line 87.

Are there any other exceptions being thrown before the ones indicated above?

Just to make sure it has nothing to do with my database file, now the Fasta file looks like this:

Looks good. I don't think there are any issues there.

[mdondrup]

You are right! I should have looked right at the beginning of the file. So the sqllite shared object cannot be loaded. Sorry for not seeing this earlier.

Sun Nov 27 14:31:05 CET 2022: PeptideShaker version 2.2.17. Memory given to the Java virtual machine: 1048576000000. Total amount of memory in the Java virtual machine: 134217728. Free memory: 116338024. Java version: 11.0.8-internal. Failed to load native library:sqlite-3.36.0.3-9348f2f1-2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so. osinfo: Linux/x86_64 java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.36.0.3-9348f2f1-2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so: /tmp/sqlite-3.36.0.3-9348f2f1 -2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted java.sql.SQLException: Error opening connection at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:244) at org.sqlite.SQLiteConnection.(SQLiteConnection.java:61) at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:28) at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:21) at org.sqlite.JDBC.createConnection(JDBC.java:115) at org.sqlite.JDBC.connect(JDBC.java:90) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:251) at com.compomics.util.db.object.ObjectsDB.establishConnection(ObjectsDB.java:866) at com.compomics.util.db.object.ObjectsDB.(ObjectsDB.java:115) at com.compomics.util.db.object.ObjectsDB.(ObjectsDB.java:76) at eu.isas.peptideshaker.PeptideShaker.importFiles(PeptideShaker.java:205) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.createProject(PeptideShakerCLI.java:1251) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.call(PeptideShakerCLI.java:243) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.main(PeptideShakerCLI.java:1467) Caused by: java.lang.Exception: No native library found for os.name=Linux, os.arch=x86_64, paths=[/org/sqlite/native/Linux/x86_64:/usr/lo cal/lib:/Home/ii/michaeld/lib:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib] at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:389) at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:68) at org.sqlite.core.NativeDB.load(NativeDB.java:63) at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:240) ... 14 more

On Dec 2, 2022, at 12:41 PM, Harald Barsnes @.***> wrote:

Both seemingly indicating that the connection itself is null. Meaning that it has most likely not been established in the first place:

connection = DriverManager.getConnection("jdbc:sqlite:" + dbFile.getAbsolutePath());

But if this fails an exception should have been thrown...

You can find the rest of the class here: https://github.com/compomics/compomics-utilities/blob/master/src/main/java/com/compomics/util/db/object/ObjectsDB.java. See the establishConnection method on line 853. And the method where it is supposed to be called at line 87.

Are there any other exceptions being thrown before the ones indicated above?

Just to make sure it has nothing to do with my database file, now the Fasta file looks like this:

Looks good. I don't think there are any issues there.

—

[mdondrup]

Also, the reason for that is - I think - that on kjempefuru (CentOS 7) /tmp is mounted noexec, so no I am changing the temp dir using the following options and it works:

java -Djava.io.tmpdir=pwd/tmp

Sorry for the fuzz. It might be that most users running PeptideShaker on CentOS should consider this.

Michael

On Dec 2, 2022, at 1:37 PM, Michael @.***> wrote:

You are right! I should have looked right at the beginning of the file. So the sqllite shared object cannot be loaded. Sorry for not seeing this earlier.

Sun Nov 27 14:31:05 CET 2022: PeptideShaker version 2.2.17. Memory given to the Java virtual machine: 1048576000000. Total amount of memory in the Java virtual machine: 134217728. Free memory: 116338024. Java version: 11.0.8-internal. Failed to load native library:sqlite-3.36.0.3-9348f2f1-2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so. osinfo: Linux/x86_64 java.lang.UnsatisfiedLinkError: /tmp/sqlite-3.36.0.3-9348f2f1-2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so: /tmp/sqlite-3.36.0.3-9348f2f1 -2025-4cc9-80be-4195b6a4faaf-libsqlitejdbc.so: failed to map segment from shared object: Operation not permitted java.sql.SQLException: Error opening connection at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:244) at org.sqlite.SQLiteConnection.(SQLiteConnection.java:61) at org.sqlite.jdbc3.JDBC3Connection.(JDBC3Connection.java:28) at org.sqlite.jdbc4.JDBC4Connection.(JDBC4Connection.java:21) at org.sqlite.JDBC.createConnection(JDBC.java:115) at org.sqlite.JDBC.connect(JDBC.java:90) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) at java.sql/java.sql.DriverManager.getConnection(DriverManager.java:251) at com.compomics.util.db.object.ObjectsDB.establishConnection(ObjectsDB.java:866) at com.compomics.util.db.object.ObjectsDB.(ObjectsDB.java:115) at com.compomics.util.db.object.ObjectsDB.(ObjectsDB.java:76) at eu.isas.peptideshaker.PeptideShaker.importFiles(PeptideShaker.java:205) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.createProject(PeptideShakerCLI.java:1251) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.call(PeptideShakerCLI.java:243) at eu.isas.peptideshaker.cmd.PeptideShakerCLI.main(PeptideShakerCLI.java:1467) Caused by: java.lang.Exception: No native library found for os.name=Linux, os.arch=x86_64, paths=[/org/sqlite/native/Linux/x86_64:/usr/lo cal/lib:/Home/ii/michaeld/lib:/usr/java/packages/lib:/usr/lib64:/lib64:/lib:/usr/lib] at org.sqlite.SQLiteJDBCLoader.loadSQLiteNativeLibrary(SQLiteJDBCLoader.java:389) at org.sqlite.SQLiteJDBCLoader.initialize(SQLiteJDBCLoader.java:68) at org.sqlite.core.NativeDB.load(NativeDB.java:63) at org.sqlite.SQLiteConnection.open(SQLiteConnection.java:240) ... 14 more

On Dec 2, 2022, at 12:41 PM, Harald Barsnes @.***> wrote:

Both seemingly indicating that the connection itself is null. Meaning that it has most likely not been established in the first place:

connection = DriverManager.getConnection("jdbc:sqlite:" + dbFile.getAbsolutePath());

But if this fails an exception should have been thrown...

You can find the rest of the class here: https://github.com/compomics/compomics-utilities/blob/master/src/main/java/com/compomics/util/db/object/ObjectsDB.java. See the establishConnection method on line 853. And the method where it is supposed to be called at line 87.

Are there any other exceptions being thrown before the ones indicated above?

Just to make sure it has nothing to do with my database file, now the Fasta file looks like this:

Looks good. I don't think there are any issues there.

—

[hbarsnes]

Aha, good that you were able to locate the problem. And thanks for sharing the solution! I guess I can then close the issue?

[mdondrup]

Yes please!

On Dec 2, 2022, at 2:10 PM, Harald Barsnes @.***> wrote:

Aha, good that you were able to locate the problem. And thanks for sharing the solution! I guess I can then close the issue?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.