No such table: PasefFrameMsMsInfo when .d -> mzML

[daichengxin]

Hi @mwang87 @gtluu. The error is reported when I run the follow command. Looking forward to your help. Thanks a lot in advance

Command:

python3 /app/bin/run.py \
  --input 20210909_PRO2_LS_16_50-0060_S09_0.635_B1_GB1_1_795.d \
  --outdir spectra \
  --mode centroid \
  --compression zlib \
   \
  --exclude_mobility \
  --encoding 64 \
  --maldi_output_file combined \
  --imzml_mode processed \
  --lcms_backend timsconvert \
  --chunk_size 10 \
  --verbose \
  --start_frame -1 \
  --end_frame -1 \
  --precision 10.0 \
  --ms1_threshold 100 \
  --ms2_threshold 10 \
  --ms2_nlargest -1

Error Info:

  2022-06-28 07:04:56.802708 [tid=0xa2d37740] [WARN ] bdal.io.tims.TdfReaderImpl: Requested recalibration file "20210909_PRO2_LS_16_50-0060_S09_0.635_B1_GB1_1_795.d/calibration.sqlite" does not exists. Fallback to instrument calibration from tdf.
  Traceback (most recent call last):
    File "/usr/local/lib/python3.7/site-packages/pandas/io/sql.py", line 1586, in execute
      cur.execute(*args, **kwargs)
  sqlite3.OperationalError: no such table: PasefFrameMsMsInfo

  The above exception was the direct cause of the following exception:

  Traceback (most recent call last):
    File "/app/bin/run.py", line 138, in <module>
      run_timsconvert(args)
    File "/app/bin/run.py", line 62, in run_timsconvert
      data = tdf_data(infile, tdf_sdk_dll)
    File "/app/bin/timsconvert/classes.py", line 299, in __init__
      self.get_pasefframemsmsinfo_table()
    File "/app/bin/timsconvert/classes.py", line 535, in get_pasefframemsmsinfo_table
      self.pasefframemsmsinfo = pd.read_sql_query(pasefframemsmsinfo_query, self.conn)
    File "/usr/local/lib/python3.7/site-packages/pandas/io/sql.py", line 332, in read_sql_query
      chunksize=chunksize,
    File "/usr/local/lib/python3.7/site-packages/pandas/io/sql.py", line 1633, in read_query
      cursor = self.execute(*args)
    File "/usr/local/lib/python3.7/site-packages/pandas/io/sql.py", line 1598, in execute
      raise ex from exc
  pandas.io.sql.DatabaseError: Execution failed on sql 'SELECT * FROM PasefFrameMsMsInfo': no such table: PasefFrameMsMsInfo

The data is here: https://www.ebi.ac.uk/pride/archive/projects/PXD030130.

[mwang87]

Thanks @daichengxin, will definitely take a look!

[jflucier]

I have the exact same problem.

tks

[jflucier]

Dont know if this could help but The table in tdf file seem to be named PrmFrameMsMsInfo:

|12:39:12|jflucier@ip29:[test]> sqlite3 "$f"/analysis.tdf
-- Loading resources from /home/jflucier/.sqliterc
SQLite version 3.31.1 2020-01-27 19:55:54
Enter ".help" for usage hints.
sqlite> .schema
CREATE TABLE GlobalMetadata (
    Key TEXT PRIMARY KEY,
    Value TEXT
    );
CREATE TABLE PropertyDefinitions (
    Id INTEGER PRIMARY KEY, 
    PermanentName TEXT NOT NULL, 
    Type INTEGER NOT NULL,
    DisplayGroupName TEXT NOT NULL, 
    DisplayName TEXT NOT NULL,
    DisplayValueText TEXT NOT NULL,
    DisplayFormat TEXT NOT NULL,
    DisplayDimension TEXT NOT NULL,
    Description TEXT NOT NULL
);
CREATE UNIQUE INDEX PropertyDefinitionsIndex ON PropertyDefinitions (PermanentName);
CREATE TABLE PropertyGroups (
    Id INTEGER PRIMARY KEY
) WITHOUT ROWID;
CREATE TABLE GroupProperties (
    PropertyGroup INTEGER NOT NULL,
    Property INTEGER NOT NULL,
    Value NOT NULL,
    PRIMARY KEY (PropertyGroup, Property),
    FOREIGN KEY (PropertyGroup) REFERENCES PropertyGroups (Id),
    FOREIGN KEY (Property) REFERENCES PropertyDefinitions (Id)
) WITHOUT ROWID;
CREATE TABLE FrameProperties (
    Frame INTEGER NOT NULL,
    Property INTEGER NOT NULL,
    Value NOT NULL,
    PRIMARY KEY (Frame, Property),
    FOREIGN KEY (Frame) REFERENCES Frames (Id)
    FOREIGN KEY (Property) REFERENCES PropertyDefinitions (Id)
) WITHOUT ROWID;
CREATE TABLE FrameMsMsInfo (
    Frame INTEGER PRIMARY KEY,
    Parent INTEGER,
    TriggerMass REAL NOT NULL,
    IsolationWidth REAL NOT NULL,
    PrecursorCharge INTEGER,
    CollisionEnergy REAL NOT NULL,
    FOREIGN KEY (Frame) REFERENCES Frames (Id)
);
CREATE TABLE CalibrationInfo (
    KeyPolarity CHAR(1) CHECK (KeyPolarity IN ('+', '-')),
    KeyName TEXT,
    Value TEXT,
    PRIMARY KEY (KeyPolarity, KeyName)
    );
CREATE TABLE Segments (
    Id INTEGER PRIMARY KEY,
    FirstFrame INTEGER NOT NULL,
    LastFrame INTEGER NOT NULL,
    IsCalibrationSegment BOOLEAN NOT NULL,
    FOREIGN KEY (FirstFrame) REFERENCES Frames (Id),
    FOREIGN KEY (LastFrame) REFERENCES Frames (Id)
);
CREATE VIEW Properties AS
    SELECT s.Id Frame, pd.Id Property, COALESCE(fp.Value, gp.Value) Value
    FROM Frames s
    JOIN PropertyDefinitions pd
    LEFT JOIN GroupProperties gp ON gp.PropertyGroup=s.PropertyGroup AND gp.Property=pd.Id
    LEFT JOIN FrameProperties fp ON fp.Frame=s.Id AND fp.Property=pd.Id
/* Properties(Frame,Property,Value) */;
CREATE TABLE ErrorLog (
    Frame INTEGER NOT NULL,
    Scan INTEGER,
    Message TEXT NOT NULL
);
CREATE TABLE MzCalibration (
    Id INTEGER PRIMARY KEY,
    ModelType INTEGER NOT NULL,
    DigitizerTimebase REAL NOT NULL,
    DigitizerDelay REAL NOT NULL,
    T1 REAL NOT NULL,
    T2 REAL NOT NULL,
    dC1 REAL NOT NULL,
    dC2 REAL NOT NULL,
    C0
, C1, C2, C3, C4);
CREATE TABLE Frames (
    Id INTEGER PRIMARY KEY,
    Time REAL NOT NULL,
    Polarity CHAR(1) CHECK (Polarity IN ('+', '-')) NOT NULL,
    ScanMode INTEGER NOT NULL,
    MsMsType INTEGER NOT NULL,
    TimsId INTEGER,
    MaxIntensity INTEGER NOT NULL,
    SummedIntensities INTEGER NOT NULL,
    NumScans INTEGER NOT NULL,
    NumPeaks INTEGER NOT NULL,
    MzCalibration INTEGER NOT NULL,
    T1 REAL NOT NULL,
    T2 REAL NOT NULL,
    TimsCalibration INTEGER NOT NULL,
    PropertyGroup INTEGER,
    AccumulationTime REAL NOT NULL,
    RampTime REAL NOT NULL,
    Pressure REAL,
    FOREIGN KEY (MzCalibration) REFERENCES MzCalibration (Id),
    FOREIGN KEY (TimsCalibration) REFERENCES TimsCalibration (Id),
    FOREIGN KEY (PropertyGroup) REFERENCES PropertyGroups (Id)
);
CREATE UNIQUE INDEX FramesTimeIndex ON Frames (Time);
CREATE TABLE TimsCalibration (
    Id INTEGER PRIMARY KEY,
    ModelType INTEGER NOT NULL,
    C0
, C1, C2, C3, C4, C5, C6, C7, C8, C9);
CREATE TABLE DiaFrameMsMsWindowGroups (
   Id INTEGER PRIMARY KEY);
CREATE TABLE DiaFrameMsMsWindows (
   WindowGroup INTEGER NOT NULL,
   ScanNumBegin INTEGER NOT NULL,
   ScanNumEnd INTEGER NOT NULL,
   IsolationMz REAL NOT NULL,
   IsolationWidth REAL NOT NULL,
   CollisionEnergy REAL NOT NULL,
   PRIMARY KEY(WindowGroup, ScanNumBegin),
   FOREIGN KEY (WindowGroup) REFERENCES DiaFrameMsMsWindowGroups (Id)
) WITHOUT ROWID;
CREATE TABLE DiaFrameMsMsInfo (
   Frame INTEGER PRIMARY KEY,
   WindowGroup INTEGER NOT NULL,
   FOREIGN KEY (Frame) REFERENCES Frames (Id),
   FOREIGN KEY (WindowGroup) REFERENCES DiaFrameMsMsWindowGroups (Id)
);
CREATE TABLE PrmTargets (
Id INTEGER PRIMARY KEY,
ExternalId TEXT CHECK(ExternalId IS NULL OR LENGTH(ExternalId) > 0) UNIQUE,
Time REAL NOT NULL,
OneOverK0 REAL NOT NULL,
MonoisotopicMz REAL NOT NULL,
Charge INTEGER NOT NULL,
Description TEXT NOT NULL
);
CREATE TABLE PrmFrameMsMsInfo (
Frame INTEGER NOT NULL,
ScanNumBegin INTEGER NOT NULL,
ScanNumEnd INTEGER NOT NULL,
IsolationMz REAL NOT NULL,
IsolationWidth REAL NOT NULL,
CollisionEnergy REAL NOT NULL,
Target INTEGER NOT NULL,
PRIMARY KEY (Frame, ScanNumBegin),
FOREIGN KEY (Frame) REFERENCES Frames(Id),
FOREIGN KEY (Target) REFERENCES PrmTargets(Id)
) WITHOUT ROWID;
CREATE TABLE PrmFrameMeasurementMode (
Frame INTEGER PRIMARY KEY,
MeasurementModeId TEXT,
FOREIGN KEY (Frame) REFERENCES Frames(Id)
);
CREATE INDEX PrmFrameMsMsInfoTargetIndex ON PrmFrameMsMsInfo (Target);
CREATE INDEX PrmFrameMeasurementModeIndex ON PrmFrameMeasurementMode (MeasurementModeId);

[jflucier]

I looked at code and it seems that this software supports PASEF acquisition not the diaPASEF acquisition.

[gtluu]

@daichengxin @jflucier Apologies for such a delayed response. Previously, diaPASEF data conversion was only supported using the option --lcms_backend tdf2mzml, and this implementation had not incorporated the TIMS data into the resulting mzML file. I am currently working on changing that using data that @daichengxin shared in https://www.ebi.ac.uk/pride/archive/projects/PXD030130. If you have any other method or raw data files that can be shared (particularly using methods that generate data in the PrmFrameMsMsInfo table), I can work on making sure that data is supported.

[gtluu]

closing this as it should be fixed with #44

support for these formats has been added