search

MannLabs / alphatims

An open-source Python package for efficient accession and visualization of Bruker TimsTOF raw data from the Mann Labs at the Max Planck Institute of Biochemistry.
https://doi.org/10.1016/j.mcpro.2021.100149
Apache License 2.0
81 stars 25 forks source link

loading data that were collected without mobility acquisition #174

Closed hanghu1024 closed 2 years ago

hanghu1024 commented 2 years ago

Describe the bug I installed this python package with Pip in a windows machine. I am able to reproduce the data access and analysis following "tutorial.ipynb". I can also access my own ESI-TIMS-Q-TOF data. But we also have a need to process data without mobility acquisition (ESI-Q-TOF). A quick try using alphatims.bruker.TimsTOF() doesn't work, I have attached the error message below. Is there a solution to access this type of data using alphatims? Or could you recommend me with an alternative python tool? Many thanks!

Logs

file1 = 'C:\\Users\\Lab Admin\\Desktop\\1_2 LG mix_5uM_1.d'
D = alphatims.bruker.TimsTOF(file1)
---------------------------------------------------------------------------
OperationalError                          Traceback (most recent call last)
File C:\Program Files\Python39\lib\site-packages\pandas\io\sql.py:2020, in SQLiteDatabase.execute(self, *args, **kwargs)
   2019 try:
-> 2020     cur.execute(*args, **kwargs)
   2021     return cur

OperationalError: no such table: GlobalMetaData

The above exception was the direct cause of the following exception:

DatabaseError                             Traceback (most recent call last)
Input In [10], in <cell line: 1>()
----> 1 D = alphatims.bruker.TimsTOF(file1)

File ~\AppData\Roaming\Python\Python39\site-packages\alphatims\bruker.py:985, in TimsTOF.__init__(self, bruker_d_folder_name, mz_estimation_from_frame, mobility_estimation_from_frame, slice_as_dataframe, use_calibrated_mz_values_as_default, use_hdf_if_available, mmap_detector_events)
    973 if mmap_detector_events and hdf_file_exists:
    974     raise IOError(
    975         f"Can only use mmap from .hdf files. "
    976         f"Since {bruker_hdf_file_name} already exists, "
   (...)
    983         "or remove the existing .hdf file."
    984     )
--> 985 self._import_data_from_d_folder(
    986     bruker_d_folder_name,
    987     mz_estimation_from_frame,
    988     mobility_estimation_from_frame,
    989 )
    990 if mmap_detector_events:
    991     self._import_data_from_hdf_file(
    992         bruker_d_folder_name,
    993         mmap_detector_events,
    994     )

File ~\AppData\Roaming\Python\Python39\site-packages\alphatims\bruker.py:1043, in TimsTOF._import_data_from_d_folder(self, bruker_d_folder_name, mz_estimation_from_frame, mobility_estimation_from_frame)
   1035 self._version = alphatims.__version__
   1036 self._zeroth_frame = True
   1037 (
   1038     self._acquisition_mode,
   1039     global_meta_data,
   1040     self._frames,
   1041     self._fragment_frames,
   1042     self._precursors,
-> 1043 ) = read_bruker_sql(bruker_d_folder_name, self._zeroth_frame)
   1044 self._meta_data = dict(
   1045     zip(global_meta_data.Key, global_meta_data.Value)
   1046 )
   1047 (
   1048     self._push_indptr,
   1049     self._tof_indices,
   (...)
   1055     int(self._meta_data["MaxNumPeaksPerScan"]),
   1056 )

File ~\AppData\Roaming\Python\Python39\site-packages\alphatims\bruker.py:181, in read_bruker_sql(bruker_d_folder_name, add_zeroth_frame, drop_polarity)
    177 logging.info(f"Reading frame metadata for {bruker_d_folder_name}")
    178 with sqlite3.connect(
    179     os.path.join(bruker_d_folder_name, "analysis.tdf")
    180 ) as sql_database_connection:
--> 181     global_meta_data = pd.read_sql_query(
    182         "SELECT * from GlobalMetaData",
    183         sql_database_connection
    184     )
    185     frames = pd.read_sql_query(
    186         "SELECT * FROM Frames",
    187         sql_database_connection
    188     )
    189     if 9 in frames.MsMsType.values:

File C:\Program Files\Python39\lib\site-packages\pandas\io\sql.py:399, in read_sql_query(sql, con, index_col, coerce_float, params, parse_dates, chunksize, dtype)
    341 """
    342 Read SQL query into a DataFrame.
    343 
   (...)
    396 parameter will be converted to UTC.
    397 """
    398 pandas_sql = pandasSQL_builder(con)
--> 399 return pandas_sql.read_query(
    400     sql,
    401     index_col=index_col,
    402     params=params,
    403     coerce_float=coerce_float,
    404     parse_dates=parse_dates,
    405     chunksize=chunksize,
    406     dtype=dtype,
    407 )

File C:\Program Files\Python39\lib\site-packages\pandas\io\sql.py:2080, in SQLiteDatabase.read_query(self, sql, index_col, coerce_float, params, parse_dates, chunksize, dtype)
   2068 def read_query(
   2069     self,
   2070     sql,
   (...)
   2076     dtype: DtypeArg | None = None,
   2077 ):
   2079     args = _convert_params(sql, params)
-> 2080     cursor = self.execute(*args)
   2081     columns = [col_desc[0] for col_desc in cursor.description]
   2083     if chunksize is not None:

File C:\Program Files\Python39\lib\site-packages\pandas\io\sql.py:2032, in SQLiteDatabase.execute(self, *args, **kwargs)
   2029     raise ex from inner_exc
   2031 ex = DatabaseError(f"Execution failed on sql '{args[0]}': {exc}")
-> 2032 raise ex from exc

DatabaseError: Execution failed on sql 'SELECT * from GlobalMetaData': no such table: GlobalMetaData

Version (please complete the following information):

swillems commented 2 years ago

Hey @hanghu1024 . Thanks for your interest in using AlphaTims. I actually have not tried AlphaTims without IMS (the "tims" part of the name was added for a reason of course). That said, it might be compatible with you data with little modification if we are lucky. Could you share a data file with me so that I can check out of I can get this to work?

hanghu1024 commented 2 years ago

Hello @swillems, you can get that data file in this link: https://drive.google.com/drive/folders/1c7yZsjoQai1sudRDSWNjtUbdc3zz7x5H?usp=sharing Thank you so much for the wonderful work, really appreciate it!

swillems commented 2 years ago

Dear @hanghu1024, thanks for sharing the data. I have checked it out, but unfortunately it would require more time to make that format compatible with AlphaTims than I currently have available. At first glance, the issue is not so much the fact that there is no mobility, but that you use a .tsf instead of .tdf with an alternative binary format that AlphaTims currently is unable to read. I have not tested this myself, but you might want to try out TIMSCONVERT to process these files (e.g. by converting them to mzml that should be a generic input for many other tools).

We might include .tsf support in the future, but this is not on any of our plannings yet...

hanghu1024 commented 2 years ago

Hello @swillems Thank you so much! I tries some codes in timsconvert, it works for .tsf data.