search

MatteoLacki / timspy

Other
16 stars 8 forks source link

'TimsPyDF' object has no attribute 'to_vaex' #6

Open animesh opened 3 years ago

animesh commented 3 years ago

Installation seems fine:

(base) C:\WINDOWS\system32>C:\intelpython\python3.7\python.exe -m pip install timspy[vaex]
Requirement already satisfied: timspy[vaex] in c:\intelpython\python3.7\lib\site-packages (0.9.3)
Requirement already satisfied: tqdm in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from timspy[vaex]) (4.62.1)
Requirement already satisfied: opentims-bruker-bridge in c:\intelpython\python3.7\lib\site-packages (from timspy[vaex]) (1.0.3)
Requirement already satisfied: opentimspy in c:\intelpython\python3.7\lib\site-packages (from timspy[vaex]) (1.0.8)
Requirement already satisfied: matplotlib in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from timspy[vaex]) (3.0.3)
Requirement already satisfied: pandas in c:\intelpython\python3.7\lib\site-packages (from timspy[vaex]) (1.3.2)
Requirement already satisfied: vaex-hdf5 in c:\intelpython\python3.7\lib\site-packages (from timspy[vaex]) (0.9.0)
Requirement already satisfied: h5py in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from timspy[vaex]) (3.1.0)
Requirement already satisfied: vaex-core in c:\intelpython\python3.7\lib\site-packages (from timspy[vaex]) (4.4.0)
Requirement already satisfied: cached-property in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from h5py->timspy[vaex]) (1.5.2)
Requirement already satisfied: numpy>=1.14.5 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from h5py->timspy[vaex]) (1.19.5)
Requirement already satisfied: python-dateutil>=2.1 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from matplotlib->timspy[vaex]) (2.8.0)
Requirement already satisfied: cycler>=0.10 in c:\intelpython\python3.7\lib\site-packages (from matplotlib->timspy[vaex]) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from matplotlib->timspy[vaex]) (1.1.0)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from matplotlib->timspy[vaex]) (2.4.2)
Requirement already satisfied: six in c:\intelpython\python3.7\lib\site-packages (from cycler>=0.10->matplotlib->timspy[vaex]) (1.15.0)
Requirement already satisfied: setuptools in c:\intelpython\python3.7\lib\site-packages (from kiwisolver>=1.0.1->matplotlib->timspy[vaex]) (51.1.2.post20210202)
Requirement already satisfied: pybind11 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from opentimspy->timspy[vaex]) (2.6.2)
Requirement already satisfied: pytz>=2017.3 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from pandas->timspy[vaex]) (2019.2)
Requirement already satisfied: colorama in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from tqdm->timspy[vaex]) (0.4.4)
Requirement already satisfied: future>=0.15.2 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from vaex-core->timspy[vaex]) (0.18.2)
Requirement already satisfied: progressbar2 in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (3.53.1)
Requirement already satisfied: requests in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (2.25.1)
Requirement already satisfied: nest-asyncio>=1.3.3 in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (1.5.1)
Requirement already satisfied: pyarrow>=3.0 in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (5.0.0)
Requirement already satisfied: pyyaml in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (5.3.1)
Requirement already satisfied: blake3 in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (0.2.0)
Requirement already satisfied: frozendict in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (2.0.6)
Requirement already satisfied: dask in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (2021.9.0)
Requirement already satisfied: cloudpickle in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (1.6.0)
Requirement already satisfied: aplus in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (0.11.0)
Requirement already satisfied: tabulate>=0.8.3 in c:\intelpython\python3.7\lib\site-packages (from vaex-core->timspy[vaex]) (0.8.9)
Requirement already satisfied: packaging>=20.0 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from dask->vaex-core->timspy[vaex]) (21.0)
Requirement already satisfied: toolz>=0.8.2 in c:\intelpython\python3.7\lib\site-packages (from dask->vaex-core->timspy[vaex]) (0.11.1)
Requirement already satisfied: partd>=0.3.10 in c:\intelpython\python3.7\lib\site-packages (from dask->vaex-core->timspy[vaex]) (1.2.0)
Requirement already satisfied: fsspec>=0.6.0 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from dask->vaex-core->timspy[vaex]) (2021.7.0)
Requirement already satisfied: locket in c:\intelpython\python3.7\lib\site-packages (from partd>=0.3.10->dask->vaex-core->timspy[vaex]) (0.2.1)
Requirement already satisfied: python-utils>=2.3.0 in c:\intelpython\python3.7\lib\site-packages (from progressbar2->vaex-core->timspy[vaex]) (2.5.6)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\intelpython\python3.7\lib\site-packages (from requests->vaex-core->timspy[vaex]) (4.0.0)
Requirement already satisfied: idna<3,>=2.5 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from requests->vaex-core->timspy[vaex]) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\animeshs\appdata\roaming\python\python37\site-packages (from requests->vaex-core->timspy[vaex]) (1.25.11)
Requirement already satisfied: certifi>=2017.4.17 in c:\intelpython\python3.7\lib\site-packages (from requests->vaex-core->timspy[vaex]) (2020.12.5)

and it seems to work up to the vaex call:

D = TimsPyDF(pathFiles) # get data handle
print(pathFiles,D) #print(len(D)) # The number of peaks.
#D.framesTIC() # Return combined intensity for each frame. # array([ 95910, 579150, 906718, ..., 406317,   8093,   8629])
print(D.intensity_per_frame())
print(D.intensity_per_frame().sum())
C:\Users\animeshs\Desktop\Morten\210902_Morten_1__Slot1-37_1_176.d TimsPyDF(1634400182 peaks)
[4673188 4635323 4620431 ... 1503755 1483193 1483877]
102522974376
from timspy.vaex import TimsVaex
D.to_vaex(pathFiles.with_suffix(".hdf"))
Traceback (most recent call last):

  File "<ipython-input-11-28303ca8fad2>", line 2, in <module>
    D.to_vaex(pathFiles.with_suffix(".hdf"))

AttributeError: 'TimsPyDF' object has no attribute 'to_vaex'
animesh commented 3 years ago

Looks like there is a method "to_hdf" which replaces "to_vaex"? However, it throws ZeroDivisionError

D.to_hdf(pathFiles.with_suffix(".hdf"))
Traceback (most recent call last):

  File "<ipython-input-27-b5afe4965bf2>", line 1, in <module>
    D.to_hdf(pathFiles.with_suffix(".hdf"))

  File "C:\intelpython\python3.7\lib\site-packages\timspy\df.py", line 268, in to_hdf
    dataset.write_direct(frame[colname], dest_sel=np.s_[data_offset:data_offset+frame_size])

  File "C:\Users\animeshs\AppData\Roaming\Python\Python37\site-packages\h5py\_hl\dataset.py", line 994, in write_direct
    for fspace in dest_sel.broadcast(source_sel.mshape):

  File "C:\Users\animeshs\AppData\Roaming\Python\Python37\site-packages\h5py\_hl\selections.py", line 293, in broadcast
    chunks = tuple(x//y for x, y in zip(count, tshape))

  File "C:\Users\animeshs\AppData\Roaming\Python\Python37\site-packages\h5py\_hl\selections.py", line 293, in <genexpr>
    chunks = tuple(x//y for x, y in zip(count, tshape))

ZeroDivisionError: integer division or modulo by zero

If i try to load the created file, i see VersionConflict

V = TimsVaex(pathFiles.with_suffix(".hdf"),pathFiles/"analysis.tdf")# conda install sqlite
ERROR:MainThread:vaex:issue loading plot
Traceback (most recent call last):
  File "C:\intelpython\python3.7\lib\site-packages\vaex\__init__.py", line 789, in <module>
    add_namespace = entry.load()
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2446, in load
    self.require(*args, **kwargs)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2469, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 775, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (vaex-core 4.4.0 (c:\intelpython\python3.7\lib\site-packages), Requirement.parse('vaex-core<3,>=2.0.0'))
ERROR:MainThread:vaex:issue loading astro
Traceback (most recent call last):
  File "C:\intelpython\python3.7\lib\site-packages\vaex\__init__.py", line 789, in <module>
    add_namespace = entry.load()
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2446, in load
    self.require(*args, **kwargs)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2469, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 775, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (vaex-core 4.4.0 (c:\intelpython\python3.7\lib\site-packages), Requirement.parse('vaex-core<3,>=2.0.0-alpha.1'))
ERROR:MainThread:vaex:issue loading reader
Traceback (most recent call last):
  File "C:\intelpython\python3.7\lib\site-packages\vaex\__init__.py", line 789, in <module>
    add_namespace = entry.load()
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2446, in load
    self.require(*args, **kwargs)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 2469, in require
    items = working_set.resolve(reqs, env, installer, extras=self.extras)
  File "C:\intelpython\python3.7\lib\site-packages\pkg_resources\__init__.py", line 775, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (vaex-core 4.4.0 (c:\intelpython\python3.7\lib\site-packages), Requirement.parse('vaex-core<3,>=2.0.0'))

but it looks like there is still some data being loaded

print(V)
TimsVaex.df
#              frame    intensity    inv_ion_mobility    mz                  retention_time    scan    tof
0              1        57           1.6014686780685545  1556.9425521623032  0.530736          33      373570
1              1        10           1.599265633491839   1554.3610302362113  0.530736          35      373155
2              1        21           1.598164049385315   1337.3390287517957  0.530736          36      336941
3              1        10           1.5970624240635969  934.2539007338193   0.530736          37      260802
4              1        27           1.5970624240635969  1370.3112314162104  0.530736          37      342623
           ...      ...          ...                 ...                 ...               ...     ...
2,106,148,664  0        0            0.0                 0.0                 0.0               0       0
2,106,148,665  0        0            0.0                 0.0                 0.0               0       0
2,106,148,666  0        0            0.0                 0.0                 0.0               0       0
2,106,148,667  0        0            0.0                 0.0                 0.0               0       0
2,106,148,668  0        0            0.0                 0.0                 0.0               0       0