scTE: TypeError: no supported conversion for types: (dtype('<U4'),)

[chaigsh]

Hi, scTE was used to detect TE expression. When I used this useful tool to analyze my cancer data, an error occurred. The commond used was: scTE -i S02-primary-rna.cancer.noSCT.pri.barcode.bam -o S02_primary_rna_out -x hg38.exclusive.idx --hdf5 True -CB CB -UMI UB -p 10 and then, an error reported below: INFO : Done fetching... 2023-03-23 11:18:43 INFO : Calculating expression... 2023-03-23 11:18:43 Traceback (most recent call last): File "/storage/anaconda3/envs/py38/lib/python3.8/site-packages/scipy/sparse/base.py", line 322, in asformat return convert_method(copy=copy) File "/storage/anaconda3/envs/py38/lib/python3.8/site-packages/scipy/sparse/coo.py", line 404, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype)) File "/storage/anaconda3/envs/py38/lib/python3.8/site-packages/scipy/sparse/sputils.py", line 51, in upcast raise TypeError('no supported conversion for types: %r' % (args,)) TypeError: no supported conversion for types: (dtype('<U4'),)

I have tried three versions of scipy, but the error still there!

Many thanks,

Guoshi Chai

[Changzhanhe]

I have met the same problem as your.

INFO : Calculating expression... 2023-04-09 16:47:38 Traceback (most recent call last): File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 376, in asformat return convert_method(copy=copy) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype)) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast raise TypeError('no supported conversion for types: %r' % (args,)) TypeError: no supported conversion for types: (dtype('<U4'),)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home1/miniconda3/envs/py38/bin/scTE", line 4, in import('pkg_resources').run_script('scTE==1.0', 'scTE') File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/pkg_resources/init.py", line 720, in run_script self.require(requires)[0].run_script(script_name, ns) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/pkg_resources/init.py", line 1559, in run_script exec(code, namespace, namespace) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 169, in main() File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 155, in main len_res, genenumber, filename = Countexpression(filename=args.out, allelement=allelement, genenumber=args.genenumber, cellnumber=args.cellnumber, hdf5=args.hdf5) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/scTE/base.py", line 506, in Countexpression adata.X = scipy.sparse.csr_matrix(adata.X) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 84, in init self._set_self(self.class( File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 33, in init arg1 = arg1.asformat(self.format) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 378, in asformat return convert_method() File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype)) File "/home1/miniconda3/envs/py38/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast raise TypeError('no supported conversion for types: %r' % (args,)) TypeError: no supported conversion for types: (dtype('<U4'),)

[ss6025]

I have got the exactly same error with command: scTE -p 8 -i ${BAM_PATH}"/"${bam}"/count/sample_alignments.bam" -x ${index} -o ${output_dir} --hdf5 True -CB CB -UMI UB

[Khreat0205]

I also encountered a similar error.

Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 376, in asformat
    return convert_method(copy=copy)
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr
    data = np.empty_like(self.data, dtype=upcast(self.dtype))
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast
    raise TypeError('no supported conversion for types: %r' % (args,))
TypeError: no supported conversion for types: (dtype('<U5'),)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/scTE", line 4, in <module>
    __import__('pkg_resources').run_script('scTE==1.0', 'scTE')
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 667, in run_script
    self.require(requires)[0].run_script(script_name, ns)
  File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1463, in run_script
    exec(code, namespace, namespace)
  File "/usr/local/lib/python3.8/dist-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 169, in <module>
    main()
  File "/usr/local/lib/python3.8/dist-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 155, in main
    len_res, genenumber, filename = Countexpression(filename=args.out, allelement=allelement, genenumber=args.genenumber, cellnumber=args.cellnumber, hdf5=args.hdf5)
  File "/usr/local/lib/python3.8/dist-packages/scTE-1.0-py3.8.egg/scTE/base.py", line 506, in Countexpression
    adata.X = scipy.sparse.csr_matrix(adata.X)
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 84, in __init__
    self._set_self(self.__class__(
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 33, in __init__
    arg1 = arg1.asformat(self.format)
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 378, in asformat
    return convert_method()
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr
    data = np.empty_like(self.data, dtype=upcast(self.dtype))
  File "/usr/local/lib/python3.8/dist-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast
    raise TypeError('no supported conversion for types: %r' % (args,))
TypeError: no supported conversion for types: (dtype('<U5'),)

[rpisalive]

Also encountered the same problem when --hdf5 is set to true, when I chose to output as csv, the problem is gone. Here is my error message:

Traceback (most recent call last): File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 376, in asformat return convert_method(copy=copy) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype)) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast raise TypeError('no supported conversion for types: %r' % (args,)) TypeError: no supported conversion for types: (dtype('<U4'),)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "user/miniconda3/envs/scTE/bin/scTE", line 4, in import('pkg_resources').run_script('scTE==1.0', 'scTE') File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/pkg_resources/init.py", line 720, in run_script self.require(requires)[0].run_script(script_name, ns) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/pkg_resources/init.py", line 1559, in run_script exec(code, namespace, namespace) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 169, in main() File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/EGG-INFO/scripts/scTE", line 155, in main len_res, genenumber, filename = Countexpression(filename=args.out, allelement=allelement, genenumber=args.genenumber, cellnumber=args.cellnumber, hdf5=args.hdf5) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scTE-1.0-py3.8.egg/scTE/base.py", line 506, in Countexpression adata.X = scipy.sparse.csr_matrix(adata.X) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 84, in init self._set_self(self.class( File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_compressed.py", line 33, in init arg1 = arg1.asformat(self.format) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_base.py", line 378, in asformat return convert_method() File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_coo.py", line 403, in tocsr data = np.empty_like(self.data, dtype=upcast(self.dtype)) File "/user/miniconda3/envs/scTE/lib/python3.8/site-packages/scipy-1.10.1-py3.8-linux-x86_64.egg/scipy/sparse/_sputils.py", line 53, in upcast raise TypeError('no supported conversion for types: %r' % (args,)) TypeError: no supported conversion for types: (dtype('<U4'),)

[koenprange]

Hi, I had the same problem. Was able to solve it by downgrading the python version in the conda env to 3.7. Good luck!

[gammertens]

Thank you for developing this tool. At this point, I am stuck at a similar error as described above when setting --hdf5 True.

@koenprange , when downgrading python in my conda env to version 3.7, I am not able to run the setup.py script. I get an error saying that python>=3.8 is required. Haven't you encountered a similar issue, or how have you been able to go about this? Running setup.py and downgrading afterwards doesn't seem to work for me as well at this point...

[koenprange]

Thank you for developing this tool. At this point, I am stuck at a similar error as described above when setting --hdf5 True.

@koenprange , when downgrading python in my conda env to version 3.7, I am not able to run the setup.py script. I get an error saying that python>=3.8 is required. Haven't you encountered a similar issue, or how have you been able to go about this? Running setup.py and downgrading afterwards doesn't seem to work for me as well at this point...

Yeah it's a bit of a dependency head scratcher... I don't recall exactly what I did, but I think I made a completely new, clean, 3.7 env and installed everything from scratch. It is also crucial to read really well the errors setup.py throws, and solve them one by one by installing the required dependencies. Good luck!

[jphe]

@ss6025 @Changzhanhe @chaigsh @Khreat0205 The h5ad format requires multiple rows and columns, and it's possible that the data has too few cells that have passed QC (less than 2). You can try exporting it as a CSV file to check.

[Changzhanhe]

Hi, I have solved this problem by adding adata.X = adata.X.astype(float) after adata = ad.AnnData(np.asarray(data),var = var,obs = obs) and before adata.X = scipy.sparse.csr_matrix(adata.X) in base.py in scTE/ folder.

[matasV99]

Hi, I tried modifying base.py as used Changzhanhe suggested, but that did not work for me to output a h5ad. My solution was to make hdf5 = FALSE, output a .csv file that looked normal to me and ran without any errors which I then transposed and loaded into Seurat (an R package I use).

[XiaoyuZhan520]

Hi, I met exactly the same problems but solved with Changzhanhe suggestion. Currently, both hdf5=False and modified base.py work well for me.