Met attributeError when doing spatial hotspot

[Mr-grasscarp]

I used demo data to follow the tutorial，but when the program ran to compute autocorrelation，an AttributeError happened——AttributeError: Can't pickle local object 'compute_hs..initializer' The codes are exactly the same as tutorial and the returns are as follows [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1297][INFO]: This is GEF file which contains traditional bin infomation. [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1298][INFO]: bin_type: bins [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1301][INFO]: Bin size list: ['bin1'] [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1307][INFO]: Resolution: 500 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1310][INFO]: Gene count: 24302 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1319][INFO]: offsetX: 0 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1322][INFO]: offsetY: 0 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1325][INFO]: Width: 26458 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1328][INFO]: Height: 26458 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1331][INFO]: Max Exp: 23 [2024-05-28 21:00:13][Stereo][35404][MainThread][26900][reader][1090][INFO]: read_gef begin ... Warning:your source data file only has gene column,but you specify return gene_names and gene ids which is invalid!so we will return an empty array which has same size with gene_names for the gene_ids [2024-05-28 21:00:32][Stereo][35404][MainThread][26900][reader][1268][INFO]: the matrix has 395978 cells, and 24302 genes. [2024-05-28 21:00:32][Stereo][35404][MainThread][26900][reader][1269][INFO]: read_gef end. [2024-05-28 21:00:32][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run cal_qc... [2024-05-28 21:00:32][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: cal_qc end, consume time 0.3800s. [2024-05-28 21:00:32][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run filter_cells... [2024-05-28 21:00:33][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: filter_cells end, consume time 0.5665s. [2024-05-28 21:00:33][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run normalize_total... [2024-05-28 21:00:33][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: normalize_total end, consume time 0.2703s. [2024-05-28 21:00:33][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run log1p... [2024-05-28 21:00:34][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: log1p end, consume time 0.4556s. [2024-05-28 21:00:34][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run highly_variable_genes... [2024-05-28 21:00:35][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: highly_variable_genes end, consume time 0.8402s. [2024-05-28 21:00:35][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run scale... [2024-05-28 21:02:31][Stereo][35404][MainThread][26900][st_pipeline][44][INFO]: scale end, consume time 116.2068s. [2024-05-28 21:02:31][Stereo][35404][MainThread][26900][st_pipeline][41][INFO]: start to run spatial_hotspot... INFO:numba.core.transforms:finding looplift candidates [2024-05-28 21:02:33][Stereo][35404][MainThread][26900][spatial_hotspot][50][INFO]: create the Hotspot object with 2000 genes and 200376 cells, model=normal. [2024-05-28 21:02:48][Stereo][35404][MainThread][26900][spatial_hotspot][52][INFO]: create_knn_graph with n_neighbors=30. [2024-05-28 21:02:53][Stereo][35404][MainThread][26900][spatial_hotspot][56][INFO]: Start compute_autocorrelations. AttributeError Traceback (most recent call last) Cell In[2], line 32 29 data.tl.scale() 31 # analysis of spatial hotspot ---> 32 data.tl.spatial_hotspot( 33 use_highly_genes=True, 34 use_raw=True, 35 hvg_res_key='highly_variable_genes', 36 model='normal', 37 n_neighbors=30, 38 n_jobs=20, 39 fdr_threshold=0.05, 40 min_gene_threshold=10, 41 res_key='spatial_hotspot', 42 )

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\stereo\core\st_pipeline.py:43, in logit..wrapped(*args, *kwargs) 41 logger.info('start to run {}...'.format(func.name)) 42 tk = tc.start() ---> 43 res = func(args, **kwargs) 44 logger.info('{} end, consume time {:.4f}s.'.format(func.name, tc.get_time_consumed(key=tk, restart=False))) 45 return res

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\stereo\core\st_pipeline.py:1169, in StPipeline.spatial_hotspot(self, use_highly_genes, hvg_res_key, model, n_neighbors, n_jobs, fdr_threshold, min_gene_threshold, outdir, res_key, use_raw) 1167 highly_genes_name = df.index[df['highly_variable']] 1168 data = data.sub_by_name(gene_name=highly_genes_name) -> 1169 hs = spatial_hotspot(data, model=model, n_neighbors=n_neighbors, n_jobs=n_jobs, fdr_threshold=fdr_threshold, 1170 min_gene_threshold=min_gene_threshold, outdir=outdir) 1171 self.result[res_key] = hs

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\stereo\algorithm\spatial_hotspot.py:57, in spatial_hotspot(data, model, n_neighbors, n_jobs, fdr_threshold, min_gene_threshold, outdir) 53 hs.create_knn_graph( 54 weighted_graph=False, n_neighbors=n_neighbors, 55 ) 56 logger.info('Start compute_autocorrelations.') ---> 57 hs_results = hs.compute_autocorrelations(jobs=n_jobs) 58 # select the genes with significant spatial autocorrelation 59 hs_genes = hs_results.index[hs_results.FDR < fdr_threshold]

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\hotspot\hotspot.py:436, in Hotspot.compute_autocorrelations(self, jobs) 412 def compute_autocorrelations(self, jobs=1): 413 """Perform feature selection using local autocorrelation 414 415 In addition to returning output, this also stores the output (...) 434 435 """ --> 436 return self._compute_hotspot(jobs)

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\hotspot\hotspot.py:397, in Hotspot._compute_hotspot(self, jobs) 370 def _compute_hotspot(self, jobs=1): 371 """Perform feature selection using local autocorrelation 372 373 In addition to returning output, this also stores the output (...) 394 395 """ --> 397 results = compute_hs( 398 self.counts, 399 self.neighbors, 400 self.weights, 401 self.umi_counts, 402 self.model, 403 genes=self.adata.var_names, 404 centered=True, 405 jobs=jobs, 406 ) 408 self.results = results 410 return self.results

File C:\ProgramData\anaconda3\envs\st120\lib\site-packages\hotspot\local_stats.py:206, in compute_hs(counts, neighbors, weights, num_umi, model, genes, centered, jobs) 202 g_D = D 204 if jobs > 1: --> 206 with multiprocessing.Pool(processes=jobs, initializer=initializer) as pool: 208 results = list( 209 tqdm(pool.imap(_map_fun_parallel, data_iter()), total=counts.shape[0]) 210 ) 211 else:

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\context.py:119, in BaseContext.Pool(self, processes, initializer, initargs, maxtasksperchild) 117 '''Returns a process pool object''' 118 from .pool import Pool --> 119 return Pool(processes, initializer, initargs, maxtasksperchild, 120 context=self.get_context())

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\pool.py:212, in Pool.init(self, processes, initializer, initargs, maxtasksperchild, context) 210 self._processes = processes 211 try: --> 212 self._repopulate_pool() 213 except Exception: 214 for p in self._pool:

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\pool.py:303, in Pool._repopulate_pool(self) 302 def _repopulate_pool(self): --> 303 return self._repopulate_pool_static(self._ctx, self.Process, 304 self._processes, 305 self._pool, self._inqueue, 306 self._outqueue, self._initializer, 307 self._initargs, 308 self._maxtasksperchild, 309 self._wrap_exception)

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\pool.py:326, in Pool._repopulate_pool_static(ctx, Process, processes, pool, inqueue, outqueue, initializer, initargs, maxtasksperchild, wrap_exception) 324 w.name = w.name.replace('Process', 'PoolWorker') 325 w.daemon = True --> 326 w.start() 327 pool.append(w) 328 util.debug('added worker')

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\process.py:121, in BaseProcess.start(self) 118 assert not _current_process._config.get('daemon'), \ 119 'daemonic processes are not allowed to have children' 120 _cleanup() --> 121 self._popen = self._Popen(self) 122 self._sentinel = self._popen.sentinel 123 # Avoid a refcycle if the target function holds an indirect 124 # reference to the process object (see bpo-30775)

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\context.py:327, in SpawnProcess._Popen(process_obj) 324 @staticmethod 325 def _Popen(process_obj): 326 from .popen_spawn_win32 import Popen --> 327 return Popen(process_obj)

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\popen_spawn_win32.py:93, in Popen.init(self, process_obj) 91 try: 92 reduction.dump(prep_data, to_child) ---> 93 reduction.dump(process_obj, to_child) 94 finally: 95 set_spawning_popen(None)

File C:\ProgramData\anaconda3\envs\st120\lib\multiprocessing\reduction.py:60, in dump(obj, file, protocol) 58 def dump(obj, file, protocol=None): 59 '''Replacement for pickle.dump() using ForkingPickler.''' ---> 60 ForkingPickler(file, protocol).dump(obj)

AttributeError: Can't pickle local object 'compute_hs..initializer'`

Looking forward to your help！

[tanliwei-genomics-cn]

Spatial hotspot is based on module hotspotsc, this is an error about multiprocess when you run this function on Windows, hotspotsc has solved this problem offiscially, but only updated codes without releasing a new package, you need to install hotspotsc with source codes to avoid this error:

pip install git+https://github.com/YosefLab/Hotspot.git

[Mr-grasscarp]

I installed the package with the source code as you suggested but the error still persists.

------------------ 原始邮件 ------------------ 发件人: @.>; 发送时间: 2024年5月30日(星期四) 下午5:21 收件人: @.>; 抄送: @.>; @.>; 主题: Re: [STOmics/Stereopy] Met attributeError when doing spatial hotspot (Issue #287)

Spatial hotspot is based on module hotspotsc, this is an error about multiprocess when you run this function on Windows, hotspotsc has solved this problem offiscially, but only updated codes without releasing a new package, you need to install hotspotsc with source codes to avoid this error:

pip install git+https://github.com/YosefLab/Hotspot.git

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: @.***>

[tanliwei-genomics-cn]

Did you run it on jupyter notebook? If so, did you restart the notebook kernel after reinstalling hotspotsc?

[tanliwei-genomics-cn]

Another possibility was that it might be reinstalled a old version from cache , you can add option --force-reinstall into pip comman to force it to reinstall

[Mr-grasscarp]

This error has been fixed.I first uninstalled hotspotsc-1.1.1, which is the default version required by stereopy.Then the master verison of hotspotsc can be installed normally. The master version can't be installed directly to override older version.