AttributeError: module 'networkx' has no attribute 'from_numpy_matrix' when I use arg --hess

[Y-Isaac]

@omerwe @jdblischak HI,

I successfully get the PIP using the estimator implemented in Susie. Now I want to use HESS to know if the output will be different from the previous result. However, when I add the arg --hess, there is an error: _AttributeError: module 'networkx' has no attribute 'from_numpymatrix'. There is the completed log:

[INFO] Loading sumstats file... [INFO] Loaded sumstats for 669237 SNPs in 4.84 seconds [INFO] cffi mode is CFFI_MODE.ANY [DEBUG] Looking for R home with: R RHOME [INFO] R home found: /public/software/anaconda3/envs/polyfun-lock/lib/R [DEBUG] Looking for LD_LIBRARY_PATH with: /public/software/anaconda3/envs/polyfun-lock/lib/R/bin/Rscript -e cat(Sys.getenv("LD_LIBRARY_PATH")) [INFO] R library path: /opt/gridview//pbs/dispatcher/lib::/usr/local/lib64:/usr/local/lib [INFO] LD_LIBRARY_PATH: /opt/gridview//pbs/dispatcher/lib::/usr/local/lib64:/usr/local/lib [DEBUG] cffi mode is InterfaceType.API [INFO] Default options to initialize R: rpy2, --quiet, --no-save [INFO] R is already initialized. No need to initialize. [INFO] Computing LD from plink fileset chr10 chromosome 10 region 18194812-21194812 Mapping files: 33%|██████████████████████████████████████████████ Mapping files: 100%|████████████████████████████████████████████████████████████████████████████████████Mapping files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:06<00:00, 2.19s/it] [INFO] Found 19350 SNPs in target region. Computing LD in 4 chunks... 25%|██████████████████████████████████████▎ 50%|████████████████████████████████████████████████████████████████████████████▌ 75%|███████████████████████████████████████████████████████████████████████████████████████████100%|███████████████████████████████████████████████████████████████████████████████████████████100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [03:12<00:00, 48.02s/it] [INFO] Done in 200.95 seconds [INFO] Flipping the effect-sign of 9197 SNPs that are flipped compared to the LD panel [INFO] Excluding SNPs with heritability less than 1.0000e-04 from the HESS estimation Traceback (most recent call last): File "/public/home/P202306/software/polyfun/finemapper.py", line 1275, in df_finemap = finemap_obj.finemap(locus_start=args.start, locus_end=args.end, num_causal_snps=args.max_num_causal, File "/public/home/P202306/software/polyfun/finemapper.py", line 778, in finemap h2_hess = self.estimate_h2_hess_wrapper(min_h2=hess_min_h2, num_samples=hess_iter) File "/public/home/P202306/software/polyfun/finemapper.py", line 685, in estimate_h2_hess_wrapper h2_hess_list = [self.estimate_h2_hess(prop_keep=prop_keep, R_cutoff=R_cutoff, pvalue_bound=pvalue_bound) \ File "/public/home/P202306/software/polyfun/finemapper.py", line 685, in h2_hess_list = [self.estimate_h2_hess(prop_keep=prop_keep, R_cutoff=R_cutoff, pvalue_bound=pvalue_bound) \ File "/public/home/P202306/software/polyfun/finemapper.py", line 656, in estimate_h2_hess G = nx.from_numpy_matrix(np.abs(R_pot_csnp)>R_cutoff) AttributeError: module 'networkx' has no attribute 'from_numpy_matrix'

And this is my code

python ~/software/polyfun/finemapper.py \ --geno chr10 \ --sumstats pheno1.10.snpvar_constrained.gz \ --n 404145 \ --chr 10 \ --start 18194812 \ --end 21194812 \ --method susie \ --max-num-causal 10 \ --out pheno1_18194812 \ --allow-missing \ --hess \ --hess-min-h2 1e-4

I try to run both in polyfun and polyfun-lock env, but the error has not been resolved.I have investigated about this, and I guess it might be because the _from_numpymatrix function has been deprecated in the newer versions of networkx. I attempted to use the following in finemapper.py line 656:

G = nx.from_numpy_array(np.abs(R_pot_csnp) > [R_cutoff)]

to replace:

G = nx.from_numpy_matrix(np.abs(R_pot_csnp) > R_cutoff)

Fortunately, this worked. However, I think this fix needs your confirmation. Please let me know if this is correct.

I am looking forward to your reply!

[jdblischak]

I confirmed that from_numpy_matrix() was removed in networkx 3.0 and that the lockfile has installed 3.1

https://github.com/omerwe/polyfun/blob/ac63a4c549801ba0615cc9cdef569ba249e9ffe6/polyfun.yml.lock#L138

This is a bit frustrating for me because I previously used the --hess flag in my work. Thus I must have had networkx<3 in my local frozen conda environment. The problem here is that the current lockfile was created and tested only against the tests in test_finemapper.py (which doesn't test --hess).

@omerwe how would you like to proceed? I see two options:

1. Update the code from from_numpy_matrix() to from_numpy_array()
2. Pin networkx<3 in polyun.yml and regenerate polyfun.yml.lock?

Regardless it would be nice to have a test added for --hess

[omerwe]

@jdblischak @Y-Isaac thanks for your support in this. I updated from_numpy_matrix() -> from_numpy_array().

I wish I had the bandwidth to add a test for --hess, but at the moment I have very low capacity for this. Hopefully there won't be any more backwards-compatability-breaking changes in the PolyFun package ecosystem for the next year or so :)

[jdblischak]

thanks for your support in this. I updated from_numpy_matrix() -> from_numpy_array().

That works. Thanks!

I wish I had the bandwidth to add a test for --hess, but at the moment I have very low capacity for this.

I totally get it when a project no longer has the priority it used to. Has happened to me many times

Hopefully there won't be any more backwards-compatability-breaking changes in the PolyFun package ecosystem for the next year or so :)

Unfortunately these aren't new breaking changes, but existing ones. I created the lockfile long after you initially developed this code base. So any code path that isn't covered by the test cases is subject to these types of failures where even the lockfile version has an incompatible version installed. But with any luck @Y-Isaac will find all the currently existing ones 🙂

[Y-Isaac]

Thanks for your confirmation! Now I'm going to close this issue.