Correlation issues

[akumar289]

GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373 
run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

Running literature correlation and running into the above error

GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373
Command '['python3', '-m', 'scripts.sample_correlations', 'pearson', '/home/gn2/production/tmp/gn2/tmpnhcigw3k/traitfile_Mlp44Q34yP', '/home/gn2/production/tmp/gn2/tmpnhcigw3k/targetdb_AhL9ppyRIq', '/home/gn2/production/tmp/gn2/tmpnhcigw3k/corrs_RDe5RaNaPn']' returned non-zero exit status 1. (error)

Running a sample r correlation and run into the above error

 GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373 
    'dict object' has no attribute 'corr' (error)

Running a partial correlation produces this above error

[BonfaceKilz]

akumar289 @.***> aliandika:

GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373 
run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

Running literature correlation and running into the above error

@Alex doesn't this mean that gn3 changes have not been seen by tux01? If so, could you ping Arun to update the server. Otherwise, could you collab with Zach on a fix and track that on gn-gemtext-threads?

@akumar289 thanks for this report. We are looking into this. Sorry for the inconvenience.

-- (I Search, I Find, I Misplace, I Research) (D4F09EB110177E03C28E2FE1F5BBAE1E0392253F (hkp://keys.openpgp.org))

[akumar289]

I was wondering if there is any update regarding this?

[zsloan]

I can't update to the latest code because it complains about not being able to import rust-correlation. I assume this is because a new profile needs to be built after updating guix-bioinformatics, but the same way I used to do this no longer seems to work (it now complains about not being able to find "genenetwork2").

On Thu, Aug 11, 2022 at 4:01 AM BonfaceKilz @.***> wrote:

akumar289 @.***> aliandika:

GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373 run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

Running literature correlation and running into the above error

@Alex doesn't this mean that gn3 changes have not been seen by tux01? If so, could you ping Arun to update the server. Otherwise, could you collab with Zach on a fix and track that on gn-gemtext-threads?

@akumar289 thanks for this report. We are looking into this. Sorry for the inconvenience.

-- (I Search, I Find, I Misplace, I Research) (D4F09EB110177E03C28E2FE1F5BBAE1E0392253F (hkp://keys.openpgp.org))

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1211722860, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGFE7TXLUKSZVRJZQVDVYS6N7ANCNFSM56FXEO6A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[BonfaceKilz]

zsloan @.***> aliandika:

I can't update to the latest code because it complains about not being able to import rust-correlation. I assume this is because a new profile needs to be built after updating guix-bioinformatics, but the same way I used to do this no longer seems to work (it now complains about not being able to find "genenetwork2").

Perhaps @Arun can help; he has access to production. FWIW, I tested out a build of the latest GN2 and created a profile for it locally, and things work just fine.

On Thu, Aug 11, 2022 at 4:01 AM BonfaceKilz @.***> wrote:

akumar289 @.***> aliandika:

GeneNetwork tux01:gene:2.11-rc2-testing-85ad78373 run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

Running literature correlation and running into the above error

@Alex doesn't this mean that gn3 changes have not been seen by tux01? If so, could you ping Arun to update the server. Otherwise, could you collab with Zach on a fix and track that on gn-gemtext-threads?

@akumar289 thanks for this report. We are looking into this. Sorry for the inconvenience.

-- (To agree without understanding is inane. To disagree without understanding is impudent.) (D4F09EB110177E03C28E2FE1F5BBAE1E0392253F (hkp://keys.openpgp.org))

[arunisaac]

It looks like the profile needs to be rebuilt.

I don't yet have access to production, though I am trying to get access. So, I can't help immediately. Maybe @pjotrp can.

[akumar289]

FWIW

Did a literature correlation using the Full Retina database and produced this error again

GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

[robwwilliams]

Dear Aman, GN team.

I confirm that this is an error for ALL symbols. I used SHH as a well known mouse symbol. Here is the trait I used:

https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P

Here is the set-up in GN2

[image: Screen Shot 2022-08-16 at 8.59.12 AM.png]

And here is the error.

GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa
run_sample_corr_cmd() got an unexpected keyword argument

'corr_method' (error)

Zach, are you the right person to fix this error today?

It has been down now for about a week or more.

On Tue, Aug 16, 2022 at 07:49 akumar289 @.***> wrote:

FWIW

Did a literature correlation using the Full Retina database and produced this error again

GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1216592466, or unsubscribe https://github.com/notifications/unsubscribe-auth/AC55V7BBMHGNI6YTR2AKERDVZOE6VANCNFSM56FXEO6A . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[zsloan]

Perhaps @Arun can help; he has access to production. FWIW, I tested out a build of the latest GN2 and created a profile for it locally, and things work just fine.

@BonfaceKilz I don't doubt that there's some way to do this; it's just that the old command for it no longer seems to work. This is the command I was using (to create a profile in my branch):

env GUIX_PACKAGE_PATH='/home/zas1024/guix-bioinformatics:/home/zas1024/guix-past/modules' /home/zas1024/.config/guix/current/bin/guix install genenetwork2 -p ~/opt/gn2__20220812 --substitute-urls="http://guix.genenetwork.org https://berlin.guixsd.org https://ci.guix.gnu.org https://mirror.hydra.gnu.org"

It complains about there being no "genenetwork2" package; I tried substituting in a couple alternatives in case the names changed at some point (since I remember it used to be "python3-genenetwork2" until it was changed to genenetwork2, IIRC, and I also tried "genenetwork"). Instead of asking like this, is there a way I can check myself?

[zsloan]

I reverted to earlier commits that I think should get the literature correlation working again (but may lose other changes from the past couple weeks).

On Tue, Aug 16, 2022 at 9:03 AM Rob Williams @.***> wrote:

Dear Aman, GN team.

I confirm that this is an error for ALL symbols. I used SHH as a well known mouse symbol. Here is the trait I used:

* https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P < https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P

*

Here is the set-up in GN2

[image: Screen Shot 2022-08-16 at 8.59.12 AM.png]

And here is the error.

GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

Zach, are you the right person to fix this error today?

It has been down now for about a week or more.

On Tue, Aug 16, 2022 at 07:49 akumar289 @.***> wrote:

FWIW

Did a literature correlation using the Full Retina database and produced this error again

GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error)

— Reply to this email directly, view it on GitHub < https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1216592466 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AC55V7BBMHGNI6YTR2AKERDVZOE6VANCNFSM56FXEO6A

. You are receiving this because you are subscribed to this thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1216682261, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGCVBKMEFM6AWGXYNT3VZONRJANCNFSM56FXEO6A . You are receiving this because you commented.Message ID: @.***>

[BonfaceKilz]

zsloan @.***> aliandika:

Perhaps @Arun can help; he has access to production. FWIW, I tested out a build of the latest GN2 and created a profile for it locally, and things work just fine.

@BonfaceKilz I don't doubt that there's some way to do this; it's just that the old command for it no longer seems to work. This is the command I was using (to create a profile in my branch):

env GUIX_PACKAGE_PATH='/home/zas1024/guix-bioinformatics:/home/zas1024/guix-past/modules' /home/zas1024/.config/guix/current/bin/guix install genenetwork2 -p ~/opt/gn2__20220812 --substitute-urls="http://guix.genenetwork.org https://berlin.guixsd.org https://ci.guix.gnu.org https://mirror.hydra.gnu.org"

Ah I see. Arun is getting access to production so he may be able to help.

It complains about there being no "genenetwork2" package; I tried substituting in a couple alternatives in case the names changed at some point (since I remember it used to be "python3-genenetwork2" until it was changed to genenetwork2, IIRC, and I also tried "genenetwork"). Instead of asking like this, is there a way I can check myself?

Yeah. One quick way, adapting you command above would be to run:

/home/zas1024/.config/guix/current/bin/guix \ search -L /home/zas1024/guix-bioinformatics \ -L /home/zas1024/guix-past/modules genenetwork2

The above will tell you what version of GN2 we are running. Alternatively, a more manual way to check where GN2 is located and what version it's at would be by looking directly in the upstream repository here: https://git.genenetwork.org/guix-bioinformatics/guix-bioinformatics/src/branch/master/gn/packages/genenetwork.scm#L158

FWIW, you could jump to where genenetwork2 - assuming the package is called "genenetwork2" which it is atm - is defined by running:

/home/zas1024/.config/guix/current/bin/guix \ edit -L /home/zas1024/guix-bioinformatics \ -L /home/zas1024/guix-past/modules genenetwork2

and you can quickly see what commit GN2 is on.

I HTH'ed.

PS:

env GUIX_PACKAGE_PATH='/home/zas1024/guix-bioinformatics:/home/zas1024/guix-past/modules' /home/zas1024/.config/guix/current/bin/guix search

is equivalent to:

/home/zas1024/.config/guix/current/bin/guix -L /home/zas1024/guix-bioinformatics -L /home/zas1024/guix-past/modulessearch

Whereby the "-L" typically refers to [from the man page]:

-L, --load-path=DIR prepend DIR to the package module search path

-- (To agree without understanding is inane. To disagree without understanding is impudent.) (D4F09EB110177E03C28E2FE1F5BBAE1E0392253F (hkp://keys.openpgp.org))

[akumar289]

I reverted to earlier commits that I think should get the literature correlation working again (but may lose other changes from the past couple weeks).

Re-ran Lit correlation and ran into a new error (I think):

Settings were lit correlation, top 1000, HEI Full Retina Illumina V6.2 (Apr10) RankInv, trait ID: ILMN_2547762

GeneNetwork tux01:gene:2.11-rc2-HEAD-e8bd857c6 Command '['python3', '-m', 'scripts.sample_correlations', 'pearson', '/home/gn2/production/tmp/gn2/tmp8443bcr5/traitfile_3xYdm0U3Mt', '/home/gn2/production/tmp/gn2/tmp8443bcr5/targetdb_2ITMn1AKO2', '/home/gn2/production/tmp/gn2/tmp8443bcr5/corrs_sdDUH2Amcb']' returned non-zero exit status 1. (error)

[zsloan]

Ah, I think that is new. Thanks for giving the dataset/options. I'll take a look.

On Wed, Aug 17, 2022, 5:32 PM akumar289 @.***> wrote:

I reverted to earlier commits that I think should get the literature correlation working again (but may lose other changes from the past couple weeks). On Tue, Aug 16, 2022 at 9:03 AM Rob Williams @.

> wrote: … <#m58585285515054626> Dear Aman, GN team. I confirm that this is an error for ALL symbols. I used SHH as a well known mouse symbol. Here is the trait I used: https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P < https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P https://genenetwork.org/show_trait?trait_id=1427571_at&dataset=HC_M2_0606_P

Here is the set-up in GN2 [image: Screen Shot 2022-08-16 at 8.59.12 AM.png] And here is the error. GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' (error) Zach, are you the right person to fix this error today? It has been down now for about a week or more. On Tue, Aug 16, 2022 at 07:49 akumar289 @.**> wrote: > FWIW > > Did a literature correlation using the Full Retina database and produced > this error again > > GeneNetwork tux01:gene:2.11-rc2-HEAD-fcd2897fa > run_sample_corr_cmd() got an unexpected keyword argument 'corr_method' > (error) > > — > Reply to this email directly, view it on GitHub > < #718 (comment) https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1216592466 , > or unsubscribe > < https://github.com/notifications/unsubscribe-auth/AC55V7BBMHGNI6YTR2AKERDVZOE6VANCNFSM56FXEO6A https://github.com/notifications/unsubscribe-auth/AC55V7BBMHGNI6YTR2AKERDVZOE6VANCNFSM56FXEO6A

. > You are receiving this because you are subscribed to this thread.Message > ID: @.> > — Reply to this email directly, view it on GitHub <#718 (comment) https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1216682261>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGCVBKMEFM6AWGXYNT3VZONRJANCNFSM56FXEO6A https://github.com/notifications/unsubscribe-auth/AANQJGCVBKMEFM6AWGXYNT3VZONRJANCNFSM56FXEO6A . You are receiving this because you commented.Message ID: @.>

re-ran Literature correlation and ran into I think a new error:

settings were lit correlation, top 1000, HEI Full Retina Illumina V6.2 (Apr10) RankInv

GeneNetwork tux01:gene:2.11-rc2-HEAD-e8bd857c6 Command '['python3', '-m', 'scripts.sample_correlations', 'pearson', '/home/gn2/production/tmp/gn2/tmp8443bcr5/traitfile_3xYdm0U3Mt', '/home/gn2/production/tmp/gn2/tmp8443bcr5/targetdb_2ITMn1AKO2', '/home/gn2/production/tmp/gn2/tmp8443bcr5/corrs_sdDUH2Amcb']' returned non-zero exit status 1. (error)HEI Full Retina Illumina V6.2 (Apr10) RankInv http://gn1.genenetwork.org/webqtl/main.py?FormID=sharinginfo&InfoPageName=Illum_Retina_BXD_RankInv0410

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1218558826, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGDTQEWN2VRL6GPEW3TVZVR7XANCNFSM56FXEO6A . You are receiving this because you commented.Message ID: @.***>

[akumar289]

Was wondering if there was any update on this? Appreciate your help. I tried the same inputs as before and the same error appears as above

GeneNetwork tux01:gene:2.11-rc2-HEAD-3e1f9f2b6 Command '['python3', '-m', 'scripts.sample_correlations', 'pearson', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/traitfile_dWcRNY6DsK', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/targetdb_qRZDu4Mmez', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/corrs_nQBccOkS5A']' returned non-zero exit status 1. (error)

[zsloan]

I think I know what's happening with the literature correlation issue (I believe it applies to any situation where you try to do literature correlation against a dataset different from the dataset of the trait in question), but probably won't be able to merge the fix for it until after this week, since we're trying to avoid touching the production server during that time.

On Tue, Aug 23, 2022 at 2:33 PM akumar289 @.***> wrote:

Was wondering if there was any update on this? Appreciate your help. I tried the same inputs as before and the same error appears as above

GeneNetwork tux01:gene:2.11-rc2-HEAD-3e1f9f2b6 Command '['python3', '-m', 'scripts.sample_correlations', 'pearson', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/traitfile_dWcRNY6DsK', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/targetdb_qRZDu4Mmez', '/home/gn2/production/tmp/gn2/tmp0_rp41hc/corrs_nQBccOkS5A']' returned non-zero exit status 1. (error)

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1224709094, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGGYS46IX5GSR6PHINLV2URPNANCNFSM56FXEO6A . You are receiving this because you commented.Message ID: @.***>

[akumar289]

Understood. However, I ran the literature correlation against the same dataset of the trait in question.

[zsloan]

Was this the Hippocampus trait 1427571_at correlated against the same Hippocampus dataset? The trait link above is for that trait, but in a later comment you mentioned the Retina dataset.

Literature correlations works for me if I do the above trait against its own dataset, but returned an error if I changed the target dataset to the Retina one.

On Tue, Aug 23, 2022 at 3:55 PM akumar289 @.***> wrote:

Understood. However, I ran the literature correlation against the same dataset of the trait in question.

— Reply to this email directly, view it on GitHub https://github.com/genenetwork/genenetwork2/issues/718#issuecomment-1224876934, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANQJGFBQR5TA6VSIRLCAADV2U3FFANCNFSM56FXEO6A . You are receiving this because you commented.Message ID: @.***>