bd2kccd / causal-cmd

16 stars 8 forks source link

Is Implementation of R4 method available in Causal-cmd? #69

Open f-arab opened 2 years ago

f-arab commented 2 years ago

Hi,

I'm planning to use the "R4" method but I couldn't find it on neither Tetrad GUI or Causal-cmd. The "introduction" part in https://bd2kccd.github.io/docs/causal-cmd/ says that R4 is among the available algorithms, however, when I check the algorithms in causal-cmd it only shows the following list: Algorithm: fas, fask, fci, fcimax, fges, fges-mb, fofc, ftfc, gfci, glasso, imgs_cont, imgs_disc, lingam, mbfs, mgm, multi-fask, pc-all, r-skew, r3, rfci, skew, ts-fci, ts-gfci, ts-imgs

Would you please tell me where can I find the implementation for R4 method?

Thanks

jdramsey commented 2 years ago

The R4 method was deprecated and removed. What is your application? Perhaps I could recommend a better algorithm for your purpose?

jdramsey commented 2 years ago

Also it's looks like you're using an old version of Tetrad currently. This is a bad website but it does list the up to date version.

https://sites.google.com/view/tetradcausal

The Tetrad GUI there is up to date; tetrad-cmd will be up to date within a week I think.

f-arab commented 2 years ago

Thank you for your quick response. Our application is the causal discovery from large-scale fMRI data. We have task fMRI data and therefore not very long recordings, but would like to do it across the whole brain, so we preferably (though not necessarily) need something that can handle tens to hundreds of nodes. Thanks

jdramsey commented 2 years ago

Honestly, ever since I hooked GitHub up into Slack I see queries from people almost right away.

Have you seen this paper?

https://direct.mit.edu/netn/article/3/2/274/2211/Estimating-feedforward-and-feedback-effective

We did this work a few years ago, but the Two-Step algorithm in particular uses similar ideas to R4 and has pretty good performance. The FASK algorithm is in Tetrad and has good performance for fMRI as well.

f-arab commented 2 years ago

Great! Yes, we have looked at the FASK method as well which is very promising, however, we haven't evaluated the results of our data yet. Thanks for the recommendation, I'll try the Two-Step algorithm as well (it seems that the MATLAB implementation is available).

f-arab commented 1 year ago

Hi @jdramsey ,

I have a question about the runtime of the FASK method on large graphs such as simulated Macaque data In the FASK paper. When we try FASK on the Full Macaque data, it takes a few hours as opposed to a few minutes mentioned in the FASK paper although our machine is quite powerful. I am running FASK with the parameters' default values except for the alpha parameter, which we use as a hyperparameter. I would like to ask if any other parameters such as penalty discount (c) also play a role in the speed here and if so, how can we set that one while running FASK on causal-cmd?

Thanks

jdramsey commented 1 year ago

To be honest, we should loop Ruben Sanchez (cc'd) into this conversation, since he had done that analysis and could probably easily answer your questions. I'll wait to see what he says.

Best,

Joe

On Tue, Dec 13, 2022 at 2:52 PM Fahimeh Arab @.***> wrote:

Hi @jdramsey https://github.com/jdramsey ,

I have a question about the runtime of the FASK method on large graphs such as simulated Macaque data In the FASK paper. When we try FASK on the Full Macaque data, it takes a few hours as opposed to a few minutes mentioned in the FASK paper although our machine is quite powerful. I am running FASK with the parameters' default values except for the alpha parameter, which we use as a hyperparameter. I would like to ask if any other parameters such as penalty discount (c) also play a role in the speed here and if so, how can we set that one while running FASK on causal-cmd?

Thanks

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1349604878, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR2WKEOQONWNDUUAASTWNDHYVANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

Thank you so much.

Best, Fahimeh

On Wed, Dec 14, 2022 at 6:51 AM Joseph Ramsey @.***> wrote:

To be honest, we should loop Ruben Sanchez (cc'd) into this conversation, since he had done that analysis and could probably easily answer your questions. I'll wait to see what he says.

Best,

Joe

On Tue, Dec 13, 2022 at 2:52 PM Fahimeh Arab @.***> wrote:

Hi @jdramsey https://github.com/jdramsey ,

I have a question about the runtime of the FASK method on large graphs such as simulated Macaque data In the FASK paper. When we try FASK on the Full Macaque data, it takes a few hours as opposed to a few minutes mentioned in the FASK paper although our machine is quite powerful. I am running FASK with the parameters' default values except for the alpha parameter, which we use as a hyperparameter. I would like to ask if any other parameters such as penalty discount (c) also play a role in the speed here and if so, how can we set that one while running FASK on causal-cmd?

Thanks

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1349604878 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACLFSR2WKEOQONWNDUUAASTWNDHYVANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1351555166, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RSZVHYXLKPN34AJVB7LWNHNFZANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

rubenSaro commented 1 year ago

Hi Fahimeh,

Sorry for the late response. Can you let me know the exact values of the parameters you are using for FASK? Also, how are you selecting alpha as a hyperparameter. Thanks.

Ruben

f-arab commented 1 year ago

Hi Ruben,

No problem. Thanks for replying. I am using FASK through causal-cmd. In terms of the parameters, everything is set as default for FASK in causal-cmd except the alpha parameter which I am using as a hyperparameter and sweep over it to find the best value. Even for simple networks, it seems that the execution times are not small as you can see from the following figure. The shown execution time here is the average of over 60 repetitions. And the number of time samples is 5000 (we used concatenated data).

[image: image.png]

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:05 PM rubenSaro @.***> wrote:

Hi Fahimeh,

Sorry for the late response. Can you let me know the exact values of the parameters you are using for FASK? Also, how are you selecting alpha as a hyperparameter. Thanks.

Ruben

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362164836, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RSZAL3KQJ4HVTK3JWYLWON5L3ANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

rubenSaro commented 1 year ago

So you are using the default parameters of causal-cmd. @jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here: https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

f-arab commented 1 year ago

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0, "/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master" )

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat" path_to_Macaque_full = "/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp  = pd.read_csv(path_to_Macaque_full+"concat_BOLDfslfilter_"

+current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd = '/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9:
    current_str = "0"+str(r)
elif 10 <= r:
    current_str = str(r)

path_to_data = path_to_Macaque_full +

'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

    path_to_fask_output = 'results_all_method_mecacque/FASK_outputs'
    start = time.time()
    os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask

--alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

    np.savez('results_all_method_mecacque/FASK_macacque_full_Time',

FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here: https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar ) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

f-arab commented 1 year ago

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0, "/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master" )

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat" path_to_Macaque_full = "/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)
df_temp  = pd.read_csv(path_to_Macaque_full+"concat_BOLDfslfilter_"
+current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd = '/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):
if r <= 9:
    current_str = "0"+str(r)
elif 10 <= r:
    current_str = str(r)

path_to_data = path_to_Macaque_full + 'concat_BOLDfslfilter_%s'%current_str+
'.txt'
for a in range(0,len(alpha)):

    path_to_fask_output = 'results_all_method_mecacque/FASK_outputs'
    start = time.time()
    os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask
--alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start
    np.savez('results_all_method_mecacque/FASK_macacque_full_Time',
FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here: https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar ) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

jdramsey commented 1 year ago

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master" )

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat" path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar ) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

jdramsey commented 1 year ago

Oops, without the asterisks, just:

blah blah blah --default blah blah blah

On Wed, Dec 21, 2022 at 6:47 PM Fahimeh Arab @.***> wrote:

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 ,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362226421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4LCRFBCRISCRCPVEDWOOJKXANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

rubenSaro commented 1 year ago

Thanks,

Before trying to replicate your code, are you getting long running times for every value of alpha tested, or just for some?

Ruben

On Wed, Dec 21, 2022, 6:03 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master" )

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat" path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar ) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362201857, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFMA2MTRQJZABOKV2BDWOOECJANCNFSM5YLID5IA . You are receiving this because you commented.Message ID: @.***>

f-arab commented 1 year ago

I could not complete the experiment for Full macaque data as it was so slow. However, for the simple networks, the execution times for different values of alpha are as follows:

[image: image.png]

So, it is getting slower by increasing the value of alpha.

Best,

On Wed, Dec 21, 2022 at 5:12 PM rubenSaro @.***> wrote:

Thanks,

Before trying to replicate your code, are you getting long running times for every value of alpha tested, or just for some?

Ruben

On Wed, Dec 21, 2022, 6:03 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master" )

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat" path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362201857 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AJUZSFMA2MTRQJZABOKV2BDWOOECJANCNFSM5YLID5IA

. You are receiving this because you commented.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362276711, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RSZSYPNPNS7LIEB23ZTWOOTHNANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

jdramsey commented 1 year ago

Hmmm... default should work... I'll look at some Python scripts of mine tomorrow and see... unfortunately I don't have Python installed on this computer...

On Wed, Dec 21, 2022 at 6:47 PM Fahimeh Arab @.***> wrote:

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 ,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362226421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4LCRFBCRISCRCPVEDWOOJKXANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

jdramsey commented 1 year ago

Hold on I have an example online for some different algorithms...

https://github.com/cmu-phil/algocompy/blob/main/causalcmd/tetrad_cmd_algs.py

On Wed, Dec 21, 2022 at 8:48 PM Joseph Ramsey @.***> wrote:

Hmmm... default should work... I'll look at some Python scripts of mine tomorrow and see... unfortunately I don't have Python installed on this computer...

On Wed, Dec 21, 2022 at 6:47 PM Fahimeh Arab @.***> wrote:

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 ,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362226421, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR4LCRFBCRISCRCPVEDWOOJKXANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

Thanks. In this example code, the version of causal-cmd is 1.4.1 but the one that I have which is the one I downloaded from https://www.ccd.pitt.edu/tools/ is 1.3.0. could this be the reason I'm getting an unrecognized option error in my code? By the way, is 1.4.1 the latest version? how can I get that?

Best,

On Wed, Dec 21, 2022 at 6:28 PM Joseph Ramsey @.***> wrote:

Hold on I have an example online for some different algorithms...

https://github.com/cmu-phil/algocompy/blob/main/causalcmd/tetrad_cmd_algs.py

On Wed, Dec 21, 2022 at 8:48 PM Joseph Ramsey @.***> wrote:

Hmmm... default should work... I'll look at some Python scripts of mine tomorrow and see... unfortunately I don't have Python installed on this computer...

On Wed, Dec 21, 2022 at 6:47 PM Fahimeh Arab @.***> wrote:

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389 ,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362226421>, or unsubscribe < https://github.com/notifications/unsubscribe-auth/ACLFSR4LCRFBCRISCRCPVEDWOOJKXANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362315428, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RS3W5NX2UTE53XUFGUTWOO4FJANCNFSM5YLID5IA . You are receiving this because you authored the thread.Message ID: @.***>

jdramsey commented 1 year ago

Oh I see... Google got all messed up and is pointing to the wrong places. The latest versions of all of the Tetrad software (including causal-cmd) are on Maven Central, here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/

If you have trouble figuring out which file to use, let me know. There is a later version 1.4.2 waiting to be published; my intention is to use some of my time over break to make that happen.

Sorry about that--I was under the weather for just about all of December.

On Thu, Dec 22, 2022 at 1:52 AM Fahimeh Arab @.***> wrote:

Thanks. In this example code, the version of causal-cmd is 1.4.1 but the one that I have which is the one I downloaded from https://www.ccd.pitt.edu/tools/ is 1.3.0. could this be the reason I'm getting an unrecognized option error in my code? By the way, is 1.4.1 the latest version? how can I get that?

Best,

On Wed, Dec 21, 2022 at 6:28 PM Joseph Ramsey @.***> wrote:

Hold on I have an example online for some different algorithms...

https://github.com/cmu-phil/algocompy/blob/main/causalcmd/tetrad_cmd_algs.py

On Wed, Dec 21, 2022 at 8:48 PM Joseph Ramsey @.***> wrote:

Hmmm... default should work... I'll look at some Python scripts of mine tomorrow and see... unfortunately I don't have Python installed on this computer...

On Wed, Dec 21, 2022 at 6:47 PM Fahimeh Arab @.***> wrote:

It does not recognize --default as an option. is this the correct way of adding it to my causal-cmd call?

os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a]) + ' --default --data-type continuous --dataset '+path_to_data+' --out '+path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s'%current_str+' --delimiter tab --test cci-test --verbose No')

On Wed, Dec 21, 2022 at 3:23 PM Joseph Ramsey @.***> wrote:

Could you try adding the flag "--default" flag to the causal command call?

Joe

On Wed, Dec 21, 2022 at 6:04 PM Fahimeh Arab @.***> wrote:

Attached is also the .py code.

On Wed, Dec 21, 2022 at 3:02 PM Fahimeh Arab @.***> wrote:

Thanks Ruben. I'm using causal-cmd-1.3.0 and the following is a sample code which I'm using which tries to run FASK over Macaque Full data for different values of alpha:

import numpy as np import time import sys import os import datetime import pandas as pd

sys.path.insert(0,

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal_discovery_for_time_series_master"

)

parameters and variables

alpha = [1e-8, 1e-6, 1e-4, 1e-2, 1e-1, 0.12, 0.14,0.16, 0.18, 0.2]

num_rep_macaque_all = 60

Load simulated data

path_to_Macaque_small_degree =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/SmallDegree/data_fslfilter_concat"

path_to_Macaque_long_range =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/LongRange/data_fslfilter_concat"

path_to_Macaque_full =

"/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/codes/Causality_Analysis/Dataset/DataSets_Feedbacks/2_Macaque_Networks/Full/data_fslfilter_concat/"

num_nodes_macaque_small_degree = 28

num_nodes_macaque_long_range = 67

num_nodes_macaque_full = 91

nodes_macaque_small_degree = ['X%d'%i for i in

range(1,num_nodes_macaque_small_degree+1)]

nodes_macaque_long_range = ['X%d'%i for i in

range(1,num_nodes_macaque_long_range+1)] nodes_macaque_full = ['X%d'%i for i in range(1,num_nodes_macaque_full+1)]

Lets load data

macaque_all_data_full = [0]*num_rep_macaque_all

for i in range(1,num_rep_macaque_all+1): if i <= 9: current_str = "0"+str(i) elif 10 <= i: current_str = str(i)

df_temp = pd.read_csv(path_to_Macaque_full+"concatBOLDfslfilter" +current_str+".txt", delimiter = '\t') macaque_all_data_full[i-1] = df_temp.to_numpy()

Run FASK on Causal-cmd

path_to_causal_cmd =

'/home/fahimeh/Volitional_Control_Neurofeedback/Non-DecNef/causal_discovery_packages/causal-cmd-1.3.0-distribution/causal-cmd-1.3.0/causal-cmd-1.3.0-jar-with-dependencies.jar'

FASK_macaque_execution_time =[[0 for t in range(0, len(alpha))] for j in range(0,num_rep_macaque_all)]

for r in range(1, num_rep_macaque_all+1):

if r <= 9: current_str = "0"+str(r) elif 10 <= r: current_str = str(r)

path_to_data = path_to_Macaque_full + 'concatBOLDfslfilter%s'%current_str+ '.txt'

for a in range(0,len(alpha)):

path_to_fask_output = 'results_all_method_mecacque/FASK_outputs' start = time.time() os.system('java -jar '+path_to_causal_cmd+ ' --algorithm fask --alpha '+ str(alpha[a])+ ' --penaltyDiscount ' + str(2.0) + ' --data-type continuous --dataset '+path_to_data+' --out ' +path_to_fask_output+' --prefix '+'macaque-full-'+ str(alpha[a])+'-%s' %current_str+' --delimiter tab --test cci-test --verbose No') end = time.time() print('repetition %d'%(r), 'alpha = %0.3f'%alpha[a], 'execution time: %0.3f s'%(end-start) ) FASK_macaque_execution_time[r-1][a] = end-start

np.savez('results_all_method_mecacque/FASK_macacque_full_Time', FASK_macaque_execution_time = FASK_macaque_execution_time)

Thanks, Fahimeh

On Wed, Dec 21, 2022 at 2:24 PM rubenSaro @.***> wrote:

So you are using the default parameters of causal-cmd. @jdramsey https://github.com/jdramsey, would you happen to know which are those. I made a test using the Tetrad GUI (latest version from here:

https://s01.oss.sonatype.org/content/repositories/releases/io/github/cmu-phil/tetrad-gui/7.1.0/tetrad-gui-7.1.0-launch.jar

) And using an alpha of 10^-7, it worked in seconds. I wonder what could be the problem. Can you send us an example of the code you are running. and the tetrad you are using. thanks

ruben

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362178389

,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RS7PSCRNVBIBD5VHS2TWON7RPANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362202799

, or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4DGQATLGO6WJJUW43WOOEHZANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub <

https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362214399 ,

or unsubscribe <

https://github.com/notifications/unsubscribe-auth/AL37RSYQMQLSF4PZMTZCIZLWOOGPXANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub < https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362226421 , or unsubscribe <

https://github.com/notifications/unsubscribe-auth/ACLFSR4LCRFBCRISCRCPVEDWOOJKXANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362315428 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AL37RS3W5NX2UTE53XUFGUTWOO4FJANCNFSM5YLID5IA

. You are receiving this because you authored the thread.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1362480546, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACLFSR7VXNEM7LM4U5AS3BDWOP3DJANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

jdramsey commented 1 year ago

@f-arab @rubenSaro I might be of more help on this in a few days; I just downloaded that macaque data and have been playing with it.

f-arab commented 1 year ago

Thanks, @jdramsey. This new version accepts the --default option. I'll let you know if I get the expected speed up after adding this option for FASK.

No worries. Hope you feel better soon. Just let me know whenever you get a chance to run my code.

jdramsey commented 1 year ago

Great! I will try your code. I've been mainly playing with the examples in the Tetrad interface, and I'm getting a pretty good idea how to get it to work, so hopefully that can be translated back into parameters for causal-cmd. (I got that idea from @rubenSaro above.) There have been a number changes to Tetrad since that article was published, but the main issue for reproducibility I think is that I don't know which exact concatenations of the 500-variable single datasets were used to produce that table in the article. (@rubenSaro Do you know?) The Box site gives 60 10-dataset concatenations (5000 variables each), but I think that table uses only one for each of 10-, 20- etc. dataset concatenations. So I don't know if exact reproducibility is possible, though one could get a general idea I think.

I'll go back through the article to see which exact parameters were being used. I know they were being tuned. When I do my own tuning manually I can actually get better results than were reported in that table for FASK for the 10-dataset concatenations, but my surmise is that I'm not using exactly the same concatenation, same problem as I just said above.

For instance with one of the concatenations (#34) for the long-range Macaque simulations, if I use FAS Stable with a p-value cutoff of 1e-7, with FASK 1 with a bias of -0.1 (the standard bias for FASK1--that is, the original FASK which I think was used in the article) I get these numbers:

  AP          0.95   Adjacency Precision                                   
  AR          0.401  Adjacency Recall                                      
  AHP         0.95   Arrowhead precision                                   
  AHR         0.354  Arrowhead recall                                      
  AHPC        1      Arrowhead precision (common edges)                    
  AHRC        0.76   Arrowhead recall (common edges)

This comes back in less than a second, and the adjacency recall is better than what is reported in that table in the Feedbacks article. But mileage can vary.

(Nevertheless obviously with careful tuning Two Step got much better recall for the concatenation used in the article.)

So what I need to do then is set this up in causal-cmd and see how it works I think.

But like I said for reproducibility we definitely need @rubenSaro 's input.

f-arab commented 1 year ago

Thanks, @jdramsey. I really appreciate your time and help on this.

rubenSaro commented 1 year ago

Hi @f-arab,

Any improvement in the running time?

Sorry, I could not see the image you posted about running time. Can you send it again?

Given the results shown by the three of us, I am inclined to think that the problem is coming from the high alpha values used in your experiment. My hypothesis is that the combination of large sample size and high alpha may be resulting in dense graphs (not a lot of edges removed in each pass of FAS) and thus resulting in a very large number of conditioning sets that need to be tested. Could you just make one run with an alpha of 1e-7 (p-value cutoff)?

Ruben

f-arab commented 1 year ago

HI @rubenSaro ,

Yes, you are right, When I'm changing the alpha value for macaque full data to 1e-7, that one becomes so fast as well in Tetrad. The reason that I tried macaque full data with alpha = 0.1 is that in your paper, it is mentioned that alpha = 0.1 gave the best results for that data, With small values of alpha, I see that FASK is super fast (when I'm running it through Tetrad), the precision very high, however, the recall is very small. So maybe, that's why you also tried higher values of alpha to increase the recall for the full macaque graph as well.

There is still one thing that I don't understand which is why the execution times are much higher when I'm using FASK with these hyperparameters on causal-cmd?!

By the way, is there any way of using Tetrad through a script or it should be done manually through the GUI? let's say we want to run FASK on Tetrad for all 60 repetitions of Full macaque data. What is the best way in that case?

Regarding the timing, I got for FASK while using causal-cmd for different values of alpha: (here are the results for data from simple networks 1 to 9)

f-arab commented 1 year ago

Hi @jdramsey ,

I tried running FASK through Tetrad with all the hyperparameters you mentioned for repetition #34 of long-range macaque data and I got the same results as you have shown and it was also very fast, Thanks

rubenSaro commented 1 year ago

Hi,

Glad the replication is working. Can you cite the section of the paper where we mentioned alpha 0.1 is the best for the data, just curious.

Ruben

On Thu, Jan 5, 2023, 6:15 PM Fahimeh Arab @.***> wrote:

HI @rubenSaro https://github.com/rubenSaro ,

Yes, you are right, When I'm changing the alpha value for macaque full data to 1e-7, that one becomes so fast as well in Tetrad. The reason that I tried macaque full data with alpha = 0.1 is that in your paper, it is mentioned that alpha = 0.1 gave the best results for that data, With small values of alpha, I see that FASK is super fast (when I'm running it through Tetrad), the precision very high, however, the recall is very small. So maybe, that's why you also tried higher values of alpha to increase the recall for the full macaque graph as well.

There is still one thing that I don't understand which is why the execution times are much higher when I'm using FASK with these hyperparameters on causal-cmd?!

By the way, is there any way of using Tetrad through a script or it should be done manually through the GUI? let's say we want to run FASK on Tetrad for all 60 repetitions of Full macaque data. What is the best way in that case?

Regarding the timing, I got for FASK while using causal-cmd for different values of alpha: (here are the results for data from simple networks 1 to 9)

[image: image] https://user-images.githubusercontent.com/49805515/210898090-a49c517f-7825-4b56-9717-b30d236c7288.png

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1372914385, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFJEAKHHXQJOMLZI26DWQ5IX7ANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

rubenSaro commented 1 year ago

Let me try a causal-cmd run and see if I also get high running times. Remind me of your version.

Ruben

On Thu, Jan 5, 2023, 6:15 PM Fahimeh Arab @.***> wrote:

HI @rubenSaro https://github.com/rubenSaro ,

Yes, you are right, When I'm changing the alpha value for macaque full data to 1e-7, that one becomes so fast as well in Tetrad. The reason that I tried macaque full data with alpha = 0.1 is that in your paper, it is mentioned that alpha = 0.1 gave the best results for that data, With small values of alpha, I see that FASK is super fast (when I'm running it through Tetrad), the precision very high, however, the recall is very small. So maybe, that's why you also tried higher values of alpha to increase the recall for the full macaque graph as well.

There is still one thing that I don't understand which is why the execution times are much higher when I'm using FASK with these hyperparameters on causal-cmd?!

By the way, is there any way of using Tetrad through a script or it should be done manually through the GUI? let's say we want to run FASK on Tetrad for all 60 repetitions of Full macaque data. What is the best way in that case?

Regarding the timing, I got for FASK while using causal-cmd for different values of alpha: (here are the results for data from simple networks 1 to 9)

[image: image] https://user-images.githubusercontent.com/49805515/210898090-a49c517f-7825-4b56-9717-b30d236c7288.png

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1372914385, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFJEAKHHXQJOMLZI26DWQ5IX7ANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

kvb2univpitt commented 1 year ago

If you're using causal-cmd version >= 1.4.1, use --default switch to use the default parameter values set in Tetrad. You can still override specific parameters with this switch.

f-arab commented 1 year ago

In the section named: Macaque-Based Networks results Here is the related paragraph:

"We choose tuning parameters for the algorithms according to the procedure described in Supporting Information B (Section B2, Sanchez-Romero et al., 2019). FASK was run with a penalty discount of c = 1 and α = 10-7 for the SmallDegree network; with penalty discount c = 1 and α = 10-1 for LongRange and LongRange Control; and with penalty discount c = 2 and α = 10-1 for the Full network."

On Thu, Jan 5, 2023 at 3:53 PM rubenSaro @.***> wrote:

Hi,

Glad the replication is working. Can you cite the section of the paper where we mentioned alpha 0.1 is the best for the data, just curious.

Ruben

On Thu, Jan 5, 2023, 6:15 PM Fahimeh Arab @.***> wrote:

HI @rubenSaro https://github.com/rubenSaro ,

Yes, you are right, When I'm changing the alpha value for macaque full data to 1e-7, that one becomes so fast as well in Tetrad. The reason that I tried macaque full data with alpha = 0.1 is that in your paper, it is mentioned that alpha = 0.1 gave the best results for that data, With small values of alpha, I see that FASK is super fast (when I'm running it through Tetrad), the precision very high, however, the recall is very small. So maybe, that's why you also tried higher values of alpha to increase the recall for the full macaque graph as well.

There is still one thing that I don't understand which is why the execution times are much higher when I'm using FASK with these hyperparameters on causal-cmd?!

By the way, is there any way of using Tetrad through a script or it should be done manually through the GUI? let's say we want to run FASK on Tetrad for all 60 repetitions of Full macaque data. What is the best way in that case?

Regarding the timing, I got for FASK while using causal-cmd for different values of alpha: (here are the results for data from simple networks 1 to 9)

[image: image] < https://user-images.githubusercontent.com/49805515/210898090-a49c517f-7825-4b56-9717-b30d236c7288.png

— Reply to this email directly, view it on GitHub <https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1372914385 , or unsubscribe < https://github.com/notifications/unsubscribe-auth/AJUZSFJEAKHHXQJOMLZI26DWQ5IX7ANCNFSM5YLID5IA

. You are receiving this because you were mentioned.Message ID: @.***>

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1372942223, or unsubscribe https://github.com/notifications/unsubscribe-auth/AL37RSZAZUTUFMS4YVXCLF3WQ5NHNANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

Thanks @rubenSaro . I'm running causal-cmd 1.4.1

rubenSaro commented 1 year ago

Hi, sorry for the delay in the response, I have been busy with other projects and holiday break.

Some of the FASK parameters got an upgrade/modification (relative to what appeared in the paper). I will let you know as soon as I get the correct new FASK parameters such that they are close to what was in the paper. I am trying with just one Full Macaque concatenated dataset (number 60)

Ruben

El jue, 5 ene 2023 a las 20:40, Fahimeh Arab @.***>) escribió:

Thanks @rubenSaro https://github.com/rubenSaro . I'm running causal-cmd 1.4.1

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1373025940, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFNPNM3U46KCP7OQ26DWQ5Z2LANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

rubenSaro commented 1 year ago

Also, I just noticed that the alpha = 0.1 for the Full Macaque data, mentioned in the paper, refers to an alpha used in the two-cycle detection step, while the FAS adjacency search step was done with a sem-bic-test with penalty discount of c = 2.

In your command-line fask runs, I am pretty sure the alpha is controlling another parameter (maybe an FAS z-score test), and not the alpha for the two-cycle detection step.

Ruben

El jue, 19 ene 2023 a las 23:41, Ruben Sanchez @.***>) escribió:

Hi, sorry for the delay in the response, I have been busy with other projects and holiday break.

Some of the FASK parameters got an upgrade/modification (relative to what appeared in the paper). I will let you know as soon as I get the correct new FASK parameters such that they are close to what was in the paper. I am trying with just one Full Macaque concatenated dataset (number 60)

Ruben

El jue, 5 ene 2023 a las 20:40, Fahimeh Arab @.***>) escribió:

Thanks @rubenSaro https://github.com/rubenSaro . I'm running causal-cmd 1.4.1

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1373025940, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFNPNM3U46KCP7OQ26DWQ5Z2LANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

Thanks, @rubenSaro. Oh, I see! I tried it now with alpha= 0.1 for the orientation part and 1e-7 for the FAS-stable and it is now fast (I actually tried it on Tetrad and I have to also see how to set these correctly on causal- cmd). Anyways, thanks for the clarification.

jdramsey commented 1 year ago

@f-arab @rubenSaro Is this issue closed? Can I close it?

rubenSaro commented 1 year ago

Not yet

On Wed, Feb 8, 2023, 1:30 AM Joseph Ramsey @.***> wrote:

@f-arab https://github.com/f-arab @rubenSaro https://github.com/rubenSaro Is this issue closed? Can I close it?

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1422083759, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFMDF2H56SRVZYQ7HQDWWM4R7ANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

rubenSaro commented 1 year ago

Hi @f-arab,

Sorry for the late response.

I was wondering how the implementation is going and also curious what is the overall goal of this work. As algorithm developers we are always interested in how these tools are used and how we can help users to make the most of them.

Also, I talked with @jdramsey some weeks ago about the differences between the most recent implementation of FASK and what are the new parameter names that map to those used in the paper implementation.

I highly recommend to check the FASK algorithm and all its parts here: https://tinyurl.com/2ku26aen It is the supplementary material in the paper. I refer to the algorithms that compose FASK in the notes below.

This is the list of parameters that FASK use in the command-line version you were using.

faskAdjacencyMethod: 1 # this run FAS-Stable (the one used in the paper). See Algorithm 2.

depth: -1. # control the size of the conditional set in the independence tests, setting this to a small integer may reduce the running time, but can also result in false positives. -1 means that it will check "all" possible sizes.

test: sem-bic-test # test for FAS adjacency

score: sem-bic-score

semBicRule: 1 # to set the Chickering Rule, used in the original Fask

penaltyDiscount: 2 # if using sem-bic as independence test (as in the paper). In the paper this is referred as c. Check step 1 and 10 in Algorithm 2 FAS stable.

skewEdgeThreshold: 0.3 # See description of Fask algorithm, and step 11 in Algorithm 1 FASK. Threshold to add edges that may have been non-inferred because there was a positive/negative cycle that result in a non-zero observed relation.

faskLeftRightRule: 1 # this run FASK v1, the original FASK from the paper

faskDelta: -0.3 # See step 1 and 11 in Algorithm 4 (this is the value set in the paper)

twoCycleScreeningThreshold: 0 # not used in the original paper implementation. Added afterwards. You can set it to 0.3, for example, to use it as a filter to run Algorithm 3 2-cycle detection, which may take some time to run.

orientationAlpha: 0.1 # this was referred in the paper as TwoCycle Alpha or just alpha, the lower it is, the lower the chance of inferring a two cycle. Check steps 17 to 28 in Algorithm 3: 2 Cycle Detection Rule.

structurePrior: 0 # prior on the number of parents. Not used in the paper implementation.

So a run of command line would look like this:

java -jar -Xmx10G causal-cmd-1.4.1-jar-with-dependencies.jar --delimiter tab --data-type continuous --dataset concat_BOLDfslfilter_60_FullMacaque.txt --prefix Fask_Test_MacaqueFull --algorithm fask --faskAdjacencyMethod 1 --depth -1 --test sem-bic-test --score sem-bic-score --semBicRule 1 --penaltyDiscount 2 --skewEdgeThreshold 0.3 --faskLeftRightRule 1 --faskDelta -0.3 --twoCycleScreeningThreshold 0 --orientationAlpha 0.1 -structurePrior 0

Hope this helps

@rubenSaro

jdramsey commented 1 year ago

@rubenSaro I'm going to put these param docs into the Fask docs...

jdramsey commented 1 year ago

Here:

https://github.com/cmu-phil/tetrad/blob/sundry_fixes_7_3_0/tetrad-lib/src/main/java/edu/cmu/tetrad/search/Fask.java

jdramsey commented 1 year ago

@f-arab @rubenSaro, Can this issue be closed yet? I wasn't sure if you were still working on it.

rubenSaro commented 1 year ago

I am good to close it.

On Thu, Mar 16, 2023, 1:45 AM Joseph Ramsey @.***> wrote:

@f-arab https://github.com/f-arab @rubenSaro https://github.com/rubenSaro, Can this issue be closed yet? I wasn't sure if you were still working on it.

— Reply to this email directly, view it on GitHub https://github.com/bd2kccd/causal-cmd/issues/69#issuecomment-1471461648, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJUZSFMW7SJNBEXJE4XZJJ3W4LAJBANCNFSM5YLID5IA . You are receiving this because you were mentioned.Message ID: @.***>

f-arab commented 1 year ago

Thanks @rubenSaro and @jdramsey . Sorry for my late reply. I'll try with the new parameter values and let you know how it goes. In the previous setting I was not able to run FASK (computationally) on a large graphs while setting the alpha to 0.1 which made the graphs denser. I'll try with these new settings.

I'm good to close it as well. I might come back at some point again :)

Thanks