Are you sure the code and data are consistent with the article?

[SZ-qing]

Hello, I encountered a few problems when using your data and code:

1. when i run your demo command, in the bulkmodel code, '--printgene=T' , The file cannot be saved because the directory you exported was not established in advance【‘np.savetxt("save/ori_result/’】I have add ['for path in [args.log,args.bulk_model,args.bulk_encoder,'save/ori_result','save/figures']:'] in line 72
2. The sampling Parameter of demo command line of the bulkmodel is 'SMOTE' , but in the scmodel is default 'None'. Causes your demo code to fail to run.
3. Run your demo GSE110894, the f1-score is about 0.60, this is very different from your article and source data 5 description. I tested the six hyperparameters described in paper, involving 480 combinations, and found that the results were far less than those in your paper.
4. Finally, I hope you can check your code again, please make sure you can reproduce the article, I think this is very important.

[SZ-qing]

And , According to the adata name you provided, I ran the GSE149383_drug_ERLOTINIB result with the parameters you provided, and found that the F1 value of scmodel was only 0.51, and the rocauc_score just 0.578:

==============

cat bulk_fGSE149383_f1_score_ori.txt integrate_data_GSE149383_drug_ERLOTINIB_bottle_512_edim_512,256_pdim_256,128_model_DAE_dropout_0.1_gene_T_lr_0.1_mod_new_sam_SMOTE 0.5107997511395633

==================

cat 2023-03-06-22-10-27scRNA_data_GSE149383_drug_ERLOTINIB_bottle_512_model_DAE_dropout_0.1_mod_new_sam_SMOTE_report.csv ,precision,recall,f1-score,support,auroc_score,ap_score 0,0.45116772823779194,0.6724683544303798,0.5400254129606099,632.0,0.5777688042076887,0.3971983302815359 1,0.6263537906137184,0.40162037037037035,0.4894217207334274,864.0,0.5777688042076887,0.3971983302815359 accuracy,0.516042780748663,0.516042780748663,0.516042780748663,0.516042780748663,0.5777688042076887,0.3971983302815359 macro avg,0.5387607594257552,0.5370443624003751,0.5147235668470187,1496.0,0.5777688042076887,0.3971983302815359 weighted avg,0.5523447054388617,0.516042780748663,0.5107997511395633,1496.0,0.5777688042076887,0.3971983302815359

==================

python bulkmodel.py --drug 'ERLOTINIB' --dimreduce 'DAE' --encoder_h_dims "512,256" --predictor_h_dims "256,128" --bottleneck 512 --data_name 'GSE149383' --dropout 0.1 --lr 0.1 --sampling 'SMOTE' --printgene 'T' --mod 'new'

python scmodel_test.py --sc_data 'GSE149383' --dimreduce 'DAE' --drug 'ERLOTINIB' --bulk_h_dims "512,256" --bottleneck 512 --predictor_h_dims "256,128" --dropout 0.1 --lr 0.1 --sampling 'SMOTE' --printgene 'T' -mod 'new' Do you have time to explain? Do you have time to explain or reproduce the results in your paper based on the code you currently provide? Thanks!

[SZ-qing]

@juychen @PegasusAM @OSU-BMBL-admin Hope to get your help and answer the doubt.

[SZ-qing]

For the dataset GSE112274-GEFITINIB,

~/anaconda3/envs/scdeal/bin/python bulkmodel.py --drug 'GEFITINIB' --dimreduce 'DAE' --encoder_h_dims "512,256" --predictor_h_dims "256,128" --bottleneck 64 --data_name 'GSE112274' --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' --mod 'new'

~/anaconda3/envs/scdeal/bin/python scmodel_test.py --sc_data 'GSE112274' --dimreduce 'DAE' --drug 'GEFITINIB' --bulk_h_dims "512,256" --bottleneck 64 --predictor_h_dims "256,128" --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' -mod 'new'

In the bulkmodel : auc=0.89 but in the scmodel is 0.005

[juychen]

Hi, thanks for your suggestions and questions. We are fixing the bugs and seeking for help from my colleague about the results issue.

[tb1over]

For the dataset GSE112274-GEFITINIB,

~/anaconda3/envs/scdeal/bin/python bulkmodel.py --drug 'GEFITINIB' --dimreduce 'DAE' --encoder_h_dims "512,256" --predictor_h_dims "256,128" --bottleneck 64 --data_name 'GSE112274' --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' --mod 'new'

~/anaconda3/envs/scdeal/bin/python scmodel_test.py --sc_data 'GSE112274' --dimreduce 'DAE' --drug 'GEFITINIB' --bulk_h_dims "512,256" --bottleneck 64 --predictor_h_dims "256,128" --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' -mod 'new'

In the bulkmodel : auc=0.89 but in the scmodel is 0.005

Did the author give an explanation of the results ？

[juychen]

For the dataset GSE112274-GEFITINIB,

~/anaconda3/envs/scdeal/bin/python bulkmodel.py --drug 'GEFITINIB' --dimreduce 'DAE' --encoder_h_dims "512,256" --predictor_h_dims "256,128" --bottleneck 64 --data_name 'GSE112274' --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' --mod 'new'

~/anaconda3/envs/scdeal/bin/python scmodel_test.py --sc_data 'GSE112274' --dimreduce 'DAE' --drug 'GEFITINIB' --bulk_h_dims "512,256" --bottleneck 64 --predictor_h_dims "256,128" --dropout 0.1 --lr 0.1 --sampling 'no' --printgene 'T' -mod 'new'

In the bulkmodel : auc=0.89 but in the scmodel is 0.005

Did the author give an explanation of the results ？

Hi, we are now testing the environment applied to generate the results. We will release the corresponding packaged environment, and the model weights soon.