search

MiraldiLab / maxATAC

Transcription Factor Binding Prediction from ATAC-seq and scATAC-seq with Deep Neural Networks
Apache License 2.0
25 stars 8 forks source link

Epoch Selection: Performance using chr 2 Training Cell Line AUPR vs Performance using Validation Dice Coefficient #35

Closed FaizRizvi closed 3 years ago

FaizRizvi commented 3 years ago

We attempted to simplify our method to choosing the best epoch from validation. Previously we are choosing the best epoch based on performance on chr2 in training cell lines.

BHLHE40_chr2_AUPR_performance_all_epoch_200_bp

This method demands a lot of resources to implement and takes quite a bit of time. We propose to use the epoch with the highest chr2 validation:

BHLHE40_RR30_revcomp_model_dice_acc

Then we benchmark in chr 1 and GM12878 and compare the performance for both of these methods:

Compare_validation_approaches

Compare_validation_approaches_boxplot

We can get fairly comparable results by using either method