search

ahmedbesbes / mrnet

Building an ACL tear detector to spot knee injuries from MRIs with PyTorch (MRNet)
https://ahmedbesbes.com
MIT License
125 stars 38 forks source link

I can not reproduce your train AUC, & valid AUC scores #1

Open linhduongtuan opened 5 years ago

linhduongtuan commented 5 years ago

Dear Besbes, Firstly, I thank for your sharing code. It is very nice. I test with sagittal dataset with anterior cruciate ligament tear. However, I could not reproduce your results like train & valid AUC scores. So could you explain to clarify the issue? Or maybe I acquire a mistake somewhere? Best regard. Linh

ahmedbesbes commented 5 years ago

Hello, what scores did you get? Have you downloaded the right MRNet dataset? Did you run the code properly (as indicated in the readme file)

linhduongtuan commented 5 years ago

My AUC scores for training and validating are 75% and 77%, respectively. I use the dataset from Stanford. I also know that another MRNet dataset comes from Croatia. I have read your code and blog throughout and hope no mistake occurs during my implementation Kindly

ahmedbesbes commented 5 years ago

The scores you are mentioning, are they related to a specific plane or to the global (stacked) model?

linhduongtuan commented 5 years ago

I just tested only ACL diagnosis with Sagittal plane. Because I guess the plane has the most important tasks for the diagnosis, especially for ACL. And this condition should be the highest AUC score? I've read both papers from Stanford Uni (first author: Nicholas Bien) and Croatia + Turkey group. I know you try to reproduce the experiments of Stanford Uni paper. Do you have any further suggestion? Best regards. Linh

ahmedbesbes commented 5 years ago

Can you share the code? or the hyperparameters you used?

linhduongtuan commented 5 years ago

I don't have HPC so I use Google Colab to reproduce your experiment! I don't know how do authors of the paper (https://doi. org/10.1371/journal.pmed.1002699) do exact to get their AUC scores? I have test certain codes following the paper. However, I have never seen they reach the scores. Here your code I manipulate to use Google Colab: https://drive.google.com/file/d/1UbzDfytwy6fY1iykkzU5xhY08ioEoMTZ/view?usp=sharing Best regards Linh

ParthTrehan commented 4 years ago

yeah, i also could not reproduce the AUCs. I saw that you are optimizing AUCs but in the papers, they are minimizing the error rate

therc01 commented 3 years ago

I don't have HPC so I use Google Colab to reproduce your experiment! I don't know how do authors of the paper (https://doi. org/10.1371/journal.pmed.1002699) do exact to get their AUC scores? I have test certain codes following the paper. However, I have never seen they reach the scores. Here your code I manipulate to use Google Colab: https://drive.google.com/file/d/1UbzDfytwy6fY1iykkzU5xhY08ioEoMTZ/view?usp=sharing Best regards Linh

Hi, im getting an error while building global acl classifier. I used the code from your blog for that. Im getting error in this loop ` predictions = []

labels = []

with torch.no_grad():

     for image, label, _ in tqdm_notebook(train_loader):
        logit = mrnet(image.cuda())
        prediction = torch.sigmoid(logit)
        predictions.append(prediction.item())
        labels.append(label.item())`

particularly in prediction.item() i used np.squeeze() to achieve the same. I did same for label.item(). It gets resolve here.

now its throwing error at ` X = np.zeros((len(predictions), 3))

X[:, 0] = results['axial'] X[:, 1] = results['coronal'] X[:, 2] = results['sagittal']`

values are not getting assigned here to X. What do i do now? What im doing wrong here.