Shape Bug in Graft

[zhanwenchen]

🐛 Bug

To Reproduce

Steps to reproduce the behavior:

1. 1. 1.

Expected behavior

Environment

Please copy and paste the output from the environment collection script from PyTorch (or fill out the checklist below manually).

You can get the script and run it with:

wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py

• PyTorch Version (e.g., 1.0):
• OS (e.g., Linux):
• How you installed PyTorch (conda, pip, source):
• Build command you used (if compiling from source):
• Python version:
• CUDA/cuDNN version:
• GPU models and configuration:
• Any other relevant information:

Additional context

[zhanwenchen]

It's not because of the random seed.

[zhanwenchen]

Let's debug from the df side

[zhanwenchen]

Also, we are saving the result to the object.

[zhanwenchen]

Maybe it's because generator or distributed (in other words, cuda=0 in legion vs on lab)?

[zhanwenchen]

Maybe implement a singleton object for vgstats

[zhanwenchen]

There is now a problem of PIL images in the dataset.

[zhanwenchen]

Is there a reason why I didn't use the dataloader dataset for the stats. Is it because of the transforms?

[zhanwenchen]

The PIL images in the dataset didn't come from getitem of the graft augmented. It's not run() either.

[zhanwenchen]

Maybe need to deepcopy dataset because it's set to None? That's it!!!! Deepcopy in graftaugmenter worked!

[zhanwenchen]

Keep running to see if there's any more sample mismatch issue.

[zhanwenchen]

There was a NotImplementedError after 400 iterations. That was probably because the graftdataset had a commented out getitem for debugging purposes.

[zhanwenchen]

Closed by #50 #86