How to modify source code for _MapDatasetFetcher

[xingyifei2016]

When I run the train code (as in train_FKD.py) as provided, it errors on the getitem method of ImageFolder_FKD_MIX not having the corresponding batch_config. I understand that this config is loaded via load_batch_config, which is the reason why _MapDatasetFetcher is modified in the first place. However, the docs do not provide an actual procedure except some code. How do I actually overwrite the original pytorch source code to become the new code provided? Do I need to run a locally compiled version of pytorch for this to work? Are there solutions that only requires a few additional lines for it to work? Thank you.

[zeyuanyin]

Thanks for your interest in this project!

It is not difficult to modify the PyTorch source code. Here are some quick tips,

• In VSCode, select Python interpreter in the correct environment where you will run the code later
• Go to train/train_FKD.py, and follow the below instruction to access the PyTorch source code https://github.com/VILA-Lab/SRe2L/blob/25591ea49da866d04222eef271c8886f02839a71/train/train_FKD.py#L23-L24
• Once you enter the source code, replace _MapDatasetFetcher with the modified version provided in train/README.md and save the file

Kindly note that the modification operation will not have any adverse effects on other PyTorch workflows, and you can continue to run other codes as usual.

[xingyifei2016]

Thank you for the reply! However, if I cannot access VS code, as I am using an online server and only have access to jupyter notebook, how would I modify the source code?

[zeyuanyin]

If you are not using VSCode or PyCharm, you cannot quickly navigate to the source file fetch.py by using Ctrl+Click.

To retrieve the location of fetch.py and proceed with modifications, you can utilize the following code. Once the location is obtained, you can navigate to the file for editing.

import torch.utils.data._utils.fetch as fetch
print(fetch.__file__)

# my output: /home/zeyuan.yin/.conda/envs/py38/lib/python3.8/site-packages/torch/utils/data/_utils/fetch.py 
# `py38` is my current conda environment name.

[xingyifei2016]

Thank you so much zeyuan! That answers my question. :)