Pretraining with my own dataset

[alskdjfasdfsadf]

Hi, thanks for your work. Whenever I try to pretrain with my own dataset, following error is happening:

torchrun --data_path=/home/user/augdata --exp_name=ptaugdata --exp_dir=/home/user/models usage: torchrun [-h] [--nnodes NNODES] [--nproc-per-node NPROC_PER_NODE] [--rdzv-backend RDZV_BACKEND] [--rdzv-endpoint RDZV_ENDPOINT] [--rdzv-id RDZV_ID] [--rdzv-conf RDZV_CONF] [--standalone] [--max-restarts MAX_RESTARTS] [--monitor-interval MONITOR_INTERVAL] [--start-method {spawn,fork,forkserver}] [--role ROLE] [-m] [--no-python] [--run-path] [--log-dir LOG_DIR] [-r REDIRECTS] [-t TEE] [--node-rank NODE_RANK] [--master-addr MASTER_ADDR] [--master-port MASTER_PORT] [--local-addr LOCAL_ADDR] training_script ... torchrun: error: the following arguments are required: training_script, training_script_args

do I need to specify training_script and training_script_args?

[keyu-tian]

Yes, you can refer to /pretrain/README.md for the complete cmd, which is:

$ cd /path/to/SparK/pretrain
$ torchrun --nproc_per_node=8 --nnodes=1 --node_rank=0 --master_addr=localhost --master_port=<some_port> main.py \
  --data_path=/path/to/imagenet --exp_name=<your_exp_name> --exp_dir=/path/to/logdir \
  --model=resnet50 --bs=512

The first line is missing, e.g. main.py is the training_script.

[alskdjfasdfsadf]

Thank you very much. In the article, it is saying that 'All models are pre-trained with 1.28 million unlabeled images from ImageNet-1K (Deng et al., 2009) training set for 1600 epochs.', which means that I can use SparK pretraining with unlabeled dataset. However, when I tried to use unlabeled image dataset , the following error is happening: File "/home/user/SparK/pretrain/utils/imagenet.py", line 39, in init super(ImageNetDataset, self).init( File "/home/user/.local/lib/python3.10/site-packages/torchvision/datasets/folder.py", line 144, in init classes, class_to_idx = self.find_classes(self.root) File "/home/user/.local/lib/python3.10/site-packages/torchvision/datasets/folder.py", line 218, in find_classes return find_classes(directory) File "/home/user/.local/lib/python3.10/site-packages/torchvision/datasets/folder.py", line 42, in find_classes raise FileNotFoundError(f"Couldn't find any class folder in {directory}.")

[keyu-tian]

You need to define a new Python class for your dataset, to replace our ImageNetDataset in https://github.com/keyu-tian/SparK/blob/main/pretrain/utils/imagenet.py#L30. Just define a class with __len__(self) and __getitem__(self, index: int) implemented. The getitem should return the index-th image in your dataset, and be processed by a transformation like the trans_train in /pretrain/utils/imagenet.py.

PS: i recommend to try your pretraining with or without --init_weight=/path/to/res50_withdecoder_1kpretrained_spark_style.pth. If this arg is used, you will pretrain from our pretrained model, rather than from scratch, which could be better.