search

tim-learn / SHOT

code released for our ICML 2020 paper "Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation"
MIT License
437 stars 78 forks source link

Confusion in generating ".txt" files for datasets aside from "Digits", and placement of datasets #8

Closed HamzaJawad98 closed 3 years ago

HamzaJawad98 commented 3 years ago

Hello, I am confused in regards to the generation of .txt files for training on all manually downloaded datasets.

Based on what I understood from this thread: https://github.com/tim-learn/SHOT/issues/2

I wrote a script to generate text files for the "office" and "office-home" dataset, the folders for which are contained in my Master directory. So, for example, a sample file for my office-home dataset would be placed in "Shot-Master/object/data/office-home/Art/Alarm_Clock/00001.jpg".

My generated Art_List file resides in "Shot-Master/object/data/office-home", and a sample line in it reads: "----Path from root to shot-master----/object/data/office-home/Art/Alarm_Clock/00001.jpg 0"

The code for point 2, when run, throws the error: The syntax of the command is incorrect. Traceback (most recent call last): File "image_source.py", line 371, in os.mkdir(args.output_dir_src) FileNotFoundError: [WinError 3] The system cannot find the path specified: 'ckps/source/uda\office\A'

In short, I am confused regarding the placement of the datasets and text file generations. Please advise.

tim-learn commented 3 years ago

Hi, I think you need to train the source model first, then adapt the pre-trained model under "./ckps/source/..." to one target domain.

tim-learn commented 3 years ago

Hello, I am confused in regards to the generation of .txt files for training on all manually downloaded datasets.

Based on what I understood from this thread: #2

I wrote a script to generate text files for the "office" and "office-home" dataset, the folders for which are contained in my Master directory. So, for example, a sample file for my office-home dataset would be placed in "Shot-Master/object/data/office-home/Art/Alarm_Clock/00001.jpg".

My generated Art_List file resides in "Shot-Master/object/data/office-home", and a sample line in it reads: "----Path from root to shot-master----/object/data/office-home/Art/Alarm_Clock/00001.jpg 0"

The code for point 2, when run, throws the error: The syntax of the command is incorrect. Traceback (most recent call last): File "image_source.py", line 371, in os.mkdir(args.output_dir_src) FileNotFoundError: [WinError 3] The system cannot find the path specified: 'ckps/source/uda\office\A'

In short, I am confused regarding the placement of the datasets and text file generations. Please advise.

Sorry for the misunderstanding, it seems that you do not have the access to create a folder called "./ckps" ? maybe difference between "ckps" and "./ckps" ? you can assign "./ckps" to the output key.

HamzaJawad98 commented 3 years ago

Hey, thanks for the response! I managed to set up ckps in a separate directory (ran the command _python image_source.py --trte val --da uda --output D:\temp\ckps\source\ --gpu_id 0 --dset office --maxepoch 100 --s 0) but now another issue has cropped up: File "image_source.py", line 218, in train_source inputs_source, labels_source = iter_source.next() UnboundLocalError: local variable 'iter_source' referenced before assignment

During handling of the above exception, another exception occurred: Traceback (most recent call last): . . . . FileNotFoundError: [Errno 2] No such file or directory: 'D:/temp/SHOT-master/object/data/office/images/amazon/desk_lamp/frame_0058.jpg'

Even though this path is exactly where the stated image exists

In short, I'm just confused as to the creation and placement of the mentioned text files. Just to clarify again, this is where my txt files exist: image

Sample lines in the first file read as such: image

The datasets are located in the exact corresponding locations that are mentioned in these .txt files above^

Could you please guide me in regards to this? Thanks.

tim-learn commented 3 years ago

Hey, thanks for the response! I managed to set up ckps in a separate directory (ran the command _python image_source.py --trte val --da uda --output D:\temp\ckps\source\ --gpu_id 0 --dset office --maxepoch 100 --s 0) but now another issue has cropped up: File "image_source.py", line 218, in train_source inputs_source, labels_source = iter_source.next() UnboundLocalError: local variable 'iter_source' referenced before assignment

During handling of the above exception, another exception occurred: Traceback (most recent call last): . . . . FileNotFoundError: [Errno 2] No such file or directory: 'D:/temp/SHOT-master/object/data/office/images/amazon/desk_lamp/frame_0058.jpg'

Even though this path is exactly where the stated image exists

In short, I'm just confused as to the creation and placement of the mentioned text files. Just to clarify again, this is where my txt files exist: image

Sample lines in the first file read as such: image

The datasets are located in the exact corresponding locations that are mentioned in these .txt files above^

Could you please guide me in regards to this? Thanks.

I think you may mistaken the path, "office/amazon/images" not "office/images/amazon"....

HamzaJawad98 commented 3 years ago

Yes, you are right. My bad. Just checking into further issues that appear as I progress (CUDA out of mem rn). I'll keep you updated, and thanks for the prompt responses.