search

gsig / actor-observer

ActorObserverNet code in PyTorch from "Actor and Observer: Joint Modeling of First and Third-Person Videos", CVPR 2018
GNU General Public License v3.0
76 stars 9 forks source link

Some questions about code #3

Open yuhuangyue opened 5 years ago

yuhuangyue commented 5 years ago

The command I used is python third_to_first_person.py. and then with some problems

  1. the CharadesEgo_val_video part always 0 samples loaded?

cachefile ./caches/third_to_first_person//CharadesEgo_train.pkl Loading cached result from './caches/third_to_first_person//CharadesEgo_train.pkl' 516965 samples loaded cachefile ./caches/third_to_first_person//CharadesEgo_val.pkl Loading cached result from './caches/third_to_first_person//CharadesEgo_val.pkl' 137321 samples loaded cachefile ./caches/third_to_first_person//CharadesEgo_val_video.pkl Loading cached result from './caches/third_to_first_person//CharadesEgo_val_video.pkl' 0 samples loaded

  1. Which dataset does the code use? CharadesEgo_v1_rgb or Charades_v1_rgb?

Thank you in advance!

gsig commented 5 years ago

CharadesEgo_val_video uses Charades_v1_rgb, but the rest uses CharadesEgo_v1_rgb.

Make sure you delete the cache at ./caches/third_to_first_person//CharadesEgo_val_video.pkl before trying again.

Let me know if that works!

yuhuangyue commented 5 years ago

Thanks for your reply! I modify the dataset path, but still can't load it. when the command I used is python third_to_first_person.py, the result is 2018-11-30 10:49:15,052:Loading cached result from './caches/third_to_first_person//CharadesEgo_train.pkl' 2018-11-30 10:49:43,658:516965 samples loaded 2018-11-30 10:50:16,245:cachefile ./caches/third_to_first_person//CharadesEgo_val.pkl 2018-11-30 10:50:16,246:Loading cached result from './caches/third_to_first_person//CharadesEgo_val.pkl' 2018-11-30 10:50:24,403:137321 samples loaded 2018-11-30 10:50:49,259:cachefile ./caches/third_to_first_person//CharadesEgo_val_video.pkl 0 samples loaded

when the command I used is python alignment_and_zeroshot.py, the result is cachefile ./caches/alignment_and_zeroshot//CharadesEgo_train.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgo_train.pkl' 516965 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesEgo_val.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgo_val.pkl' 137321 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesEgo_val_video.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgo_val_video.pkl' 0 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesEgoPlusRGB_train.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgoPlusRGB_train.pkl' 3190776 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesEgoPlusRGB_val.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgoPlusRGB_val.pkl' 1021851 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesEgoPlusRGB_val_video.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesEgoPlusRGB_val_video.pkl' 40175 samples loaded cachefile ./caches/alignment_and_zeroshot//CharadesMeta_val_video.pkl Loading cached result from './caches/alignment_and_zeroshot//CharadesMeta_val_video.pkl' 2575 samples loaded

The Charades Ego_val_videos has not been loaded correctly. Is there something wrong with my path modification? here is my opt.py def parse(): print('parsing arguments') parser = argparse.ArgumentParser(description='PyTorch Charades-Ego Training') parser.add_argument('--data', metavar='DIR', default='/home/yhy/actor-observer-master/datasets/Charades_v1_rgb', help='path to dataset') parser.add_argument('--dataset', default='fake', help='name of dataset under datasets/') parser.add_argument('--egocentric-test-data', default='/home/yhy/actor-observer-master/datasets/labels/CharadesEgo_v0_egocentric_test.csv', help='path to labels for egocentric classification') parser.add_argument('--original-charades-train', default='/home/yhy/actor-observer-master/datasets/labels/Charades_v1_train.csv', help='Original Charades Train') parser.add_argument('--original-charades-test', default='/home/yhy/actor-observer-master/datasets/labels/Charades_v1_test.csv', help='Original Charades Test') parser.add_argument('--original-charades-data', default='/home/yhy/actor-observer-master/datasets/Charades_v1_rgb', help='path to original charades frames') parser.add_argument('--train-file', default='/home/yhy/actor-observer-master/datasets/labels/CharadesEgo_v1_train.csv', type=str) parser.add_argument('--val-file', default='/home/yhy/actor-observer-master/datasets/labels/CharadesEgo_v1_test.csv', type=str) parser.add_argument('--arch', '-a', metavar='ARCH', default='alexnet', help='model architecture: ') parser.add_argument('--subarch', default='alexnet') parser.add_argument('--subloss', default='MarginRank') parser.add_argument('--loss', default='CrossEntropyLoss') parser.add_argument('--workers', default=4, type=int, metavar='N', help='# data loading workers (default: 4)') parser.add_argument('--epochs', default=20, type=int, metavar='N', help='number of total epochs to run') parser.add_argument('--start-epoch', default=0, type=int, metavar='N', help='manual epoch number') parser.add_argument('--batch-size', default=256, type=int, metavar='N', help='mini-batch size (default: 256)') parser.add_argument('--lr', '--learning-rate', default=1e-3, type=float, metavar='LR', help='initial learning rate') parser.add_argument('--lr-decay-rate', default=6, type=int) parser.add_argument('--momentum', default=0.9, type=float, metavar='M', help='momentum') parser.add_argument('--decay', default=0.9, type=float) parser.add_argument('--finaldecay', default=0.9, type=float) parser.add_argument('--margin', default=0.0, type=float) parser.add_argument('--clsweight', default=1.0, type=float) parser.add_argument('--metric', default='wtop1val', help='metric to find best model') parser.add_argument('--weight-decay', '--wd', default=1e-4, type=float, metavar='W', help='weight decay (1e-4)') parser.add_argument('--print-freq', '-p', default=10, type=int, metavar='N', help='print frequency (10)') parser.add_argument('--resume', default='', type=str, metavar='PATH', help='path to latest checkpoint (none)') parser.add_argument('--evaluate', dest='evaluate', action='store_true', help='evaluate on val sets') parser.add_argument('--pretrained', dest='pretrained', action='store_true', help='use pre-trained model') parser.add_argument('--no-logger', dest='no_logger', action='store_true') parser.add_argument('--cache-buster', dest='cache_buster', action='store_true') parser.add_argument('--valvideo', dest='valvideo', action='store_true') parser.add_argument('--valvideoego', dest='valvideoego', action='store_true') parser.add_argument('--alignment', dest='alignment', action='store_true') parser.add_argument('--usersalignment', dest='usersalignment', action='store_true') parser.add_argument('--nopdb', dest='nopdb', action='store_true') parser.add_argument('--pretrained-weights', default='', type=str) parser.add_argument('--pretrained-subweights', default='', type=str) parser.add_argument('--inputsize', default=224, type=int) parser.add_argument('--world-size', default=1, type=int, help='number of distributed processes') parser.add_argument('--manual-seed', default=0, type=int) parser.add_argument('--dist-url', default='tcp://224.66.41.62:23456', type=str, help='url for distributed training') parser.add_argument('--dist-backend', default='gloo', type=str, help='distributed backend') parser.add_argument('--train-size', default=1.0, type=float) parser.add_argument('--val-size', default=1.0, type=float) parser.add_argument('--cache-dir', default='./cache/', type=str) parser.add_argument('--name', default='test', type=str) parser.add_argument('--nclass', default=157, type=int) parser.add_argument('--accum-grad', default=4, type=int) args = parser.parse_args() args.distributed = args.world_size > 1 args.cache = args.cache_dir + args.name + '/' if not os.path.exists(args.cache): os.makedirs(args.cache)

return args
gsig commented 5 years ago

Is this actually causing your code to crash? I was looking over the code, and I remember val_video is not being used for charadesego. In the paper the was no task for converting "video to video" completely, so it is all just frame to frame which can be used with standard train/val setup.

See https://github.com/gsig/actor-observer/blob/99a81f59d3ff9dd623982cb3f42899f6b96a5e43/datasets/charadesego.py#L42

gsig commented 5 years ago

Also I'm attaching a log from a recent third_to_first run, that you can compare against: https://www.dropbox.com/s/x81bv8mwy6s774z/third2first_log.txt?dl=0

yuhuangyue commented 5 years ago

Thank you for your detailed answer! Actually, this error doesn't crash my code, I just wonder why it's zero here. I'm running the alignment_and_zeroshot. py , and I'll keep in touch with you if I have any questions! Thank you again for your kind help!