The Evaluation Results in CAMPUS GARDEN2 Dataset

sontaptrung commented 4 months ago

Hello @chengche6230,

I trust this message reaches you in good spirits. To start, I'd like to convey my gratitude for your outstanding contributions. Your dedication is truly appreciated.

I have conducted an evaluation on the CAMPUS - GARDEN2 dataset on Website and rename file in dataset followed format:

view-HC1.txt -> view-HC0.txt view-HC2.txt -> view-HC1.txt view-HC3.txt -> view-HC2.txt view-HC4.txt -> view-HC3.txt

with the following configuration:

MODEL:
  DEVICE: "cuda"
  DEVICE_ID: ('0')
  MODE: 'test'
  DETECTION: 'gt' # {'gt'}
  RESUME: false
  LAST_CKPT_FILE: './'

DATASET:
  DIR: './datasets/'
  NAME: 'CAMPUS'
  SEQUENCE: ['Garden2'] # {'Garden1' | 'Garden2' | 'Parkinglot'}
  CAMS: 4
  TOTAL_FRAMES: 6000 # 'Garden1':2849, 'Garden2': 6000, 'Parkinglot': 6475

FE: # Feature Extractor
  CHOICE: 'CNN'
  INPUT_SIZE: (256, 128)

SOLVER:
  TYPE: 'SG' # {'SG': Spatial Graph, 'TG': Temporal Graph}
  EPOCHS: 100
  EVAL_EPOCH: 2
  BATCH_SIZE: 100
  LR: 0.01 # Learning Rate
  MAX_PASSING_STEPS: 4
  W: 3 # Temporal Window Size
  W_TEST: 5
  FOCAL_ALPHA: 0.98
  FOCAL_GAMMA: 5

OUTPUT:
  VISUALIZE: False
  LOG: True
  CKPT_DIR: './logs/ckpts'
  INFERENCE_DIR: './logs/inference'

TEST:
  CKPT_FILE_SG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_SG_epoch12_train80.pth'
  CKPT_FILE_TG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_TG_epoch6_train80.pth'
  FRAME_START: 4800 # 'Garden1': 2280; 'Garden2': 4800; 'Parkinglot': 5828
  EDGE_THRESH: 0.9

I obtained the following results: when compared to the results in your paper: \ I notice differences between the obtained metrics. Could I have missed something in the evaluation process?

chengche6230 commented 4 months ago

Hi,

I'm not sure what problem occurred indeed. Check if the homography matrix is correct, the debugging pictures in the log folder is somehow reasonable. I can get the same result using the provided trained weights.

bboyay commented 3 months ago

Is that how you put the data?请问数据是这么放吗？微信图片_20240614121614 Why can't I get the processed frames？为什么我无法得到处理后的frames？微信图片_20240614121836

jingliang0910 commented 3 months ago

Is that how you put the data?请问数据是这么放吗？ Why can't I get the processed frames？为什么我无法得到处理后的frames？

嗨，对于这个问题，需要在preprocess文件中调用对视频处理函数加入到process函数中

WangzcBruce commented 1 month ago

Hello @chengche6230,

I trust this message reaches you in good spirits. To start, I'd like to convey my gratitude for your outstanding contributions. Your dedication is truly appreciated.

I have conducted an evaluation on the CAMPUS - GARDEN2 dataset on Website and rename file in dataset followed format:

view-HC1.txt -> view-HC0.txt view-HC2.txt -> view-HC1.txt view-HC3.txt -> view-HC2.txt view-HC4.txt -> view-HC3.txt

with the following configuration:
MODEL:
  DEVICE: "cuda"
  DEVICE_ID: ('0')
  MODE: 'test'
  DETECTION: 'gt' # {'gt'}
  RESUME: false
  LAST_CKPT_FILE: './'

DATASET:
  DIR: './datasets/'
  NAME: 'CAMPUS'
  SEQUENCE: ['Garden2'] # {'Garden1' | 'Garden2' | 'Parkinglot'}
  CAMS: 4
  TOTAL_FRAMES: 6000 # 'Garden1':2849, 'Garden2': 6000, 'Parkinglot': 6475

FE: # Feature Extractor
  CHOICE: 'CNN'
  INPUT_SIZE: (256, 128)

SOLVER:
  TYPE: 'SG' # {'SG': Spatial Graph, 'TG': Temporal Graph}
  EPOCHS: 100
  EVAL_EPOCH: 2
  BATCH_SIZE: 100
  LR: 0.01 # Learning Rate
  MAX_PASSING_STEPS: 4
  W: 3 # Temporal Window Size
  W_TEST: 5
  FOCAL_ALPHA: 0.98
  FOCAL_GAMMA: 5

OUTPUT:
  VISUALIZE: False
  LOG: True
  CKPT_DIR: './logs/ckpts'
  INFERENCE_DIR: './logs/inference'

TEST:
  CKPT_FILE_SG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_SG_epoch12_train80.pth'
  CKPT_FILE_TG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_TG_epoch6_train80.pth'
  FRAME_START: 4800 # 'Garden1': 2280; 'Garden2': 4800; 'Parkinglot': 5828
  EDGE_THRESH: 0.9
I obtained the following results: when compared to the results in your paper: I notice differences between the obtained metrics. Could I have missed something in the evaluation process?

Hello, have you fixed this problem? I also met it.

WangzcBruce commented 1 month ago

MODEL: DEVICE: "cuda" DEVICE_ID: ('0') MODE: 'test' DETECTION: 'gt' # {'gt'} RESUME: false LAST_CKPT_FILE: './'

DATASET: DIR: './datasets/' NAME: 'CAMPUS' SEQUENCE: ['Garden2'] # {'Garden1' | 'Garden2' | 'Parkinglot'} CAMS: 4 TOTAL_FRAMES: 6000 # 'Garden1':2849, 'Garden2': 6000, 'Parkinglot': 6475

FE: # Feature Extractor CHOICE: 'CNN' INPUT_SIZE: (256, 128)

SOLVER: TYPE: 'TG' # {'SG': Spatial Graph, 'TG': Temporal Graph} EPOCHS: 100 EVAL_EPOCH: 2 BATCH_SIZE: 100 LR: 0.01 # Learning Rate MAX_PASSING_STEPS: 4 W: 3 # Temporal Window Size W_TEST: 5 FOCAL_ALPHA: 0.98 FOCAL_GAMMA: 5

OUTPUT: VISUALIZE: False LOG: True CKPT_DIR: './logs/ckpts' INFERENCE_DIR: './logs/inference'

TEST: CKPT_FILE_SG: "/mnt/data/wzc/ReST-main/logs/ckpts/CAMPUS_Garden2_SG_epoch12_train80.pth" CKPT_FILE_TG: "/mnt/data/wzc/ReST-main/logs/ckpts/CAMPUS_Garden2_TG_epoch6_train80.pth" FRAME_START: 4800 # 'Garden1': 2280; 'Garden2': 4800; 'Parkinglot': 5828 EDGE_THRESH: 0.9

Hello @chengche6230,

I trust this message reaches you in good spirits. To start, I'd like to convey my gratitude for your outstanding contributions. Your dedication is truly appreciated.

I have conducted an evaluation on the CAMPUS - GARDEN2 dataset on Website and rename file in dataset followed format:

view-HC1.txt -> view-HC0.txt view-HC2.txt -> view-HC1.txt view-HC3.txt -> view-HC2.txt view-HC4.txt -> view-HC3.txt

with the following configuration:
MODEL:
  DEVICE: "cuda"
  DEVICE_ID: ('0')
  MODE: 'test'
  DETECTION: 'gt' # {'gt'}
  RESUME: false
  LAST_CKPT_FILE: './'

DATASET:
  DIR: './datasets/'
  NAME: 'CAMPUS'
  SEQUENCE: ['Garden2'] # {'Garden1' | 'Garden2' | 'Parkinglot'}
  CAMS: 4
  TOTAL_FRAMES: 6000 # 'Garden1':2849, 'Garden2': 6000, 'Parkinglot': 6475

FE: # Feature Extractor
  CHOICE: 'CNN'
  INPUT_SIZE: (256, 128)

SOLVER:
  TYPE: 'SG' # {'SG': Spatial Graph, 'TG': Temporal Graph}
  EPOCHS: 100
  EVAL_EPOCH: 2
  BATCH_SIZE: 100
  LR: 0.01 # Learning Rate
  MAX_PASSING_STEPS: 4
  W: 3 # Temporal Window Size
  W_TEST: 5
  FOCAL_ALPHA: 0.98
  FOCAL_GAMMA: 5

OUTPUT:
  VISUALIZE: False
  LOG: True
  CKPT_DIR: './logs/ckpts'
  INFERENCE_DIR: './logs/inference'

TEST:
  CKPT_FILE_SG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_SG_epoch12_train80.pth'
  CKPT_FILE_TG: '/content/drive/MyDrive/MTMOT/ReST/weights/CAMPUS_Garden2_TG_epoch6_train80.pth'
  FRAME_START: 4800 # 'Garden1': 2280; 'Garden2': 4800; 'Parkinglot': 5828
  EDGE_THRESH: 0.9
I obtained the following results: when compared to the results in your paper: I notice differences between the obtained metrics. Could I have missed something in the evaluation process?

The following one is the corresponding config: MODEL: DEVICE: "cuda" DEVICE_ID: ('0') MODE: 'test' DETECTION: 'gt' # {'gt'} RESUME: false LAST_CKPT_FILE: './'

DATASET: DIR: './datasets/' NAME: 'CAMPUS' SEQUENCE: ['Garden2'] # {'Garden1' | 'Garden2' | 'Parkinglot'} CAMS: 4 TOTAL_FRAMES: 6000 # 'Garden1':2849, 'Garden2': 6000, 'Parkinglot': 6475

FE: # Feature Extractor CHOICE: 'CNN' INPUT_SIZE: (256, 128)

SOLVER: TYPE: 'TG' # {'SG': Spatial Graph, 'TG': Temporal Graph} EPOCHS: 100 EVAL_EPOCH: 2 BATCH_SIZE: 100 LR: 0.01 # Learning Rate MAX_PASSING_STEPS: 4 W: 3 # Temporal Window Size W_TEST: 5 FOCAL_ALPHA: 0.98 FOCAL_GAMMA: 5

OUTPUT: VISUALIZE: False LOG: True CKPT_DIR: './logs/ckpts' INFERENCE_DIR: './logs/inference'

TEST: CKPT_FILE_SG: "/mnt/data/wzc/ReST-main/logs/ckpts/CAMPUS_Garden2_SG_epoch12_train80.pth" CKPT_FILE_TG: "/mnt/data/wzc/ReST-main/logs/ckpts/CAMPUS_Garden2_TG_epoch6_train80.pth" FRAME_START: 4800 # 'Garden1': 2280; 'Garden2': 4800; 'Parkinglot': 5828 EDGE_THRESH: 0.9

chengche6230 / ReST

The Evaluation Results in CAMPUS GARDEN2 Dataset #17