Only 8 FPS on CPU

[ghimireadarsh]

Tested the code with provided model checkpoint on 11th Gen Intel(R) Core (TM) i7-11700 @ 2.50GHz, but I am only getting maximum of 8 FPS.

[HDshuo]

Hello, can you tell me how to run through the code? I don't quite understand

[HDshuo]

使用提供的模型检查点测试了代码，但我最多只能获得 8 FPS。11th Gen Intel(R) Core (TM) i7-11700 @ 2.50GHz Hello, can you tell me how to run through the code? I don't quite understand

[ghimireadarsh]

Hi, do you want to run for inference or training?

使用提供的模型检查点测试了代码，但我最多只能获得 8 FPS。11th Gen Intel(R) Core (TM) i7-11700 @ 2.50GHz Hello, can you tell me how to run through the code? I don't quite understand

[HDshuo]

inference or training?

I want to try it first inference，If the algorithm is real-time, I want to try training

[ghimireadarsh]

inference or training?

I want to try it first inference，If the algorithm is real-time, I want to try training

I ran it from pytracking, and it is not giving me real-time performance at all.

[HDshuo]

After I run pytracking, there is a problem with this code. Do you have any solutions？

His frame rate is very low, right? I think other comments say it has something to do with the time function

def track(self, im, state_input, writer=None): self.frame_num += 1

    state = state_input['previous_output']
    p = state['p']
    avg_chans = state['avg_chans']
    window = state['window']
    target_pos = state['target_pos']
    target_sz = state['target_sz']

    hc_z = target_sz[1] + p.context_amount * sum(target_sz)
    wc_z = target_sz[0] + p.context_amount * sum(target_sz)
    s_z = np.sqrt(wc_z * hc_z)

    scale_z = p.exemplar_size / s_z
    d_search = (p.instance_size - p.exemplar_size) / 2  # slightly different from rpn++
    pad = d_search / scale_z
    s_x = s_z + 2 * pad

    x_crop, _ = get_subwindow_tracking(im, target_pos, p.instance_size, python2round(s_x.item()), avg_chans)
    state['x_crop'] = x_crop.clone()  # torch float tensor, (3,H,W)
    x_crop = self.normalize(x_crop)
    x_crop = x_crop.unsqueeze(0)
    debug = True

    if debug:
        target_pos, target_sz, _, cls_score = self.update(x_crop, target_pos, target_sz * scale_z,
                                                          window, scale_z, p, debug=debug, writer=writer)
        state['cls_score'] = cls_score
    else:
        target_pos, target_sz, _ = self.update(x_crop, target_pos, target_sz * scale_z,
                                               window, scale_z, p, debug=debug, writer=writer)

    target_pos[0] = max(0, min(state['im_w'], target_pos[0]))
    target_pos[1] = max(0, min(state['im_h'], target_pos[1]))
    target_sz[0] = max(10, min(state['im_w'], target_sz[0]))
    target_sz[1] = max(10, min(state['im_h'], target_sz[1]))

    #print("cropped x shape: ", x_crop.shape)
    #print("target pos shape: ", target_pos.shape)
    #print("target size shape: ", target_sz.shape)
    #print("target size: ", target_sz)

    # TODO: compute appropriate bounding box in x,y,w,h format (?) and return it
    location = cxy_wh_2_rect(target_pos, target_sz)

    # set lighttrack params
    state['target_pos'] = target_pos
    state['target_sz'] = target_sz
    state['p'] = p

    # set pytracking params
    state['target_bbox'] = location        
    #print("location: ", location)

[ghimireadarsh]

Yes, the frame rate is very low. Well, I remember debugging the code and fixing the issues. And the first one which I remember is commenting out the state = state_input['previous_output'].

[HDshuo]

state = state_input['previous_output'] The first is to comment out that I have finished but there are still errors. Do you remember.？

[ghimireadarsh]

Can you put the error you are getting so that I can directly look into those codes and will put the resolution here?

[HDshuo]

Traceback (most recent call last): File "E:/Track/E.T.Track/ettrack-main/pytracking/run_webcam.py", line 41, in main() File "E:/Track/E.T.Track/ettrack-main/pytracking/run_webcam.py", line 37, in main run_webcam(args.tracker_name, args.tracker_param, args.debug, visdom_info) File "E:/Track/E.T.Track/ettrack-main/pytracking/run_webcam.py", line 22, in run_webcam tracker.run_webcam(debug, visdom_info) File "E:\Track\E.T.Track\ettrack-main\pytracking\evaluation\tracker.py", line 451, in run_webcam out = tracker.track(frame, info) File "E:\Track\E.T.Track\ettrack-main\pytracking\tracker\et_tracker\et_tracker.py", line 181, in track p = state['p'] KeyError: 'p'

[ghimireadarsh]

Make the following edits.

diff --git a/pytracking/evaluation/tracker.py b/pytracking/evaluation/tracker.pyindex 634173f..e7ba305 100644
--- a/pytracking/evaluation/tracker.py
+++ b/pytracking/evaluation/tracker.py
@@ -295,7 +295,7 @@ class Tracker:

                 x, y, w, h = cv.selectROI(display_name, frame_disp, fromCenter=False)
                 init_state = [x, y, w, h]
-                tracker.initialize(frame, _build_init_info(init_state))
+                state_input = tracker.initialize(frame, _build_init_info(init_state))
                 output_boxes.append(init_state)
                 break

@@ -308,8 +308,14 @@ class Tracker:
             frame_disp = frame.copy()

             # Draw box
-            out = tracker.track(frame)
-            state = [int(s) for s in out['target_bbox'][1]]
+            timer = cv.getTickCount()
+            out = tracker.track(frame, state_input)
+            fps = cv.getTickFrequency() / (cv.getTickCount() - timer)
+            # state = [int(s) for s in out['target_bbox'][1]]
+            state = [int(s) for s in out['target_bbox']]
+
+
+

diff --git a/pytracking/tracker/et_tracker/et_tracker.py b/pytracking/tracker/et_tracker/et_tracker.py
index 02645a5..d70656e 100644
--- a/pytracking/tracker/et_tracker/et_tracker.py
+++ b/pytracking/tracker/et_tracker/et_tracker.py
@@ -48,7 +48,7 @@ class TransconverTracker(BaseTracker):
         tic = time.time()

         # Get target position and size
-        state = torch.tensor(info['init_bbox']) # x,y,w,h
+        state = torch.tensor(info['init_bbox'][1]) # x,y,w,h

[HDshuo]

I have modified all three of your settings, but there are still problems running on cmd.I put the checkpoint_ E35. pth is placed under the pytracking \networks file. Is there anything wrong with meHere is my error report.

(pptracking3.7) E:\Track\E.T.Track\ettrack-main\pytracking>python run_video.py et_tracker et_tracker 11.pm4 checkpoint epoch provided: 35 checkpoint path: ./checkpoints/et_tracker\checkpoint_e35.pth loading model from: ./checkpoints/et_tracker\checkpoint_e35.pth loading the checkpoint strict: True ERROR:root:No checkpoint found at './checkpoints/et_tracker\checkpoint_e35.pth' Traceback (most recent call last): File "run_video.py", line 38, in main() File "run_video.py", line 34, in main run_video(args.tracker_name, args.tracker_param,args.videofile, args.optional_box, args.debug, args.save_results) File "run_video.py", line 20, in run_video tracker.run_video(videofilepath=videofile, optional_box=optional_box, debug=debug, save_results=save_results) File "..\pytracking\evaluation\tracker.py", line 257, in run_video tracker.initialize_features() File "..\pytracking\tracker\et_tracker\et_tracker.py", line 17, in initialize_features self.params.net.initialize(self.params.model_name, checkpoint_epoch=checkpoint_epoch) File "..\tracking\basic_model\et_tracker.py", line 124, in initialize load_lighttrack_model(model=self, model_name=model_name, checkpoint_epoch=checkpoint_epoch) File "..\lib\utils\utils.py", line 179, in load_lighttrack_model load_checkpoint(model, checkpoint_path, strict=strict) File "..\supernet_backbone\lib_back\utils\helpers.py", line 56, in load_checkpoint state_dict = load_state_dict(checkpoint_path, use_ema) File "..\supernet_backbone\lib_back\utils\helpers.py", line 52, in load_state_dict raise FileNotFoundError() FileNotFoundError

[ghimireadarsh]

put the checkpoint file inside checkpoints/et_tracker/checkpoint_e35.pth

[HDshuo]

Sorry, I'm a novice, I don't understand very much, I didn't find it checkpoints/et_tracker/checkpoint_e35.pth ，where is 'checkpoints' file？Does this algorithm need the same environment as pytrackin? Sorry to bother you.

[ghimireadarsh]

In the root of this repository, not inside the pytracking.

[HDshuo]

In the root of this repository, not inside the pytracking.

Thank you very much. I'm already running. But also, my frame rate is very low

[ghimireadarsh]

In the root of this repository, not inside the pytracking.

Thank you very much. I'm already running. But also, my frame rate is very low

You are welcome. May I know how much frame rate are you getting? And your device information?

[cnchange]

Tested the code with provided model checkpoint on 11th Gen Intel(R) Core (TM) i7-11700 @ 2.50GHz, but I am only getting maximum of 8 FPS.

你是不是把测试数据存放在了外置机械硬盘里面？我用i5-10400也跑到了平均50FPS。

[cnchange]

In the root of this repository, not inside the pytracking.

Thank you very much. I'm already running. But also, my frame rate is very low

You are welcome. May I know how much frame rate are you getting? And your device information?

我明白了！作者使用的计算帧率的方法与我们的想法不同，你的8FPS应该是用序列的帧数除以处理该序列所需的时间得到（这样算的话我只有7.6FPS），但是，在代码（/pytracking/evaluation/tracker.py 211-218行）中，作者只计算了模型处理一帧图像所需的时间（约0.019s），而不是我们感性认识到的处理时间（其中还包含了从硬盘中加载帧图像的时间！），所以，按照作者的计算方法，确实可以达到50+FPS！

[ChuangGe66]

How to training?Thanks,please tell me

[HDshuo]

How to training?Thanks,please tell me It seems that there is no publicly available training code yet

[HDshuo]

In the root of this repository, not inside the pytracking.

Thank you very much. I'm already running. But also, my frame rate is very low

You are welcome. May I know how much frame rate are you getting? And your device information?

i running in gpu gtx1050，only around 10fps

[faicaiwawa]

In the root of this repository, not inside the pytracking.

Thank you very much. I'm already running. But also, my frame rate is very low

You are welcome. May I know how much frame rate are you getting? And your device information?

我明白了！作者使用的计算帧率的方法与我们的想法不同，你的8FPS应该是用序列的帧数除以处理该序列所需的时间得到（这样算的话我只有7.6FPS），但是，在代码（/pytracking/evaluation/tracker.py 211-218行）中，作者只计算了模型处理一帧图像所需的时间（约0.019s），而不是我们感性认识到的处理时间（其中还包含了从硬盘中加载帧图像的时间！），所以，按照作者的计算方法，确实可以达到50+FPS！

The time taken to read the image has been excluded, and the author covers all frames with the processing time of the first frame to achieve higher speeds, without explanation, which I think is a serious problem. I changed the fps calculation method and found that it barely satisfies real-time on my CPU(amd ryzen 6900hx), around 24fps on the GOT10K test set.