Some questions about FLOPs calculation

[Ananwang2002]

Hi, I am very interested in your outstanding work, thank you for sharing! I have some questions I'd like to consult with you while I was reproducing this work. In the process of calculating FLOPs, I noticed that you seemed to have used the 'calculate_flops' method from the 'calflops' library, but then you commented it out. The FLOPs calculated using this part of the code are abnormally large (which seems to be incorrect, and this error is likely due to my oversight). I would like to ask how you calculate FLOPs, especially the FLOPs for the segmentation part.

[LMD0311]

Thank you for noticing our work!

In practice, we found that the FLOPs results calculated by various third-party libraries can vary, possibly due to confusion between MACs and FLOPs definitions. To align with existing work, such as the results of Table 1 in ReCon (ICML'23), we found that the fwd MACs values from the calflops library and the FLOPs output from profile match those reported in other publications.

To calculate the FLOPs for the segmentation part, you can use profile from thop library to simplify the process for multi-input.

Specifically, you can find the code around here: https://github.com/LMD0311/PointMamba/blob/6a3eec629b33c476367e7eb684c3eea2a26b3125/part_segmentation/main.py#L238 and insert (before the mentioned code) this:

from thop import profile
flops, params = profile(classifier, inputs=(points, to_categorical(label, num_classes)))
print('FLOPs = ' + str(flops / 1000 ** 3) + 'G')

Also, when calculating FLOPs, make sure the batch size of the input point is set to 1 by running: CUDA_VISIBLE_DEVICES=<GPU> python main.py --config cfgs/config.yaml --log_dir flops_calculate --batch_size 1.

We sincerely appreciate your interest in our work! Please feel free to ask any questions!

[Ananwang2002]

Thank you very much for your prompt reply, which helps me a lot ! I have another thing I would like to consult about. I used the two third-party libraries you mentioned for calculating FLOPs ( calflops and thop ) but I found a significant discrepancy in the computed parameters (I noticed that your paper recorded the parameter amount provided by _calculateflops). Do you know why this might be?

[LMD0311]

I am not sure what is wrong with thop🤔. The code for parameter amount calculation is a for loop that adds all tunable parameters (requires_grad) together, and is available here 👇: https://github.com/LMD0311/PointMamba/blob/6a3eec629b33c476367e7eb684c3eea2a26b3125/utils/misc.py#L257-L280 and is used in 👇: https://github.com/LMD0311/PointMamba/blob/6a3eec629b33c476367e7eb684c3eea2a26b3125/tools/runner_finetune.py#L109

[Harr7y]

When I used the profile from thop to compute the FLOPs of Point-MAE, an error occurred. RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu I am sure that the 'model' and the 'input points' are both at cuda:0.

Can you provide some advice? Thanks a lot~

[LMD0311]

When I used the profile from thop to compute the FLOPs of Point-MAE, an error occurred. RuntimeError: module must have its parameters and buffers on device cuda:0 (device_ids[0]) but found one of them on device: cpu I am sure that the 'model' and the 'input points' are both at cuda:0.

Can you provide some advice? Thanks a lot~

try flops, params = profile(model.module, inputs = (input, ))

[Harr7y]

Thanks a lot!🫡