MultiClass training

[abhigoku10]

@skyhehe123 hi can you share the code / config for multi class training or suggest the modifications to be made in multiclass training Thanks in adavance

[abhigoku10]

@stalkermustang have you tried or done multitask training modifications

[stalkermustang]

I train some separate models

[stalkermustang]

I have tried to make multiclass PointRCNN on Lyft challenge prev. year, but get only bad results (later i read in some papers that this don't work, lol. This make me angry and sad). Thus i didn't change Sa-SSD to multiclass.

[abhigoku10]

@stalkermustang oh man i am also keeping pointrcnn as the reference and making changes in sa-ssd but runnign into errors in focal loss calculation

[stalkermustang]

Think you made mistakes, because i train on Waymo for 4 epochs to get 0.46 AP on Vehicles, and now train 22 epochs without errors (14 epochs to go, see AP later, but currently it comparable).

After Waymo challenge ends i'm prepare codebase, refactor it and publish with docker env and some changes.

[abhigoku10]

@stalkermustang did u do this for multiclass ?? can you share your modifications so that i double check it

[stalkermustang]

No, i don't make changes for multiclass, as writed above (thanks for PointRCNN).

[abhigoku10]

@stalkermustang wokay i shall try to modify can you share mail id or other means where i can connect with you

[jiangzhuo1111]

Think you made mistakes, because i train on Waymo for 4 epochs to get 0.46 AP on Vehicles, and now train 22 epochs without errors (14 epochs to go, see AP later, but currently it comparable).

After Waymo challenge ends i'm prepare codebase, refactor it and publish with docker env and some changes.

May I know what did you modified for waymo dataset, I got bad result on waymo dataset using SA-SSD, still debugging.

[stalkermustang]

Think you made mistakes, because i train on Waymo for 4 epochs to get 0.46 AP on Vehicles, and now train 22 epochs without errors (14 epochs to go, see AP later, but currently it comparable). After Waymo challenge ends i'm prepare codebase, refactor it and publish with docker env and some changes.

May I know what did you modified for waymo dataset, I got bad result on waymo dataset using SA-SSD, still debugging.

i did a lot of changes, and not sure that i publish it earlier than 1-2 weeks. Also codebase contains non-reproduceble parts, because i.e. i'm storing data in SQL-like table, and use specified libaries to work around.

So my proof here: https://waymo.com/open/challenges/entry/?challenge=DETECTION_3D&email=stalkermustang@gmail.com&timestamp=1590963097332289 Keys that i found: build RIGHT environment (i post dockerfile later) and accurate preprocess data from waymo.

[jiangzhuo1111]

Thanks for reply. So if I did correct dataloader from waymo, and other configs and code stay the same,I will get an reasonable result,right? because u said the key part is the "accurate preprocess".

[stalkermustang]

Yeah, that's right. Generally you need to down center_z to lower plane of object, like in KITTI dataset (new_z=centrer_z - h/2) and change rotation: new_heading = -old_heading-np.pi/2 Waymo provide labels in lidar coord system, and there's no reason to use calibraions, rotations and other.

[stalkermustang]

ops, i don't read right your comment. No, you need change configs and code. There's some KITTI-specific parts, like occulusion back side of pointcloud, but in waymo we need to detect all objects around

[jiangzhuo1111]

OK, Thank you.I’ll check my data preprocess.

[jiangzhuo1111]

ops, i don't read right your comment. No, you need change configs and code. There's some KITTI-specific parts, like occulusion back side of pointcloud, but in waymo we need to detect all objects around

OK,these kind of parts I have already changed(points range, anchor size, anchor offsets, do I miss some thing?).May I ask three more questions: 1.My whole loss is about 0.9, is that near to yours? 2.When I do the test, after Pswarp operation and reorder, there are many false predictions's score is very high, have u met this situation?

[stalkermustang]

do I miss some thing?)

As i remember, there's part of definations in functions signatures. Search for "70.", "40" or smth in codebase and you get places, where you should replace them. Look here for example:

https://github.com/skyhehe123/SA-SSD/blob/e8d4db87fe7651dff3b3693c53b8fc2d4afb17cc/mmdet/models/necks/cmn.py#L208

and check default args here: https://github.com/skyhehe123/SA-SSD/blob/e8d4db87fe7651dff3b3693c53b8fc2d4afb17cc/mmdet/core/bbox/transforms.py#L218

[stalkermustang]

also @skyhehe123 Billy, just fyi check exmaple above pls.

[stalkermustang]

1.My whole loss is about 0.9, is that near to yours?

i got 0.6 in Vehicles, 0.4 on cyclists and 1.3 (lol?) for pedestrians.

[stalkermustang]

How many epochs/steps you do durning training ?

[jiangzhuo1111]

How many epochs/steps you do durning training ?

I trained on mini datasets first,20 epochs for 20000 frames,and loss don't decline after 12 epochs, maybe the datasets is also not big enough.

[jiangzhuo1111]

do I miss some thing?)

As i remember, there's part of definations in functions signatures. Search for "70.", "40" or smth in codebase and you get places, where you should replace them. Look here for example:

https://github.com/skyhehe123/SA-SSD/blob/e8d4db87fe7651dff3b3693c53b8fc2d4afb17cc/mmdet/models/necks/cmn.py#L208

and check default args here: https://github.com/skyhehe123/SA-SSD/blob/e8d4db87fe7651dff3b3693c53b8fc2d4afb17cc/mmdet/core/bbox/transforms.py#L218

oh, I think it's very important ,use the default args "offset" (0., -40., -3.) is for waymo not correct, it has to be consistent to the config file, right?

[stalkermustang]

Yeah, that's it. I don't count, how many times i'm change this in code, here's just example. You should look into, search for and review.

[jiangzhuo1111]

Yeah, that's it. I don't count, how many times i'm change this in code, here's just example. You should look into, search for and review. OK,its really kind of u and really helpful ,thank you very much!!!

[jiangzhuo1111]

Yeah, that's it. I don't count, how many times i'm change this in code, here's just example. You should look into, search for and review.

Sorry for disturbing you again.I have already fixed some bugs and got better results. But my results are not good at long distance（50~80m）, about 0.2 ap. May I know if you do some changes for long distance detection？

[stalkermustang]

Yeah, that's it. I don't count, how many times i'm change this in code, here's just example. You should look into, search for and review.

Sorry for disturbing you again.I have already fixed some bugs and got better results. But my results are not good at long distance（50~80m）, about 0.2 ap. May I know if you do some changes for long distance detection？

I don’t remember anything like that, that is, for me there was no difference, 10 or 70 meters

[Divadi]

Hi, for another dataset, for line https://github.com/skyhehe123/SA-SSD/blob/24c91497e981d0ea70e35a9efa4eb774beb543bf/configs/car_cfg.py#L28 should the values match the point cloud range in the following line?https://github.com/skyhehe123/SA-SSD/blob/24c91497e981d0ea70e35a9efa4eb774beb543bf/configs/car_cfg.py#L116

So the grid offset would be (beginning of x, end of y)? It felt a little odd, so I wasn't sure.

@stalkermustang I would much appreciate any advice

[stalkermustang]

In first line you should redefine offsets according to your setup in cfg, i.e. for waymo i set: extra_head=dict( type='PSWarpHead', grid_offsets = (70.4, 70.4), featmap_stride=.4, in_channels=512, num_class=1, num_parts=28, ), so minimal of my X's and y's is always zero (say there's a point in scene with coords XYZ [-70.389, -70.0, 3.0] after add ofsets point become [0.011, 0.4, 3.0])

So yes. My PC ranges are: point_cloud_range=[-70.4, -70.4, -1., 70.4, 70.4, 3.], @Divadi

[Divadi]

Ah, so it's the offset b/n the minimum of the pointcloud & (0,0), thank you!

[abhigoku10]

@Divadi @skyhehe123 did you guys perform the multiclass training for the current sassd pipeline ??

[skyhehe123]

@Divadi @abhigoku10 @stalkermustang The multiclass training is done, I will release it this week after cleaning. Sorry for the late release

[abhigoku10]

@skyhehe123 @stalkermustang i have modified the code for multi-class training, taking car and peds as labels, but my guided anchors give zero output after certain epochs due to which i get nan in my loss calculation, so when i debugged it my classification loss is not satisfying the condition of "selected = top_scores > thr" in ssd_rotatehead.py . Any pointers on this

[abhigoku10]

@stalkermustang in your multiclass training for PointRCNN did u use softmax or sigmoid for multiclass training ? i am getting no predictions in my code when i make the changes as classfication is no working properly @skyhehe123 any update on the multiclass training activity

[skyhehe123]

@abhigoku10 @stalkermustang @Divadi Hi, the multiclass training is added

[abhigoku10]

@skyhehe123 thanks for sharing multiclass code , has the inference timing changed with addition of multiple classess

[kaushik333]

In first line you should redefine offsets according to your setup in cfg, i.e. for waymo i set: extra_head=dict( type='PSWarpHead', grid_offsets = (70.4, 70.4), featmap_stride=.4, in_channels=512, num_class=1, num_parts=28, ), so minimal of my X's and y's is always zero (say there's a point in scene with coords XYZ [-70.389, -70.0, 3.0] after add ofsets point become [0.011, 0.4, 3.0])

So yes. My PC ranges are: point_cloud_range=[-70.4, -70.4, -1., 70.4, 70.4, 3.], @Divadi

Hi,

May I know how you came up with these values? I am trying to adapt SASSD to nuscenes dataset. however, the loss doesnt go down as well as Kitti. I dont know whether the problem is in the configuration settings or in the training hyperparameters.