About training

[HaoranLv]

Dear author：

When I use the configuration file you provided and the training command "python tools/train.py /home/ec2-user/SageMaker/table2/mmdetection/CascadeTabNet/Config/cascade_mask_rcnn_hrnetv2p_w32_20e.py", the training process stops when it runs to the position shown in the below (It is not training but the process is terminated. Use "nvidia-smi" below to check that there is no process). Do you know the reason? thank you very much! ：）

best wishes！ lv

(pytorch_latest_p36) sh-4.2$ /home/ec2-user/anaconda3/envs/pytorch_latest_p36/bin/python tools/train.py /home/ec2-user/SageMaker/table2/mmdetection/CascadeTabNet/Config/cascade_mask_rcnn_hrnetv2p_w32_20e.py . . . 2020-08-14 03:33:44,281 - mmdet - INFO - load checkpoint from /home/ec2-user/SageMaker/table2/mmdetection/CascadeTabNet/checkpoints/epoch_36.pth 2020-08-14 03:33:45,596 - mmdet - INFO - resumed epoch 36, iter 6156 2020-08-14 03:33:45,600 - mmdet - INFO - Start running, host: ec2-user@ip-172-16-32-137, work_dir: /home/ec2-user/SageMaker/table2/mmdetection/CascadeTabNet/workplace 2020-08-14 03:33:45,600 - mmdet - INFO - workflow: [('train', 1)], max: 36 epochs**

(pytorch_latest_p36) sh-4.2$ nvidia-smi Fri Aug 14 03:34:04 2020
+-----------------------------------------------------------------------------+ | NVIDIA-SMI 440.33.01 Driver Version: 440.33.01 CUDA Version: 10.2 | |-------------------------------+----------------------+----------------------+ | GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. | |===============================+======================+======================| | 0 Tesla V100-SXM2... On | 00000000:00:1E.0 Off | 0 | | N/A 37C P0 25W / 300W | 0MiB / 16160MiB | 0% Default | +-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+ | Processes: GPU Memory | | GPU PID Type Process name Usage | |=============================================================================| | No running processes found | +-----------------------------------------------------------------------------+

[DevashishPrasad]

The model training is resuming from epoch 36 and in config file it is total_epochs = 36.

So training has already reached its limit.

You need to change the total_epochs=50 (for eg. if you want to continue till 50 epochs)

[Letdreamfly]

@HaoranLv @DevashishPrasad i even did not find the tools/train.py, could you please tell me where is the train scripts?

[myh12138]

@HaoranLv @DevashishPrasad i even did not find the tools/train.py, could you please tell me where is the train scripts?

I met the same question

[Letdreamfly]

@HaoranLv @DevashishPrasad i even did not find the tools/train.py, could you please tell me where is the train scripts?

I met the same question

maybe the tools/train.py means tools/train.py from https://github.com/open-mmlab/mmdetection/tree/master/tools/train.py.

[myh12138]

@HaoranLv @DevashishPrasad i even did not find the tools/train.py, could you please tell me where is the train scripts?

I met the same question

maybe the tools/train.py means tools/train.py from https://github.com/open-mmlab/mmdetection/tree/master/tools/train.py.

thank you, now i can train it

[ramprakashyadav]

Hello friends,

can anyone tell me how i will train this model from the beggining and what process i have to follow it please help me

[RAravindDS]

Hello friends,

can anyone tell me how i will train this model from the beggining and what process i have to follow it please help me

did you find it, If yes please provide the code. I am looking for it.