raise Exception when export params

[r1marcus]

Hello, i trained the model with: python main.py -p quantized_params/QLSTM128_W2B8A4I32_FC_W32B32.json after that i want to export the params with python main.py -i /home/marcus/pytorch-ocr/experiments/20190806_105154_QLSTM128_W2B8A4IA32_FC_W32B32/checkpoints/epoch_best_wer.tar -p quantized_params/QLSTM128_W2B8A4I32_FC_W32B32.json --export

but then always come the Exception:

2019-08-08 09:12:16,382 Resuming training from /home/marcus/pytorch-ocr/experiments/20190806_105154_QLSTM128_W2B8A4IA32_FC_W32B32/checkpoints/epoch_best_wer.tar 2019-08-08 09:12:16,382 {u'random_seed': 123456, u'target_height': 32, 'recurrent_weight_quantization': 'SIGNED_FIXED_UNIT', u'num_workers': 0, 'neuron_type': 'QLSTM', u'lr_gamma': 0.5, 'fc_bias_quantization': 'FP', u'batch_size': 32, u'epochs': 4000, 'recurrent_bias_quantization': 'SIGNED_FIXED_UNIT', u'quantize_input': True, u'internal_activation_bit_width': 32, u'bidirectional': True, 'lr_schedule': 'FIXED', u'recurrent_activation_bit_width': 4, u'mask_padded': True, u'lr_step': 40, u'recurrent_bias_enabled': True, 'fc_weight_quantization': 'FP', u'prefused_bn_fc': False, u'recurrent_weight_bit_width': 2, u'recurrent_bias_bit_width': 8, u'seq_to_random_threshold': 20, u'fc_weight_bit_width': 32, u'lr': 0.0001, 'recurrent_activation_quantization': 'SIGNED_FIXED_UNIT', u'fc_bias_bit_width': 32, u'max_norm': 400, u'checkpoint_interval': 10, 'reduce_bidirectional': 'CONCAT', u'layer_size': 128} 2019-08-08 09:12:16,382 {'pretrained_policy': 'RESUME', 'eval': False, 'recurrent_weight_quantization': None, 'lr_gamma': None, 'fc_bias_quantization': None, 'fc_bias_bit_width': None, 'epochs': None, 'bidirectional': None, 'experiments': '/home/marcus/pytorch-ocr/experiments', 'export': True, 'internal_activation_bit_width': None, 'lr_schedule': None, 'recurrent_activation_bit_width': None, 'dry_run': True, 'recurrent_weight_bit_width': None, 'recurrent_bias_bit_width': None, 'sortedtrain': '/home/marcus/pytorch-ocr/db_files_uw3-500/sortedTrain.txt', 'fc_weight_bit_width': None, 'valid': '/home/marcus/pytorch-ocr/db_files_uw3-500/valid.txt', 'params': '/home/marcus/pytorch-ocr/quantized_params/QLSTM128_W2B8A4I32_FC_W32B32.json', 'init_bn_fc_fusion': False, 'pe': 1, 'input': '/home/marcus/pytorch-ocr/experiments/20190806_105154_QLSTM128_W2B8A4IA32_FC_W32B32/checkpoints/epoch_best_wer.tar', 'simd_factor': 1, 'random_seed': None, 'no_cuda': True, 'num_workers': None, 'neuron_type': None, 'mask_padded': None, 'batch_size': None, 'lr_step': None, 'recurrent_bias_quantization': None, 'quantize_input': None, 'target_height': None, 'imgs': '/home/marcus/pytorch-ocr/db_files_uw3-500/imgs', 'export_test_image': False, 'recurrent_bias_enabled': None, 'fc_weight_quantization': None, 'seq_to_random_threshold': None, 'lr': None, 'recurrent_activation_quantization': None, 'max_norm': None, 'checkpoint_interval': None, 'reduce_bidirectional': None, 'layer_size': None} 2019-08-08 09:12:16,382 Architecture: OCRModule( (recurrent_layer): QuantizedLSTM(32, 128, bidirectional=True) (batch_norm_fc): FusedBatchNorm1dLinear( (batch_norm): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True) (linear): QuantizedLinear(in_features=256, out_features=82, bias=True) ) (output_layer): Sequential( (0): SequenceWise ( FusedBatchNorm1dLinear( (batch_norm): BatchNorm1d(256, eps=1e-05, momentum=0.1, affine=True) (linear): QuantizedLinear(in_features=256, out_features=82, bias=True) )) ) ) 2019-08-08 09:12:16,382 Number of parameters: 187474 Traceback (most recent call last): File "main.py", line 151, in trainer.export_model(args.simd_factor, args.pe) File "/home/marcus/pytorch-ocr/ocr/trainer.py", line 414, in export_model self.model.export(os.path.join(self.output_dir_path, 'r_model_fw_bw.h'), simd_factor, pe) File "/home/marcus/pytorch-ocr/ocr/model.py", line 191, in export fc_weight_decl, fc_weight_string = self.batch_norm_fc.linear.hls_weight_string(self.reduce_factor) File "/home/marcus/pytorch-quantization/quantization/modules/quantized_linear.py", line 77, in hls_weight_string return quantized_linear_hls_weight_string(weight, self.weight_quantization_scheme, hls_var_name) File "/home/marcus/pytorch-quantization/quantization/function/quantized_linear.py", line 45, in quantized_linear_hls_weight_string int_weight = quantization_scheme.to_int(weight).numpy().astype(int) File "/home/marcus/pytorch-quantization/quantization/function/quantization_impl.py", line 39, in to_int raise Exception Exception

do you have some tips?

[volcacius]

Hi,

The script you are using is not quantizing the FC layer, which is why an Exception is raised when you try to export the trained model. The script is supposed to be only a reference and doesn't guarantee that the trained model is exportable to HW. If you want to go to HW, edit the script to quantize all the parameters.

Alessandro

[r1marcus]

okey thank you very much for the information. i changed the code from: @staticmethod def to_int(q_params, x): raise Exception

to this one: ` @staticmethod
def to_int(q_params, x): int_weight2 = x << 2 int_weight2 = int_weight2.int() return int_weight2

raise Exception`

and the output seems to be right.

[volcacius]

The Exception is there for a reason, if you are not quantizing your parameters, there is no int version to generate. The code you posted doesn't mean anything. The correct way to proceed, as I said in my previous comment, is to enable quantization for the FC layer by modifying the json file.

Alessandro

[r1marcus]

Thank You!