search

doonny / PipeCNN

An OpenCL-based FPGA Accelerator for Convolutional Neural Networks
Apache License 2.0
1.22k stars 369 forks source link

Input format for resnet-50. #117

Open ignvinay opened 4 years ago

ignvinay commented 4 years ago

Hi Donny, I got resnet-50 working with your code. But i wanted to test it with different inputs (other than cat) . So i tried converting cat.jpg to 224,224,3 and tried feeding as input. But network badly fails. And then further investigation in to your image.data file and my converted 224,224,3, i found lot of differences in my data vs your image.data. such as : (first few values of image.data) ['195', '194', '195', '196', '196', '196', '195', '195', '198', '198', '197', '199', '200'] my data : array(50, 57,26,48,56,27,50,58,29,51, dtype=uint8)

Why would this difference would be ?

I am also attaching my python code. Please let me know in what format your data is ?

Is there any RGB mean value differences you are doing at input level and then feeding to network ? rgb_means = (123.68, 116.78, 103.94)

Also i wanted to understand how you have converted weights of .caffemodel to .data file, as i see matlab code to convert, but i am unable to understand how is weight format for this opencl code.

jpeg_read.zip

r0drigor commented 4 years ago

I'm not completely sure, but I think this uses BGR instead of RGB. Also, I think it transposes the data (transformation from lines to columns).

ignvinay commented 4 years ago

Thanks r0drigor. I am finding hard to convert weights of resnet-50. Any ideas how we can get this done ?

r0drigor commented 4 years ago

They're taken from the caffemodel. There's info about it in this url: https://github.com/doonny/PipeCNN/tree/master/data#how-to-prepare-the-cnn-models-for-pipecnn. I don't know too much of how it's done or what's the format, but I'm guessing they just go to each layer and just take the data.

doonny commented 4 years ago

Please refer to the function "loadImageToBuffer" in the host program, which loads the JPEGs into the FPGA.

ignvinay commented 4 years ago

I meant format of fixed point representation. As i see all weights are quantized by 8 bits and there are no bias values in weights_qt.data , may be because the bias values of convolution are so low , that you left them , is that true ? Also i see fc_1024.data seperately maintained with 8 bit quantization , correct ? So what are the represetation of these weights/scale/alpha/beta etc. Are they all following Q1.7 format ? I am asking these questions as i did not find script to get these for resnet-50. Please help, doony.

Regards Vinay

ignvinay commented 4 years ago

It would be great if you can share your data format represetations for weights_qt.data fc_1024.data mean.data alpha.data beta.data var.data

I tried quantizing values with 8 bits, i can see conv1 (layer1) data matching but other data i am not able to. Can you please share how you converted and what Qx.y format you used, please !

ignvinay commented 4 years ago

Hi Doony, Looks like you are trying to implement the fused batch normalization. But i am not able to get your format of representation for mean,var and alpha and beta. Can you please share your script for resnet-50 also(similar to the one of alexnet )?