treveradams / C-ICAP-Classify

This is a content classification module for C-ICAP
GNU Lesser General Public License v3.0
17 stars 1 forks source link

Can`t load module #2

Closed gdalbergs closed 6 years ago

gdalbergs commented 6 years ago

Ive built C-ICAP-classify from source (as well as c-icap itself) and had no``` errors there. Now when I include the module in my c-icap configuration like this: Module common srv_classify.so I get an error message from the c-icap server (when starting it) like this: Loading service :common path srv_classify.so Error loading module srv_classify.so:/usr/local/c-icap/lib/c_icap/srv_classify.so: undefined symbol IMAGE_SCALE_DIMENSION Error while loading module srv_classify.so Error loading module srv_classify.so, module path common Fatal error while parsing config file: "/usr/local/c-icap/etc/c-icap.conf"`

Also, will the image get rescaled even if I don`t have any data sets to base the classification upon? I want to deny the possibility to send malicious content within image.

treveradams commented 6 years ago

Hmmm. I will have to dig into this. If you are not using the image classification code (since it is impossible to train the classifiers with newer OpenCV, this shouldn't be used most likely), you should compile without support.

Are you sure you are using the latest code and not the last release? (I forgot to do the release.)

treveradams commented 6 years ago

As for bad code in images, my code does NOT do virus/malware scanning. You will want the virus scanning module from the main c-icap tree for that.

gdalbergs commented 6 years ago

I was using the code downloadable as the zip file from the main repo page. (master.zip) I have the latest built from source OpenCV. So the image classification wouldnt work even if I provided my own data sets? About the resizing: I would just want the image resizing to wipe out any potential malicious contetnt in the meta information and alike. I wouldnt require any scanning.

treveradams commented 6 years ago

OpenCV 3.0 and newer are NOT supported at all. If you have an older version, if you have your own data sets, it will work. Those data sets are haar feature trees as recognized by OpenCV. They changed to a C++ interface only for various necessary functions. As I don't have a data set I am happy with I didn't update the code. My module has nothing to do with meta information in images.

The zip file you downloaded is probably broken in many ways with newer opencv and newer c-icap. Give me a few minutes to tag a new release and push the release. Let me know if it fixes those things.

treveradams commented 6 years ago

Okay, please download release 20180405. That should fix your problem. If it does, please comment and close. If it doesn't, please let me know. If you wish to provide a patch to get the image code working with OpenCV >= 3.0, I will test and accept.

treveradams commented 6 years ago

Also, will the image get rescaled even if I don`t have any data sets to base the classification upon?

Image classification only takes place if you have the appropriate haar cascade files for use with OpenCV and have configured C-ICAP Classify properly to use them. Otherwise, no operations are done on images. This is also true for text. Without data configured, nothing takes places (although I believe this nothing is a little more expensive as it is assumed that text will be processed always).

gdalbergs commented 6 years ago

Thank You for Your comments and rapid response! The problem persists with the new release though. Looks like I will be disregarding my efforts towards this product. Reason for that being lack of time. The problem was produced on a VM running ubuntu 16.04 server if that helps. Should I close the issue as it seems no one else had this problem?

treveradams commented 6 years ago

Would it be possible for you to try to compile and tarball that up and send it to me at the email address in the AUTHORS file? The symbol you are mentioning should not exist nor be asked for if you don't have OpenCV or have the wrong version. Did you try doing this in a clean directory, or did you just do make again?

I am sorry that this would take more time than you are interested in. I am hoping some recent legal cases here in the USA will allow me to release the text (FHS) classification data in the near future. (It is not perfect, but is at least passingly usable for English.)

gdalbergs commented 6 years ago

The new release was indeed in a clean directory. I removed all the previous files and did it fresh. I did still have the openCV set up (looks like I misunderstood You regarding openCV). Would it be enough to give it a try by just not passing the openCV directory to the configure script? Also, maybe You could add the info about image part and openCV not working to the README or point me to where that was said since I for one spent unneeded time on this and maybe could prevent others from doing that. Also, You could maybe do a some sort of data flow explanation of the product, so people get a better idea of what they are dealing with earlier on. Not trying to be negative in any aspect, just giving my feedback. The text classification data would be a great thing and would be something worth a tutorial post. I really feel how people are going away from the open source software (regarding MITM SSL inspection with squid at least) and choosing commercial products instead and I think having a tutorial on such content filtering would be something to keep the hope alive. At least until some IDS/IPS will finally understand ICAP protocol or someone writes a c-icap plugin/module for and IDS/IPS.

treveradams commented 6 years ago

I have updated the README to mention the problem with newer OpenCV. The contrib directory contains some documentation. This is no mentioned in the README. I also have figured out why my autoconf was not working properly. I have made a new tagged release. I do not feel I should document squid/sslBump, although it is mentioned in various places so people can look at it. There are problems with it. It is easy to get wrong as well. I am open to help with documentation. This should compile for you, including a notice about it disabling OpenCV support.

gdalbergs commented 6 years ago

Thank you! I will close this issue. Hopefully this bump attracted some potential contributors.