search

CSIRT-MU / Stream4Flow

A framework for the real-time network traffic analysis based on world-leading technologies for distributed stream processing, network traffic monitoring, and visualization.
https://csirt.muni.cz/?lang=en
MIT License
99 stars 36 forks source link

Setting up a cluster deployment. #85

Closed zaxeqaz closed 5 years ago

zaxeqaz commented 5 years ago

Hello,

I'm trying to set up a cluster deployment. it seems the url for the FastBit library is broken.

http://liberouter.org/~thorgrin//libfastbit-Ubuntu-18.tar.gz

TASK [ipfixcol : Download FastBit compiled library (2/2)] *** fatal: [producer]: FAILED! => {"changed": false, "dest": "/tmp/libfastbit.tar.gz", "msg": "Request failed", "response": "HTTP Error 404: Not Found", "state": "absent", "status_code": 404, "url": "http://liberouter.org/~thorgrin//libfastbit-Ubuntu-18.tar.gz"}

Thank you

tomjirsa commented 5 years ago

Hi, thank you for your report. We look into it, contact the link owner, and update the link as soon as possible.

tomjirsa commented 5 years ago

Hi, we received a reply from the author of IPFIXcol who owns the link. Unfortunately, the IPFIXcol installation using ansible is available only for Ubuntu 16.04. (link). Ubuntu 18 you are using is not supported now.

Author of the IPFIXcol responded, that he could prepare the version for Ubuntu 18 during the next week.

Once the build for Ubuntu 18 is ready, I let you know.

In general, we tested the deployment for Ubuntu 16. We plan to add support and test the Stream4Flow for the newest Ubuntu server 18 in early 2019.

thorgrin commented 5 years ago

I've just built the libfastbit library on Ubuntu 18.04 and uploaded the binaries. I did not test it yet. Could you give it a try?

zaxeqaz commented 5 years ago

Hello,

Thank you for the quick response. I gave it a try on some fresh machines this morning and it looks like the libfastbit is working, unfortunately, it is now failing to build the IPFIXcol base claiming there is no makefile. I've attached the error output.

Thank you. [error.txt](https://github.com/CSIRT-MU/Stream4Flow/files/2535373/error.txt)

thorgrin commented 5 years ago

Yeah, well. As I said, it is not tested on this version of Ubuntu. However, since it is the current version now, I might as well do it.

There was a problem with libssl package. On newer Ubuntu the libssl1.0-dev is needed instead of libssl-dev. I've updated the ansible in the ipfixcol repository (devel branch), stream4flow guys need to reflect the change here.

tomjirsa commented 5 years ago

We reflected the changes as suggested by @thorgrin. The changes are in branch 85-cluster-deployment.

We do not have Ubuntu 18, so we have not tested it yet. Could you please give it a try? (if it works, we merge it to master)

tomjirsa commented 5 years ago

Tested with vagrantbox ubuntu/bionic64 (ubuntu 18.04) and I am getting errors. I'll recheck and get to you back when it works.

zaxeqaz commented 5 years ago

Thanks for the updates! :)

tomjirsa commented 5 years ago

Hi, I have tested the ipfixcol install on the vagrantbox and it is working now. The working version is in master now.

zaxeqaz commented 5 years ago

Hello,

I just ran ansible again and it seems like everything is working. Thank you so much for the help and updates, I look forward to trying out S4F.

Thank you.

tomjirsa commented 5 years ago

Great, we hope S4F meets your expectation:) Closing ticket.