Understand Vagrant Networking Configurations Better, Multiple VM Networking

[dougmcclure]

In some of my testing using a RedHat 6.4 base box I've built, I'm struggling with being able to eliminate a network problem or build problem. I can install nightly builds, ssh into the vagrant box from my host but can't reach the default ILA url/port.

I need to identify if I need to move to a bridged network setup. I need to know if my setup, boxes I'm using, etc. is correct or if I have a problem with installed build. I need to know best setup for multiple VM scenario.

I question myself b/c I have this working with milestone driver 1 on the CentOS 5.8 box with only the default vagrant network setup. Why wouldn't this work with newer builds and RedHat 6 base box? My gut tells me it's on my side.

TODO: Figure the best set up for vagrant and networking that will allow me to demonstrate multiple vagrant boxes spun up, communicating, shipping logs to ILA for more realistic blog articles on how to use ILA.

[c835722]

Yep, I think I understand and agree with that objective. So you seek a convenient setup that mirrors a standard enterprise operational topology. Lets says a Web Server, App Server and DB server all streaming to the Syslog server running ILA. Then you can show how a logging event can be correlated across those as a singular transaction such that it can be drilled into via a support type engineer. Further you need to reflect on how these can be aggregated by the tooling into a representations of meaning to showcase the tooling. Eg. Business, Application, System. Platform events. I need to do some more hunting and pecking around Vagrant.

[dougmcclure]

Scenario #1

CentOS 6.4 Box 1 - ILA Driver 1 CentOS 6.4 Box 2 - "An App" - Web server, app server, db server, ? maybe a more modern application like the "DayTrader" one we use a lot internal at IBM (we could just stage logs on this box from various things as well), ILA Log File Agent installed (customize silent installer to only install LFA here)

Get boxes networked properly for shipping logs each way, credentials in place Demonstrate remote log collection from box 1 to box 2 Demonstrate streaming logs from box 2 to box 1 Capture what is learned Blog & share on Git

[dougmcclure]

I'm pretty close to this working. I have added some comments in the provisioning script for the remote lfa box where I'm currently stuck trying to automate ssh/rsync file transfer between the two boxes and then install the LFA.

Need to firm up the scenario path to take - simulate by placing some WebSphere or DB2 logs on the box or actually install community version of WebSphere or DB2 and have live logs stream in.

[c835722]

Can you possibly commit your current change as a branch up to the public github repo? Does the log file adaptor support tailing a file from a remote host? The scenario interests me. As the WebSphere Liberty Profile is available to be network provisioned and mongo db is available form the repo you might be able to showcase such an implementation running a REST based interface. This would allow your demonstration scenario to also be scripted and "live" which is preferable from a demonstration perspective I think. That is "here come some events into the app eg. PUT customer-name" and "now we search for customer-name in ILA" and voila! Counter-point is starting with static logs and KISS. Need to consider the alternatives here. A puppet/chef/maven application provision of OOTB app may be preferable. Needs looking in to.

[dougmcclure]

Still a bit of a Git noob so if you can point me to where I'd do that I certainly can!

The LFA that came with driver 1 is version 6.2.3.x and supports the typical monitoring of individual logs or entire directories on the local system. In the next milestone driver, the LFA is upgraded to the new 6.3 agent which adds the ability to monitor remote logs via ssh. More details can be seen here: http://pic.dhe.ibm.com/infocenter/tivihelp/v61r1/index.jsp?topic=%2Fcom.ibm.itm.doc_6.3%2Flogfile%2Fklo_remote_logfile.htm . I have some scenarios in mind for that when we get our next driver.

I'd love to have this as "live" as possible and all automated. I'll try to look into the WebSphere Liberty stuff some after I get past the remote LFA box build challenges.

On Mon, Apr 8, 2013 at 11:42 PM, c835722 notifications@github.com wrote:

Can you possibly commit your current change as a branch up to the public github repo? Does the log file adaptor support tailing a file from a remote host? The scenario interests me. As the WebSphere Liberty Profile is available to be network provisioned and mongo db is available form the repo you might be able to showcase such an implementation running a REST based interface. This would allow your demonstration scenario to also be scripted and "live" which is preferable from a demonstration perspective I think. That is "here come some events into the app eg. PUT customer-name" and "now we search for customer-name in ILA" and voila! Counter-point is starting with static logs and KISS. Need to consider the alternatives here. A puppet/chef/maven application provision of OOTB app may be preferable. Needs looking in to.

— Reply to this email directly or view it on GitHubhttps://github.com/dougmcclure/ILA-Open-Beta/issues/5#issuecomment-16092561 .

[dougmcclure]

I see the create branch drop down - do I just create one with a new name (v1.1?) and go? Is that something I should do every time, every so often, at a significant milestone?

[c835722]

http://stackoverflow.com/questions/5423517/how-do-i-push-a-local-git-branch-to-master-branch-in-the-remote You can push a branch to origin often so it can be discussed/merged/diffed to master

[dougmcclure]

Got a lot done today - including a bit of reading on how to use Git better. Need to develop some habits there over these experiments and I'm sure I'll break something.

I created a branch Experiments Round 2 which I'm doing local development against and will synch up to that one on the main Git hub. I think this is the objective. I'm not sure how the git magic happens on my side (Windows) but it appears to keep those changes in that branch and not in the master (which is probably messed up with stuff I did the other day). Not sure of the clone/copy stuff yet - my guess is I clone the master or a branch and make changes to those files then push them up. Need some practice there.

I have manually gone through the steps to install the Websphere Liberty profile and a sample application that could be fun to play with. I'm going to finish the automation steps in my provisioning file tomorrow. It's all pretty simple!

It's becoming clear that I need to figure out how chef could help with some of these steps - e.g. install pre-reqs, install liberty, install derby, install app, etc.

Also need to figure out how one all inclusive vagrantfile could be used to build and provision both boxes rather than having two separate sets of files. I know this is possible, and could help now as I have a "box pre-req" that the master ILA box is up first so the LFA files become available for the second box.

Logs look good on the Liberty side and will give some easy demo scenario content. Need to look for how to set up logging levels and where we might insert failures to have things show up in logs like we do with DayTrader and it's JDBC connection pool depletion.

Enough for now -bedtime! (too much vagrant today!!)

[c835722]

Hi, The multiple directory usage of git is not a workflow I am used to. (versus one project one repo where user bootstraps the code from root directory via arbitrary executable) Traditionally the singular code base would develop over time such that a point in time it works with a particular version of software. eg. Centos58 progressing to Centos64. If a github user wanted to see the Centos58 that would check out the appropriately tagged code instance. So thats something to consider. I know we are just finding out feet. Have a look at this application to assist getting your head around git. There is a windows version. http://www.sourcetreeapp.com/ Chef solo is probably the way to go for this project. I think I appreciate why you wish to go in that direction. We will have to understand how a chef-solo recipe in the main repo can be kept up to date. Certainly moving as much as possible out of the custom script and into the chef section of the Vagrantfile is preferable as is singular Vagrantfilr for multi-box scenario. Please outline the difference between LFA and ILA components. Next chance I get I will check the main repo again and validate I can build the expected outcome. Will also consider how a logical operational topology model can be best represented here so a common runtime understanding can be shared. And yes, these items need to be split into distinct issues/requirements rather than grouped together in clumps in this thread. All the best.