Ability to share (and specify) configuration of a group of toolkit hosts

[arlake228]

Original issue 578 created by arlake228 on 2012-01-13T19:36:46.000Z:

Provide a way to enable some amount of centralized configuration for a group of toolkit hosts. This grouping could be part of the 'community' or some other grouping concept.

Think of this from the perspective of a university (or VO) that wants all their hosts to do some number of active tests with all other hosts in a particular grouping.

[arlake228]

Comment #1 originally posted by arlake228 on 2012-01-13T22:46:20.000Z:

A vote/partial functionality definition:

-------- Original Message -------- Subject: Re: [perfsonar-user] Measurement Node install Date: Tue, 5 Jul 2011 15:09:17 -0400 (EDT) From: royboy@umich.edu Reply-To: royboy@umich.edu To: Jason Zurawski zurawski@internet2.edu

On 7/5/11 2:40 PM, thus spake royboy@umich.edu: Jason, Thanks for your reply. I would be interested in doing this, I have multiple toolkit nodes setup in our lab and am trying to setup what I was discribing. The toolkit was a big help in getting things rolling, but now that you made the first part easy, I am moving right along.

Another question I have is has any thoughts gone into leveraging the lookup service for multicast testing and monitoring?

Thanks, -Roy Hockett

Network Architect, ITS Networking & Telecommunications, University of Michigan Tel: (734) 763-7325 Fax: (734) 615-1727 email: royboy@umich.edu

On Tue, 5 Jul 2011, Jason Zurawski wrote:

Hi Roy;

I apologize, I misunderstood your question.

The use case you are describing is not one that the performance toolkit by itself was designed to solve. Each toolkit instance is meant to be 'standalone' for the most part - all of the testing and storage capabilities in one machine. They currently don't have a way to cooperate by default, and 'push' the data to a designated central location. The underlying perfSONAR-PS software makes ot possible to do this, but requires tweaking of some internal files and can't be done through the GUI.

If you would like to set up a complete institutional solution with multiple test points and a centralized archive there either needs to be modifications to each toolkit instance (if this is what you were starting with), or you may consider setting up individual services through RPMs. Several campus and regional networks have started doing this, but it is a bit more of a 'get your hands dirty' project.

I would be happy to talk about this with you offlist if you are interested.

Thanks;

-jason

On 7/5/11 2:26 PM, thus spake royboy@umich.edu:

Jason, Not really, as I thought a location (Institution) would want to have one Measurement Archive, but multiple test nodes. If I understand the diagnostic mode, it is a test point but no data is stored, however the node initiating the test against the diagnostic node would archive the test data. What if I want the test data archived, but not on this node, but on the institutional measurement archive. So I guess I am looking for the mode where the node is a able to test against, but archives the measurement results in a central measurement archive, not locally. Does this make sense? Is this possible?

Thanks, -Roy Hockett

Network Architect, ITS Networking & Telecommunications, University of Michigan Tel: (734) 763-7325 Fax: (734) 615-1727 email: royboy@umich.edu

On Tue, 5 Jul 2011, Jason Zurawski wrote:

Hi Roy;

I am going to CC the performance-node-users list on this response. On 7/5/11 2:06 PM, thus spake royboy@umich.edu:

The toolkit does a good job bringing up all services, but if you just want a measurement node to only test against and the the measurement data pushed to your measurement archive, is there good install script for this? The toolkit can be brought up in 'diagnostic mode', e.g. don't configure any regular testing options at all, and it will function simply as testing point/beacon on the network. It will not attempt to store measurement data on the host's harddrive when used in this manner. Other nodes will be able to then test against this node, provided they know it's hostname/address. Every measurement host that does set up a test with the beacon, will then be in possession of test results. Does this answer your question? Thanks; -jason

[arlake228]

Comment #2 originally posted by arlake228 on 2012-01-13T22:49:05.000Z:

And someone from XSEDE wants this too:

-------- Original Message -------- Subject: Re: perfSONAR Measurement Archive Date: Tue, 16 Aug 2011 16:57:04 -0400 From: Kathy Benninger benninger@psc.edu Organization: Pittsburgh Supercomputing Center To: zurawski@internet2.edu CC: Matthew J Zekauskas matt@internet2.edu, Stephen Wolff swolff@internet2.edu, Eric Boyd eboyd@internet2.edu, Kathy Benninger benninger@psc.edu

Hi Jason,

My answers inline, based on what I think I know at this point:

On 8/7/2011 7:50 PM, Jason Zurawski wrote: Hi Kathy;

I CCed a couple others from Internet2 with Interest in XSEDE. I put some other questions and comments inline:

On 8/4/11 10:51 AM, thus spake Kathy Benninger: Hi Jason,

I read your 5-July-2011 response to Roy Hockett @ UMich:

"If you would like to set up a complete institutional solution with multiple test points and a centralized archive... I would be happy to talk about this with you offlist if you are interested."

and thought, "That sounds like what I'm thinking about for XSEDE!"

The XSEDE deployment will include a perfSONAR MP at each of the eight sites. It would seem to me that a central MA (i.e. a central point to collect and provide access to all the perfSONAR information) for the XSEDE sites makes sense.

I have a couple of questions/comments about your proposed setup:

• Is the idea to collect a "full mesh" of measurements for all 8 sites?

I believe we will be interested in a "full mesh" of measurements between all XSEDE sites.

• What sorts of measurements are planned? Bandwidth? Latency? Others?

There will be some scheduled active measurements (pingER, BWCTL. OWAMP), though less emphasis on these than there was with TeraGrid where we used to run an hourly GbE iperf mesh. The XSEDE ops-network group particularly wants to have on-demand bandwidth test capability (I am not sure if that will need to encompass anything beyond manually running iperf). Latency and jitter measurement are lower priority than bulk data transfer throughput measurement.

• In a central MA setup, the MA machine has to have enough disk (and cpu/main mem) to absorb both the data, and the requests from GUIs/Clients for the data. The MPs could be a little less powerful because of this distribution of load.

We plan to use the same hardware for MPs and the MA, with additional disk on the MA.

We aren't ready to deploy yet (in the process of ordering hardware), but will be in the next few months.

Speaking of hardware, do you have a recommendation for the storage requirement of a MA that would be set up to service multiple MPs (assuming that is an appropriate config for us)? I have't found guidelines on storage requirements.

We have some information posted here:

http://psps.perfsonar.net/toolkit/hardware.html

For the Internet2 Network Observatory, we collect data from 27 MPs (3 each at 9 sites) and store things in a central machine with 500G of disk. The data is stored in MySQL on the backend, so it is possible to compress or backup data as time goes on.

I think we will have 8 or so MPs with mirrored 2TB disk (2 x 2TB disks) and mirrored 4TB on the MA. We would backup the MA rather than compressing data.

Hope this helps, as you get closer to your deployment feel free to reach out to us if you want to talk via mail or phone on specifics. We normally recommend interacting with the mailing lists for 'support' related questions, but we are always happy to work on a more 1:1 basis.

Thanks! I'm sure I'll have more questions.

Kathy

[arlake228]

Comment #3 originally posted by arlake228 on 2012-02-08T16:29:04.000Z:

Attaching comments that relate to this from ATLAS

[arlake228]

Comment #4 originally posted by arlake228 on 2012-03-13T17:40:56.000Z:

Issue 93 has been merged into this issue.

[arlake228]

Comment #5 originally posted by arlake228 on 2012-03-13T20:39:13.000Z:

<empty>

[arlake228]

Comment #6 originally posted by arlake228 on 2012-03-27T17:10:49.000Z:

May I suggest that a first step might involve the ability to "export" the toolkit configuration information as a file, maybe a simple (cough) XML file, that can be saved off to a thumb drive, etc. so that the non-host specific configuration can be duplicated quickly and deterministically?

I'd suggest that this export/import scheme be separate from the OS configuration such as users, network, etc.

[arlake228]

Comment #7 originally posted by arlake228 on 2012-09-20T18:04:36.000Z:

<empty>

[arlake228]

Comment #8 originally posted by arlake228 on 2013-03-18T14:28:17.000Z:

<empty>