Pinpoint Dedicated Installation Pre-Requisites

[dinesh4747]

Hi Pinpoint Team -

we had an initial demo (with Quickstart plugins) of pinpoint which looks very potential and quite interesting one and it almost suits our stack topology and now we have decided to implement in real tests

I have some questions regarding to that

1) Does the Quickdemo setup/package itself is sufficient enough to testing out with multiple agents across heterogeneous systems

2) Does the Hbase cluster setup comes as part of Quickdemo setup would scale automatically when we load so much of agent data during load testing?

3) If suppose QuickDemo setup doesn't hold good and sufficient enough then could you please help us in giving dedicated Collector/Hbase/Web-UI inside a same Virtual Machine

Please do also list down the requirements as follows:

a) Capacity required for Hbase (Memory Heap GB/Disk setup)

b) Is there any specific procedure/document available to create Hbase Cluster setup for pinpoint

c) Do we need any additional parameters for installing/configuring Collector/Web-UI setup

Kindly let me know If I am missing something here

Regards, Dinesh N

[dinesh4747]

In terms of huge data volume - how do we manage the Hbase data management like interms of archival/house keeping

[Xylus]

Hi @dinesh4747 thanks for a detailed question, Unfortunately, most of your questions depend entirely on your specific use case so it'll be hard to give a generalized answer that'll suit your every needs. That said,

1. Does the Quickdemo setup/package itself is sufficient enough to testing out with multiple agents across heterogeneous systems
   Quickstart setup does support multiple agents (as long as they're on java applications), but it won't give the kind of scalability you're looking for in a production environment. The main issue is due to how hbase is set up which I'll go into in the following question.
2. Does the Hbase cluster setup comes as part of Quickdemo setup would scale automatically when we load so much of agent data during load testing?
   Hbase launched via the quickstart script is run as a standalone instance. This runs everything in a single JVM, and doesn't use hdfs, and most importantly, won't scale out. If you're trying to set up Pinpoint for your environment I would suggest taking a look at pseudo-distributed local install and move on to a fully distributed hbase setup. Using Cloudera manager, or similar hbase management system is also a good choice that'll simplify a lot of the management stuff down the road.
3. If suppose QuickDemo setup doesn't hold good and sufficient enough then could you please help us in giving dedicated Collector/Hbase/Web-UI inside a same Virtual Machine
   Again, it all depends on the size of your system and how much you trace. If your system is small then using quickstart would be enough. If not, you should run collector and web in their own instances on top of running hbase in distributed mode forming it's own cluster as well.

As for the requirements: a) Capacity required for Hbase (Memory Heap GB/Disk setup)
Again, this depends on your environment - I'd suggest consulting hbase manual for this (plus googling) b) Is there any specific procedure/document available to create Hbase Cluster setup for pinpoint
We don't have one specifically for Pinpoint as there isn't anything different from any other hbase cluster setup. This is the official hbase documentation on setting up a hbase cluster. Again, check out Cloudera manager or other hbase management system that could make your cluster installation/management a lot easier. c) Do we need any additional parameters for installing/configuring Collector/Web-UI setup
You can edit the collector related configuration options in pinpoint-collector.properties, and web in pinpoint-web.properties. Comments should give you enough idea about what each of the options do, but feel free to ask away if you're unclear of anything.

In terms of huge data volume - how do we manage the Hbase data management like interms of archival/house keeping Again, I would recommend checking out Cloudera manager or other hbase management systems (Hortonworks is another one).