damaex17 / whisper-to-influxDB

whisper-to-influxDB
6 stars 2 forks source link

top level directory being dropped when using multiple directory levels for organization #1

Open mshirley opened 10 years ago

mshirley commented 10 years ago

When whipser-to-influxdb imports it's looking recursively for directory/.wsp and then uses directory as the timeseries name. This is fine when you have something like hostname/.wsp but not when you have hostname/category/*.wsp.

The script will drop hostname and only import category/*.wsp.

I have solved one of my problems by replacing:

time_series = whisper_file.replace('//','/').split('/')[-2].split('/')[-1]

with:

time_series = whisper_file.replace('//','/').split('/')[-3] + "." + whisper_file.replace('//','/').split('/')[-2]

This allows for a timeseries name "hostname.category" to be created with the proper key and value.

Before:

/var/lib/carbon/whisper/servers/hostname/memory/Shmem.wsp memory Shmem 1398228900.0 1699840.000000

After:

/var/lib/carbon/whisper/servers/hostname/memory/Shmem.wsp hostname.memory Shmem 1398228900.0 1699840.000000

Unfortunatly this still only supports 2 levels of directories. I have some metrics that are hostname/category/subcategory/subsubcategory/*.wsp.

It would be nice if you could specify a top level directory and then allow the script to use all the subdirectories after that as the timeseries name.

I'll try to spend some time making a patch.

mshirley commented 10 years ago

This might be a good start if you know that the 6th array element of your directory structure is your top level.

whisper_file = "/var/lib/carbon/whisper/servers/hostname/network/em4/rx_packets.wsp"
'.'.join(whisper_file.replace('//','/').split('/')[6:-1])
hostname.network.em4