Suggestions for multi-site configurations

[vabiro]

Hi,

I'd like to tap into the braintrust here for some suggestions on how I should/could configure for my environment. I have scanned through the posts here and can't seem to find a similar configuration to what my municipality has done.

We have a Motorola P25 (partially) Phase 2 P25 system with 4 sites (http://bit.ly/2CZszcy) North, South, East, West. To monitor the entire city I have tried including all the (1) CCs in a single system , (2) setup separate systems for each site, or (3) just focusing on the south tower, which is the most active, and seems to have the most overlap.

There are problems with each of the configurations: 1: I rapidly run out of recorders (I have 4x4sdrs) because much of the traffic is duplicated on two + sites. I also get multiple recordings of the same radio traffic; 2: Same results as 1, but it is even better at getting duplicate recordings for some reason; 3: Not all traffic touches the south tower;

Is there a way to make sure that multiple sites, with the same system ID and TGIDs don't run more than one recorder for one TGID activity?

Cheers,

[amlethojalen]

I have a similar issue, in NSW Australia we have the GRN Network, which is a Motorola Trunked system.

I have a site (K) about 10km away which has about 10 data channels, and i have another site (G) about 25 kms away that has about 20 data channels.

The problem that i would want to cater for is that i can decode the control channel 100% site K & G, with the data channels i get 100% from site K but only about 50% from site G. Site G has a lot more capacity so there are alot more calls patched though it. What i would to do is put a priority on site K, so if the same talkgroup is is patched on both sites then default to site F, but if talkgroup is only patched to site G then attempt to decode on site G. The patches change hour to hour depending on the utilization of the system.

thinking out loud I guess if detect the TG on site G first, start recording, if then the TG is detected on site K, i guess i could just retune to the K freq.

Thinking about your issue @vabiro , i guess when a call is added, check to see if there is an existing call with the same TG, if there is dont add the new (duplicate) call. I guess it we could have two systems that are 'linked', and the application wouldn't allow two (or more) recordings of the TG at the same time within the 'linked' systems.

[bctrainers]

Is there a way to make sure that multiple sites, with the same system ID and TGIDs don't run more than one recorder for one TGID activity?

This may be a feature request here for what you're saying... Ensure only one recording is active with two of the same system ID configurations.

While this doesn't exist on the stock trunk-recorder, such a feature somewhat exists on the repo here: https://github.com/Treehouseman/trunk-recorder - with sysID and sysSite variables.

[robotastic]

Adding a feature like @bctrainers would probably be the easiest approach and it would prevent duplicates. With this approach, the first system you hear the call on, you record on. I think it could just be a change to this line: https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687

What would be more challenging is having: 1) a prioritization rank for different sites or 2) some sort of weighting based on signal strength.

With 1) it could be do-able, we would just need to delay for some amount of time to collect which sites the call is available on and then pick the best site.

2) is trickier because I am not sure of a good way to get signal strength for a channel in Gnuradio and you would also probably have to record for a bit to get signal strength.

[amlethojalen]

Yep for @vabiro changes, i was thinking just to change https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687 ,guess its just a matter of enumerating all the matching systems and see if the TG is currently being recorded.

For my problem, id prefer 1) over 2). I know where my sdr is located, and i have a fixed priority. What i was thinking was

• set a priority to a system from config
• link the system (either manually in config, or using the sysid and syssite)
• start recording the call as soon as it arrives
• retune the recorder if the call arrives on a system with a better priority

From what i can work out the issues with implementing it this way

• a call would start on one system, and effectivly end on a second system ** looking in the call class i dont see a big deal with it, assuming the same upload scripts are used for both systems, the system reference only seems to be used for the short name.
• don't know how a recorder would go with a retune in the middle of a datastream

Otherwise a simple delay would work as well, i dont know what the latency is of the calls between the two sites, assuming it would be <500ms, then delaying might be the safest option, although it would be a little more complex to implement, we would need to store the TrunkMessages for later processing (after the delay)

• when an incoming TrunkMessage is received check to see if it is already being recorded, discard message check to see if there is already a delayed TrunkMessage for the same TalkGroup, if yes, add the TrunkMessage to the Talkgroup message collection. ** check to see if there is already a delayed TrunkMessage for the same TalkGroup, if no, then create a collection to store all the TrunkMessages for the TalkGroup, create a one shot timer for the delay
• When the delay timer ticks for a given talkgroup then find the 'best' TrunkMessage, and start recording.
• clean up the Talkgroup message collection.

Hmmm, adding priority (either hard coded or based on signal strength) seems a little too complicated.

[kcwebby]

Just to add my $0.02, this is something I've wanted for a while... however, my "recorders" are spread out geographically, and therefore are on separate PC's for the best reception..

So what I have done, is start to implement a "home recorder" and assigning talk groups to those recorders that have the highest likelihood of reception. I.E. the most actual subscribers associated with it on a particular talkgroup, so that we have the "best chance" of RX that talkgroup 100% of the time. This generally lines up to be the site closest to the dispatch center, or the city center of the talkgroup. Signal strength isn't really an issue for me with my setup, but if I could do the type of setup that your refering to (which is similar to a voter), then I could potentially get away with less recording sites.

I have approx 30 SDR's across four recording PC's linked up to the same database, for the different areas of our regional system. Since our system is regional and capable of roaming from tower to tower, we have to be flexible as far as which talkgroups are available on which towers. It's flexible as heck, but a logic problem to be able to get 100% of the recordings.

On Wed, Jan 10, 2018 at 3:44 PM, Amleth Ojalen notifications@github.com wrote:

Yep for @vabiro https://github.com/vabiro changes, i was thinking just to change https://github.com/robotastic/trunk-recorder/blob/master/ trunk-recorder/main.cc#L687 ,guess its just a matter of enumerating all the matching systems and see if the TG is currently being recorded.

For my problem, id prefer 1) over 2). I know where my sdr is located, and i have a fixed priority. What i was thinking was

• set a priority to a system from config
• link the system (either manually in config, or using the sysid and syssite)
• start recording the call as soon as it arrives
• retune the recorder if the call arrives on a system with a better priority

From what i can work out the issues with implementing it this way

• a call would start on one system, and effectivly end on a second system ** looking in the call class i dont see a big deal with it, assuming the same upload scripts are used for both systems, the system reference only seems to be used for the short name.
• don't know how a recorder would go with a retune in the middle of a datastream

Otherwise a simple delay would work as well, i dont know what the latency is of the calls between the two sites, assuming it would be <500ms, then delaying might be the safest option, although it would be a little more complex to implement, we would need to store the TrunkMessages for later processing (after the delay)

• when an incoming TrunkMessage is received check to see if it is already being recorded, discard message check to see if there is already a delayed TrunkMessage for the same TalkGroup, if yes, add the TrunkMessage to the Talkgroup message collection. ** check to see if there is already a delayed TrunkMessage for the same TalkGroup, if no, then create a collection to store all the TrunkMessages for the TalkGroup, create a one shot timer for the delay
• When the delay timer ticks for a given talkgroup then find the 'best' TrunkMessage, and start recording.
• clean up the Talkgroup message collection.

Hmmm, adding priority (either hard coded or based on signal strength) seems a little too complicated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/163#issuecomment-356746725, or mute the thread https://github.com/notifications/unsubscribe-auth/AKTZAq0JFU6U2e6s0F20MpJUxaJeRctxks5tJS8tgaJpZM4RYzky .

[vabiro]

Wow, it sounds like I'm not the only one having a similar problem.

@robotastic In your suggestion to change https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687 would that be something like adding && call_found != true since it would have been set to true during an earlier call? That would definitely fix the immediate problems of multiple recordings and recorders being used up.

The idea of being able to set a priority site for a TGID would also work in some situations. In my environment the city is divided into quadrants, but each service does it differently. For example, fire's North Tac 1 should have most activity on the North site. The problem is that EMS's South East has an equal chance of being closer to the East tower or the South.

The idea of having signal strength as a deciding factor would be ideal, since it would insure that the recording would be the best available, as opposed to the first past the post. As you suggested, this will pose its own set of problems. My biggest concern would be delays starting recording that may result in missed short transmissions. If there was a way to spool the available recordings in RAM until the best SS is determined, but I suspect that has its own set of problems.

[robotastic]

Yea - so instead of checking the Sys_num (which is just the position of that system in config file), it would check the Sys_ID (which is the ID used in RF system). That was if the TG was already being recorded by a different system with the same Sys_ID, it wouldn’t be recorded a second time. The problem to avoid is if there are 2 conventional systems, where is SYS_ID is not available, we want to make sure that both systems could record TG 1 at the same time. So I will have to add in some sort of fallback to sys_num if SYS_ID is not available.

On Jan 11, 2018, at 1:07 AM, vabiro notifications@github.com wrote:

Wow, it sounds like I'm not the only one having a similar problem.

@robotastic https://github.com/robotastic In your suggestion to change https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687 https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687 would that be something like adding && call_found != true since it would have been set to true during an earlier call? That would definitely fix the immediate problems of multiple recordings and recorders being used up.

The idea of being able to set a priority site for a TGID would also work in some situations. In my environment the city is divided into quadrants, so fire's North Tac 1 should have most activity on the North site. The problem is that EMS's South East has an equal chance of being closer to the East tower or the South.

The idea of having signal strength as a deciding factor would be ideal, since it would insure that the recording would be the best available, as opposed to the first past the post. As you suggested, this will pose its own set of problems. My biggest concern would be delays starting recording that may result in missed short transmissions. If there was a way to spool the available recordings in RAM until the best SS is determined, but I suspect that has its own set of problems.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/163#issuecomment-356835845, or mute the thread https://github.com/notifications/unsubscribe-auth/AAG53GvGUMIWfgUC4wb121M4pZThFB6yks5tJaUHgaJpZM4RYzky.

[kcwebby]

Luke,

The other complication to this, is what do you do when you physical seperate the trunk-recorders.. I.E. multiple instances, seperated by geography on a big regional or a national system.

If there was a priority that was confirmed to make it not record, you could just handle it that way.. I know I don't want to turn into a "whitelist only" because they are adding users and talkgroups to our system on a regular basis and I want to continue to operate in a "discovery" type mode, while ignoring the talkgroups.

What I'm planning on doing is having the "same" list of talkgroups that operate system wide, and then setting the priority specifically for each instance of trunk-recorder, so that each site records only those talkgroups closest to them.

Is that a possibility with the priority field?

Thanks

On Wed, Jan 10, 2018 at 7:22 AM Luke Berndt notifications@github.com wrote:

Adding a feature like @bctrainers https://github.com/bctrainers would probably be the easiest approach and it would prevent duplicates. With this approach, the first system you hear the call on, you record on. I think it could just be a change to this line: https://github.com/robotastic/trunk-recorder/blob/master/trunk-recorder/main.cc#L687

What would be more challenging is having: 1) a prioritization rank for different sites or 2) some sort of weighting based on signal strength.

With 1) it could be do-able, we would just need to delay for some amount of time to collect which sites the call is available on and then pick the best site.

1. is trickier because I am not sure of a good way to get signal strength for a channel in Gnuradio and you would also probably have to record for a bit to get signal strength.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/robotastic/trunk-recorder/issues/163#issuecomment-356600673, or mute the thread https://github.com/notifications/unsubscribe-auth/AKTZAu4QAsWW6rErY3kQpX_JAihLnfEIks5tJLmwgaJpZM4RYzky .

[ab0tj]

Any luck with this one? I live in an area covered by multiple sites of the same system, but some sites have a different set of talkgroups than others. Being able to grab the traffic from multiple sites and ignoring duplicates would be great.

[VVakko]

In a similar situation, I use a small script to remove duplicates, and select the best file with the least number of errors. The script does not completely eliminate duplicates, but there are significantly fewer of them.

#!/bin/bash

# Rdio Scanner Data for Upload
rdio_url="rdio-server:3000"
rdio_key="aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee"

# Temporary Directories
tmpfs="/app/tmpfs"
tmpfs_locks="${tmpfs}/locks"
tmpfs_media="${tmpfs}/media"
[ -d "${tmpfs_locks}" ] || mkdir -p ${tmpfs_locks}
[ -d "${tmpfs_media}" ] || mkdir -p ${tmpfs_media}

# Extract Radio Data from Filename
system="${1:-0}"
filename="${2%.*}"
site="${filename#*/}"
site="${site#*/}"
site="${site#*/}"
site="${site%%/*}"
temp="${filename%-call*}"
temp="${temp##*-}"
time="${temp%_*}"
date=$(date -ud @${time} +"%Y-%m-%dT%T.%3NZ")
freq="${temp#*_}"
tgid="${filename##*/}"
tgid="${tgid%-call*}"
tgid="${tgid%-*}"

#echo "File: ${filename}"
#echo "Syst: ${system}"
#echo "Site: ${site}"
#echo "Time: ${time}"
#echo "Date: ${date}"
#echo "Freq: ${freq}"
#echo "TGID: ${tgid}"

tmp_files="${tmpfs_media}/${tgid}-${time}"
ln -sr "${filename}.wav"  "${tmp_files}-${site}.wav"
ln -sr "${filename}.json" "${tmp_files}-${site}.json"

tmp_lock="${tmpfs_locks}/${tgid}-${time}.lock"

sleep 0.$[ ( $RANDOM % 10 ) + 1 ]s
if [ -f "${tmp_lock}" ]; then
    echo "Skip duplicate ${filename}.wav"
    exit 0
else
    touch "${tmp_lock}"
    sleep 1.0s
fi

best_error_count=999999
for json in ${tmp_files}-*.json; do
    error_count=`jq -r '.freqList[0].error_count' ${json}`
    comparison=`echo "if (${error_count} < ${best_error_count}) 1 else 0" | bc`
    #echo $json:$error_count:$comparison:$best_error_count
    if [[ ${comparison} -eq 1 ]]; then
    #    echo "SAVE: $error_count --> best_error_count"
        best_error_count=${error_count}
        best_json=${json}
    #else
    #    echo "SKIP: $error_count > best_error_count"
    fi
done

filename=${best_json%.*}

#echo "best_error_count: ${best_error_count}"
#echo "best_json: ${best_json}"
#echo "json: ${filename}"
echo "Save wav ${filename}.wav"

curl --silent "http://${rdio_url}/api/trunk-recorder-call-upload" \
     -F "key=${rdio_key}" -F "system=${system}" \
     -F "audio=@${filename}.wav;type=audio/wav" \
     -F "meta=@${filename}.json;type=application/json"

rm -f "${tmp_lock}"
rm -f ${tmp_files}-*

[UberPlexCa]

I wonder with the plugin system if we could do this some way. I run multiple sites but with remote boxes covering a large area. I just removed the duplicating talkgroups from the .csv file. But I wish there was a way to each remote box to know what other ones are capturing and to stop them.

Communication between the boxes is easy I have them all on TailScale.

[robotastic]

Right now there are not really the hooks in the plugin system for real commands to come from the plugins to trunk recorder. It could be possible to build out a "should_record" hook that would be called at the start of each recording. The different site recorders could coordinate among themselves.

I still want to build out a better version that would have a central director that would get in all the raw trunking messages and decide which sites should record.... not sure I will ever have the time to get that done though.

[UberPlexCa]

That would be cool, but yeah I would expect time is the factor.

[robotastic]

There was a PR that adds support for multi-site. Closing, but if there are suggestion for improving the current approach, please open a new Issue.