Simplifying P25 call handling process

[rosecitytransit]

First of all, for those who don't know you can set verbosity=10 here to see raw P25 voice channel metadata; this should ideally be exposed as a configuration option and may be best to only do it with one recorder set as the data isn't tagged.

But reading #326, #544, #674, #701, #731, #760, #773 and maybe others and the complicated call handling diagram, I'm wondering if the process could be simplified. Specifically:

• Treat control channel grant and update messages the same, and base recording logic on the voice channel and its meta data

• Eliminate the record_more_transmissions and termination flags and inactive and idle states

• Optionally let the decoding of the voice channel continue until the stream stops (e.g. there's been no more frames seen) or squelch is activated; this would help in cases of poor control channel decoding or where there aren't update messages

• Base new transmissions on seeing TDUs in the voice stream, or a change of unit or talk group ID in the voice meta data

• If the talk group in the voice meta data changes, drop frames and start a new call (if the subroutine can't do that, at least have a voice talk group variable and have the main thread start a new call on next check)

• Base unit IDs on the voice meta data, except at the start where it could come from the control channel message or when it is 0 or empty

• Make transmission mode a configuration option; don't change to a new file on TDU or unit ID change if set to off

SO:

Control channel grant/update message -> start voice channel decoding -> TDUs seen or unit ID changes -> start new file if transmission mode enabled -> data flow on voice channel ceases -> end call

[rosecitytransit]

Just realized that there is a message on the voice channel that marks the end of a call:

Example: NAC 0xe27 TDU15: LCW: ec=0, pb=0, sf=0, lco=15 : 00 00 05 00 06 e9 00 00 00, gly_errs=0

So instead of creating a recorder->last_update process, it may be as simple as removing the two process_TTDU(); calls here https://github.com/robotastic/trunk-recorder/blob/672df88ba69516cef3b928617059a9f14cbe80d9/lib/op25_repeater/lib/p25p1_fdma.cc#L401 and here https://github.com/robotastic/trunk-recorder/blob/672df88ba69516cef3b928617059a9f14cbe80d9/lib/op25_repeater/lib/p25p1_fdma.cc#L413

and instead calling it calling it inside process_LCW with I think if ((framer->duid == 0x0f) && (lco == 15)) process_TTDU();

[robotastic]

I think this is pretty close to how things are handled right now. The tricky bit is that there isn't perfect alignment between the Control Channel and the Voice Channels. The processing chain on the voice channel side can sometimes be slower than the control channel, esp on long transmissions. A new Grant call can come in on on the same TG / Freq while the Voice channel is still processing the previous transmission.

The TDU messages are used to mark the end of a transmission... but tracking the last write time is there as a fallback just in case some TDUs are missed. The P25 processing code put a tag on the stream of samples when it decodes a TDU and the wav_sink gets it and ends the current transmission it is tracking.

Of course non of this works with Analog. You can generally use the Squelch as a TDU.... I can't remember if that is being done now.

So instead of creating a recorder->last_update process, it may be as simple as removing the two process_TTDU(); calls here

trunk-recorder/lib/op25_repeater/lib/p25p1_fdma.cc Line 401 in 672df88 process_TTDU(); and here trunk-recorder/lib/op25_repeater/lib/p25p1_fdma.cc Line 413 in 672df88 process_TTDU(); and instead calling it calling it inside process_LCW with I think if ((framer->duid == 0x0f) && (lco == 15)) process_TTDU();

I am not sure what you mean here. Is this a different approach for capturing the TDUs? Are some being missed with the current approach?

[rosecitytransit]

I think this is pretty close to how things are handled right now.

Now, if the control channel messages stop, it kills the call. This sets the state to INACTIVE, which results it being ended once config.call_timeout has been reached: https://github.com/robotastic/trunk-recorder/blob/9d9def8ce7afbbd324a36ccd7cb746f7d749d73d/trunk-recorder/main.cc#L855-L859

A new Grant call can come in on on the same TG / Freq while the Voice channel is still processing the previous transmission.

With my proposal, as long as the call is still active, additional control channel messages about the call would essentially be ignored since the call already exists. The processing would all be based on the voice channel messages.

Is this a different approach for capturing the TDUs?

Correct. This switches it to only consider the TDUs that mark the end of the call, and not just the transmission. I know you'd still have to handle the other ones (or maybe watch for source ID to change) in order to know when to change to a new transmission.

[robotastic]

With my proposal, as long as the call is still active, additional control channel messages about the call would essentially be ignored since the call already exists. The processing would all be based on the voice channel messages.

One challenge I have seen is having a Grant message come in on the Control Channel before the recorder has finished recording the current call. That is what the record_more_transmissions flag helps with. Without the prompt from the Control Channel, it wouldn't know if the next transmission is from the same talkgroup or not. The talkgroup info isn't always available on the voice channel at the start of the call.

Is part of the issue that unit-to-unit calls do not always get update messages on the Control Channel?

Correct. This switches it to only consider the TDUs that mark the end of the call, and not just the transmission. I know you'd still have to handle the other ones (or maybe watch for source ID to change) in order to know when to change to a new transmission.

Oh wait - are there 2 types of TDUs? The ones at the beginning and the ones at the End? If there is a way to only get the ones at the end, that would help clean up the codes a little

[rosecitytransit]

The talkgroup info isn't always available on the voice channel at the start of the call.

That's not good. If you can't confirm that the next transmission is part of the same call or not, that does complicate things. But at least my logging (sample attached) shows it always available. I don't ever see info for unit-unit calls, but I think that's just a lack of handling. Do you have samples from systems that don't show it?

recorder2-20230403-003307.log

Is part of the issue that unit-to-unit calls do not always get update messages on the Control Channel?

Yes, but I also thought simpler might be better and avoid some of the issues that others have reported, and maybe if control channel decoding goes bad but voice channel stays good.

Oh wait - are there 2 types of TDUs?

This is a summary of what I see:

NAC 0xe27 LDU1: LCW: ec=0, pb=0, sf=0, lco=0 : 00 00 05 00 06 e9 00 89 a4, srcaddr=35236, grpaddr=1769
IMBE ...
NAC 0xe27 LDU2: ESS: algid=80, keyid=0, mi=00 00 00 00 00 00 00 00 00, rs_errs=0
IMBE ...
NAC 0xe27 TDU3:  
NAC 0xe27 TDU15:  LCW: ec=0, pb=0, sf=0, lco=0 : 00 00 05 00 06 e9 00 89 a4, srcaddr=35236, grpaddr=1769, gly_errs=0
NAC 0xe27 TDU15:  LCW: ec=0, pb=0, sf=0, lco=0 : 00 00 05 00 06 e9 00 00 00, srcaddr=0, grpaddr=1769, gly_errs=0
...
NAC 0xe27 TDU15:  LCW: ec=0, pb=0, sf=1, lco=15 : 4f 00 00 00 00 00 ff ff ff, gly_errs=0
...

There's lots of TDU15s with lco=0 which are silence after the end of a transmission, and then a few TDU15s with lco=15 that mark the end of the call.

The ones at the beginning and the ones at the End?

Have you seen them at the beginning? I think I've only seen them on unit-unit calls, normally I'd expect that there should be voice right away.

[robotastic]

Oh wait - I think you are right. The Group is always there before voice frames some - the Src ID isn't always. I am seeing TDUs at the beginning of transmissions but it could be that I am not processing things correctly.

I would be game to create a branch and see if things could be simplified a bit.

On the Voice Channel how would you know when the transmissions for a group are done and things should be uploaded? An obvious sign would be that a different talkgroup starts transmitting on the freq. you might need to have a timer in each recorder. If it has been x seconds since a TDU the recorder would close things or if it has been x sec since the last voice frame or squelch.

On trunk systems though the ground truth for what talkgroup has which freq is the control channel... but there maybe enough control info on the voice channels that we don't need to rely on it.

[rosecitytransit]

Meant to (also) send this last night

I've created a draft of my proposal of how to differentiate between transmission end and call end here: https://github.com/robotastic/trunk-recorder/commit/c48502b7ce02d87b1df4b3ba489c022702404ea7 My thought is that transmission_sink.cc (or whatever) should have the option on transmission end to either a) close the wav file/transmission and start a new one, b) keep the wav file open (enable conversation mode) or maybe c) always end the call (keeping every transmission in a separate call and maybe rely on new grant message for next transmission; would also need to process every grant message ignoring any current call)

I also coded a recorder->since_last_update process that depends on the last voice channel message rather than just the last file write at https://github.com/robotastic/trunk-recorder/commit/c7bc0bb2292c2057acdaebd6b367ba07ce5f05e6 With the change to only terminate on call end (TDU15/lco=15) I think even the troublesome unit-unit calls don't need that.

[rosecitytransit]

On the Voice Channel how would you know when the transmissions for a group are done and things should be uploaded?

In order of correctness (but any apply): 1) call termination (TDU15/lco=15) is seen; 2) different talk group appears on voice channel (in which case really should start a new call or notify main thread to) 3) no more voice frames seen 3) it's been a while since last control channel message and wav file last write 4) squelch is activated

[tadscottsmith]

On trunk systems though the ground truth for what talkgroup has which freq is the control channel... but there maybe enough control info on the voice channels that we don't need to rely on it.

If you think of it from a Subscriber Unit (radio) perspective, the control channel information is only used to tell the radio to retune to the traffic channel when a talk group it's programmed to monitor receives a grant. After that, the radio needs to get all of its information from the traffic channel and has no visibility into what's happening on the control channel.

According to the specs, on a P25 Trunked System the radio should remain on the traffic channel until one of the following occurs:

• The radio receives the call termination message (TDU15/lco=15).
• The radio receives a packet addressed to a different group address.
• The radio fails to detect any data on the traffic channel for a predetermined amount of time.
• The radio receives data related to a different service than it moved to the traffic channel for (e.g. unit-to-unit call information on a channel it tuned for a group call).

After that, the radio should return to the control channel and continue monitoring.

My only concern with this approach is that what the P25 specs consider a "call" and what you've tried to logically define as a call (e.g. a series of transmissions on the same talk group within a specified period of time), might be different. In my opinion, what sets trunk-recorder apart from other programs is the ability to provide a listening experience where short conversations are recorded as a single call. Listening "transmission by transmission" style just doesn't work on busy systems.

That being said, I'm always open to help write or test new code!

[robotastic]

That is a great point @tadscottsmith!!

and overall I am probably confusing things by trying to bunch together common transmissions, vs just keeping them in separate files.

The terminology I invented (which may not 100% match P25 specs) is: Transmission - goes from a GRANT to a TDU Calls - are multiple Transmission recordings that a merged together into a single file. A Call last from the first GRANT for a TG on a freq and stops either when another TG has a GRANT on that Freq or after a timeout period where another GRANT is given for that TG/Freq.

I think all of the fundamentals are there to easily change things around and see how they work. Right now, recording termination is based on Receiving a TDU on the Voice Channel. Right now it can get overridden if there hasn't been and Update for that TG/Freq assignment in a while, but that can be turned off. Each of the transmissions is a separate file.

@rosecitytransit I guess I am still a little confused on what you think should change and how it would simplify things.

[robotastic]

Meant to (also) send this last night

I've created a draft of my proposal of how to differentiate between transmission end and call end here: c48502b

Ah - so there are multiple types of TTDUs? Only some of which are at the end of the call? I do a check in Transmission_sink.cc to ignore the ones at the beginning - but I can make this change and get rid of that check

[robotastic]

I also coded a recorder->since_last_update process that depends on the last voice channel message rather than just the last file write at https://github.com/robotastic/trunk-recorder/commit/c7bc0bb2292c2057acdaebd6b367ba07ce5f05e6 With the change to only terminate on call end (TDU15/lco=15) I think even the troublesome unit-unit calls don't need that.

With the current approach - call->last_update() is based on the last time it received an UPDATE message on the Control Channel. This is how you know the freq is still reserved for that TG. You could switch it to the last time the Transmission Sink received a voice frame.

With this switch, the Calls would instead terminate when there has been activity for a TG on a Freq for X period of time.... which I think could work? If there is a GRANT on that Freq for a different TG, you can still set the record_more flag to false for the recorder and it would immediately stop when it receives the TDU or gets the flag for a different talkgroup on the freq.

It might be worth tracking the last UPDATE on the Control Channel still, as well as the last Sample at the Transmission Sink...

I can go make a branch and give this a try....

[rosecitytransit]

no visibility into what's happening on the control channel.

On Smart Net systems; the sub-audible data stream provides certain GRANT messages to enable priority listening. I think P25 is actually a down grade there.

what the P25 specs consider a "call" and what you've tried to logically define as a call (e.g. a series of transmissions on the same talk group within a specified period of time), might be different.

Only technically so. In the past, t-r did attempt to retune frequency if the next transmission was on a different one; this could be desirable on Smart Net systems where there's very little hang time and conversations do rapidly switch. Otherwise I think it would only be different if the time out on the system is different than the time out set in the code or though callTimeout

timeout period where another GRANT

I think that should actually be "timeout period after the last GRANT or UPDATE message"

more...

[natecarlson]

and overall I am probably confusing things by trying to bunch together common transmissions, vs just keeping them in separate files.

For the longer term, might it make sense to not bunch together the common transmissions, but instead add logic at the playback side to optionally group them? It seems like the logic would be pretty simple; group calls by talkgroup where there is less than X seconds between the end of one call and the start of another?

[rosecitytransit]

I guess I am still a little confused on what you think should change and how it would simplify things.

Check the last 2 commits on https://github.com/rosecitytransit/trunk-recorder/tree/trimet4.5 (https://github.com/robotastic/trunk-recorder/commit/c48502b7ce02d87b1df4b3ba489c022702404ea7 and https://github.com/robotastic/trunk-recorder/commit/84e0dd6cb8c2d891176f18e89f8943e04fbbb079)

Ah - so there are multiple types of TTDUs? Only some of which are at the end of the call?

Don't you have a copy of the P25 specs? But as I said, I'm seeing a TDU3 and (TDU15's w/lco!=15) that mark the end of a transmission/are silence after the end of a transmission and then there's TDU15's w/lco==15 that mark the end of a call. I've only seen a very few TDUs at the start of unit-unit calls, and those are all less than 1 second from start.

If there is a GRANT on that Freq for a different TG,

the recorder should soon gets the flag for a different talkgroup on the freq. and immediately stop

[robotastic]

@rosecitytransit - sorry about that, I just surfaced online little before lunch but then had to dive back to work. I think I have some ideas based on those commits. Let me see how much I can drive from the Voice Channel vs the Control Channel and we can see if that makes things cleaner / better.

I will also print out the different TDU values, I may have been been lump them all together - I will go checkout what I am seeing on my local systems and on SmartNet system that have digital voice channels.

[rosecitytransit]

So I've been playing around with things (see https://github.com/rosecitytransit/trunk-recorder/tree/trimet4.5 ) and I'm thinking I might just go with the recorder->since_last_update option ( https://github.com/rosecitytransit/trunk-recorder/commit/c7bc0bb2292c2057acdaebd6b367ba07ce5f05e6 ). Is there any downside to that? Seeing how you're now using GNU Radio stream tags to send data, would it be possible/better to add timestamp tags? It would not rely on successfully receiving a call termination message.

I'd be willing to open a PR with this method and maybe work on eliminating the record_more_transmissions, etc if you're interested.

[robotastic]

Cool -it is probably cleaner to keep track of the last time a voice sample came in inside the transmission sink. Tags are only passed when new samples come in.  It should also work for analog. I wouldn’t worry about ripping out the old approach yet. We can try to get a lot of people to test it out first. It may need so tweaks based upon how different systems are implemented. The Main loop can check each call for the last transmission and that can just call the recorder for each call which hits the sink. On Apr 16, 2023, at 10:56 PM, Jason McHuff @.> wrote: So I've been playing around with things (see https://github.com/rosecitytransit/trunk-recorder/tree/trimet4.5 ) and I'm thinking I might just go with the recorder->since_last_update option ( @. ). Is there any downside to that? Seeing how you're now using GNU Radio stream tags to send data, would it be possible/better to add timestamp tags? It would not rely on successfully receiving a call termination message. I'd be willing to open a PR with this method and maybe work on eliminating the record_more_transmissions, etc if you're interested.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[rosecitytransit]

it is probably cleaner to keep track of the last time a voice sample came in inside the transmission sink

Can you clarify how? The transmission_sink.cc is a part that's less clear to me. I'm just seeing that somehow, manage_calls() in main.cc needs to know when the flow of samples has stopped so the call can be closed.wond

Also, making record_more_transmissions always be true results in even the problematic unit-unit calls successfully getting recorded by transmission; I've attached the JSON details if you're interested in how it looks. (I was going to deal with things by changing the termination flag in p25p1_fdma.cc to only get set on TDU15/lco 15 and then do a work around to handle the source ID info)

102-1681690624_772356250-call_22.json.txt

Edit: Well, it appears the rx_status().last_update may indeed by Phase 1 only. But I'm wondering if you could check if either d_sample_count has stopped increasing or if there's been 0 input_items or output_items

[robotastic]

Ah - so transmission sink captures the output from OP25 or the Analog capture and collects the voice samples and metadata. It just keeps a timestamp of the last time a voice sample was passed to it. It is a GnuRadio block and part of the Record flow graph so as things get decoded they get passed directly to it. I am trying to keep as much custom code out of the OP 25 stuff as possible so that it is easy to keep up to date. Sounds good - if you are able to document things well in comments then it we can try out different variants as needed. There are so many corner cases that it takes a bit of experimentation to get things to work. On Apr 17, 2023, at 3:32 AM, Jason McHuff @.***> wrote:

it is probably cleaner to keep track of the last time a voice sample came in inside the transmission sink

Can you clarify how? The transmission_sink.cc is a part that's less clear to me. I'm just seeing that somehow, manage_calls() in main.cc needs to know when the flow of samples has stopped so the call can be closed. Also, making record_more_transmissions always be true results in even the problematic unit-unit calls successfully getting recorded by transmission; I've attached the JSON details if you're interested in how it looks. (I was going to deal with things by changing the termination flag in p25p1_fdma.cc to only get set on TDU15/lco 15 and then do a work around to handle the source ID info) 102-1681690624_772356250-call_22.json.txt

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[rosecitytransit]

I tried monitoring noutput_items, output_items.size() and input_items.size() and as I suspected, transmission_sink only gets the actual audio, while you'd need (at least I want) to know if there's anything on the voice channel, audio or TDUs/silence.

[robotastic]

Yep - that metadata info can only be passed back with Samples from the p25_framer in OP25. If you check it out that file you can see how you can pass a single NULL sample so you can add a Group or Source ID tag to it. Another option is to pass the TDU messages onto a gnu radio massage queue, the same way it is done with the trunk channel. Depending on what you want to do with that info would help determine the best way to route it. On Apr 19, 2023, at 7:33 AM, Jason McHuff @.***> wrote: I tried monitoring noutput_items, output_items.size() and input_items.size() and as I suspected, transmission_sink only gets the actual audio, while you'd need (at least I want) to know if there's anything on the voice channel, audio or TDUs/silence.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[rosecitytransit]

Depending on what you want to do with that info would help determine the best way to route it.

Just want to know when there is nothing more coming in on the voice channel and therefore the call can be closed, whether that be by checking the timestamp of the latest frame, if there's zero in the queue, or some other method. Don't need any actual data. I can do some more playing around when I have time.

[rosecitytransit]

This is what I've decided to go with; I've checked and it works for both Phase 1 and 2. https://github.com/rosecitytransit/trunk-recorder/commit/c9a6e07dd7dd3ff50a369718e13baa76d5699be7

If you want to do things differently, I'd be willing to video chat with you sometime, or maybe you could submit a PR that I could work from. I've spent WAYYYYY too much time trying to figure things out and how I could pass things from the frame assembler to the transmission sink is not something I really understand.

[robotastic]

Cool - Let me try taking a swing at it this week. I will create a branch and do something similar except move it into the Transmission Sink.

[rosecitytransit]

I decided to change from using a counter to checking the last time (check trimet4.5). Also, FWIW the unit-unit call example I posted may not be normal as there's never see trunking messages from the remote side. And when you have time, tell me your thoughts on the log file in 482.

[robotastic]

I haven't forgotten!! I am going to try and get something coded up this weekend. Sorry!

How can I get to that log file?

[rosecitytransit]

It's OK. I know you've said you have kids and apparently a newish job. I got things figured out for me. And I can be available this weekend if you wanted to discuss things.

By log file, I meant issue 482 and how a daily log file should be laid out.

[robotastic]

ok - I am doing a little experimentation. From the docs, it looks like both TDUs with a Data ID of 3 and 15 are the same. Both should be treated as terminators.

In my testing I am seeing terminators show up at the beginning of Transmissions... but I am not sure if that is an artifact of some of the Samples being stuck in a buffer some where

[rosecitytransit]

In the second call (the one that begins at 2023-04-03 00:33:26) in the log file I attached in a previous comment above does show a single TDU15 and then 3 TDU3s before the HDU and first LDU. I thought I read TDU3 is supposed to mark the end of a transmission, TDU15s w/LCO0 silence, and TDU15s w/LCO15 call end.

But my thought is to avoid being dependent on receiving those and instead simply keep decoding things until there's been a second or two with no new frames (or of course a different talk group appears). I can be available in about an hour or sometime later if you want to discuss.

[robotastic]

Gotcha - I will keep going through the Docs to see what I can find. That could totally be. I would like to keep the distinct transmissions separate, so I am going to try and keep using the TDUs for that. I didn't realize that Transmissions and Calls were actually P25 Terms! I will have to make sure I am using them sort of correctly.

I did find this mention which is interesting:

[rosecitytransit]

I would like to keep the distinct transmissions separate, so I am going to try and keep using the TDUs for that.

Oh, definitely. Though I would like an option to turn the feature off since having to combine the transmissions and delete the WAV versions seems like extra work. I see there's a conversationMode config option but it doesn't seem to do anything; I'd be willing to try to hook it up and submit a PR if you want.

I will have to make sure I am using them sort of correctly.

I believe it's working fine regarding separating transmissions. Either someone's talking or there's silence. The issue is just when to close the call; if calls could be kept open until the stream of frames end, the different types of TDUs wouldn't matter.

I did find this mention which is interesting:

I included that in my comment a month ago: https://github.com/robotastic/trunk-recorder/issues/791#issuecomment-1493816119

[natecarlson]

I would like to keep the distinct transmissions separate, so I am going to try and keep using the TDUs for that.

Oh, definitely. Though I would like an option to turn the feature off since having to combine the transmissions and delete the WAV versions seems like extra work. I see there's a conversationMode config option but it doesn't seem to do anything; I'd be willing to try to hook it up and submit a PR if you want.

Yeah, conversationmode is something that was started ages ago, never really worked properly, and got removed (I believe?)

I'd still like to see the option for each transmission (each individual transmission, ie entry on srcList) be treated separately; makes duplicate detection much easier, especially if you have calls coming from SDRTrunk too (it appears to only record the individual transmissions separately; no grouping option.) I also suspect that audio normalization would work better if each transmission was isolated, but have no evidence to back that up.

If we wanted to avoid major changes to the call logic, could just have a toggle that prevents combining the calls into a consolidated wav file at the end, and then include the individual wav files in the srcList objects. Downside is that this wouldn't help make the transmissions available faster; it'd still wait until the entire call is done before hitting the upload functions.

[rosecitytransit]

As I mentioned in this comment https://github.com/robotastic/trunk-recorder/issues/791#issuecomment-1496320926 I've thought the recording mode should be at least a 3 way option.

Upon termination flag (transmission end):

• call is always closed, putting each transmission in a separate call (future transmissions are started based upon their grant messages by ignoring check for existing call)
• (current mode) file is closed but call is kept open
• file is kept open and closed on call end

To do conversion mode only involves:

1. not closing the file in end_transmission() here https://github.com/robotastic/trunk-recorder/blob/e33e441278ae130a6884d464a5d75b97655ca5fd/trunk-recorder/gr_blocks/transmission_sink.cc#L224
2. adding close_wav() to stop_recording() https://github.com/robotastic/trunk-recorder/blob/e33e441278ae130a6884d464a5d75b97655ca5fd/trunk-recorder/gr_blocks/transmission_sink.cc#L257 since end_transmission() will no longer do it
3. having a separate sample counter so that close_wav() creates a correct header
4. Ignoring or removing the error messages about the file not closing/being open
5. modifying call concluder to handle the entire call being in the first transmission file

[Dygear]

I'm +1 for having each srcId on any given call result in a separate wav / json file. I generally keep the callTimeout value to 1 to have the same effect. Doing that via a config option could be handy and effectively ignores the callTimeout value completely.

[rosecitytransit]

I generally keep the callTimeout value to 1 to have the same effect.

I'm not sure that would always work though in cases where a transmission immediately follows another one, since there's a fixed 1.0 seconds after the last trunking message before calls go inactive and can be closed: https://github.com/robotastic/trunk-recorder/blob/e33e441278ae130a6884d464a5d75b97655ca5fd/trunk-recorder/main.cc#L869

But my first bullet option would achieve that.

[Dygear]

Yeah for sure, I'm not saying it's the same thing, but it gets me closer to the desired effect.

[robotastic]

Let's move different types of approaches for handling Transmissions / Calls to a new issue. I have to go back and check on the conversation_mode config option... I thought it was working but it must have broke along the way.

First order of business before that though, is seeing if some of the transmission recoding logic can be moved from the Trunk channel to the Voice channel and the recorders.... and then doing ton of testing to make sure it can work for:

• analog recorders
• SmartNet
• conventional P25
• P25 Phase 1 / 2
• DMR

[rosecitytransit]

Let's move different types of approaches for handling Transmissions / Calls to a new issue.

That's fine. I/we got a little off-topic.

But I think I may have it! I read your April 19th comment https://github.com/robotastic/trunk-recorder/issues/791#issuecomment-1514294085 again about sending tags and I think it may just be a matter of removing this line https://github.com/robotastic/trunk-recorder/blob/dd20c8a175d4cfdaef7635872985a3e51b2154a7/lib/op25_repeater/lib/p25_frame_assembler_impl.cc#L216 so as to always send tag(s) when there's no voice. This would also solve my issue of the length/error/spike counts not including the hang time after the last transmission. I'll open a PR.

[robotastic]

Cool! I almost have a solid branch working. Check out the new_call_mgmt branch. It is doing some TG id stuff to work with Smartnet. I should have something ready to test tonight.  On May 7, 2023, at 12:14 PM, Jason McHuff @.***> wrote:

Let's move different types of approaches for handling Transmissions / Calls to a new issue.

That's fine. I/we got a little off-topic. But I think I may have it! I read your April 19th comment #791 (comment) again about sending tags and I think it may just be a matter of removing this line so as to always send tag(s) when there's no voice. This would also solve my issue of the length/error/spike counts not including the hang time after the last transmission. I'll open a PR.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

[tadscottsmith]

ok - I am doing a little experimentation. From the docs, it looks like both TDUs with a Data ID of 3 and 15 are the same. Both should be treated as terminators.

In my testing I am seeing terminators show up at the beginning of Transmissions... but I am not sure if that is an artifact of some of the Samples being stuck in a buffer some where

@robotastic Do you have access to TIA-102.AABD-B - Trunking Procedures? It's normal to have TDUs prior to the voice message on the traffic channel.

After CC assignment the traffic channel sends ETDU/LCW (e.g. GVCU for group call) data units until the requesting SU begins its transmission.

[rosecitytransit]

I believe Luke does have a copy of the docs. But legally or not, someone uploaded (an older version of?) them to the Internet Archive: https://archive.org/details/TIA-102_Series_Documents/ the same user also uploaded MOTOTRBO documents and other stuff: https://archive.org/details/MototrboDocuments

Also, I have my call handling proposal here: https://github.com/rosecitytransit/trunk-recorder/tree/tdus-to-tsink particularly https://github.com/rosecitytransit/trunk-recorder/commit/a3ae26e13dd799b6eeef7b0fc13ea8b2e3e07769

With that, transmission_sink knows when the call is active and not just a transmission, so the recorder states could be replaced by tracking the timestamp of the last sample (they could be tracked in main.cc instead, and be just recording or stopped). record_more_transmissions could also be eliminated since it should now keep going until the flow of samples ends, as well as possibly the call IDLE state. Also, if there's not, there needs to be a way to halt recording if the talk group changes on the voice channel; I think by doing a comparison here and similar for other recorders: https://github.com/robotastic/trunk-recorder/blob/f7ece07fea315df95e86fca2269c3a8c1c6b96a3/lib/op25_repeater/lib/p25_frame_assembler_impl.cc#L191

[tadscottsmith]

I believe Luke does have a copy of the docs. But legally or not, someone uploaded (an older version of?) them to the Internet Archive: https://archive.org/details/TIA-102_Series_Documents/ the same user also uploaded MOTOTRBO documents and other stuff: https://archive.org/details/MototrboDocuments

Also, I have my proposal here: https://github.com/rosecitytransit/trunk-recorder/tree/tdus-to-tsink particularly rosecitytransit@a3ae26e

I asked because TIA-102.AABD-B is not in that location, and is the document that thoroughly explains the handoff from the control channel to the traffic channels and how radios should switch back and forth.

[rosecitytransit]

Check out the new_call_mgmt branch

Looks nice, but I can't get it to pick up the control channel, even though I verified it's there using heatmap.py, had been testing my tdus-to-tsink branch and even tried deleting everything in the working folder and starting from scratch.

[robotastic]

@tadscottsmith Thanks - that is a good doc, just paged through it.

Is Unit to Group the standard type of call? Is Group the same thing as a Talkgroup? The weird thing about this diagram is that the Call Terminator comes from the RFSS, but that might just be because it is relaying a voice channel terminator from an SU?

Potentially the default timeout is 3 seconds - this list the Max, Default & Min. Based on the references, this may just be for System ID on the Control Channel.

I have definitely noticed that if you remove a Call after there has been a gap of only 1 or 2 secs of voice transmissions on the traffic channel, you will still get UPDATE messages for that group on the Control Channel.

It does look like GRANTS and UPDATES should sort of be treated the same... I think the trick will be not removing a Call when a Recorder has timed out, while the TG is still getting updates on the Control Channel:

the solution for this could just be to wait until both the Recorder has timed out based on the Traffic Channel and the Call has timed out based on UPDATEs on the Control Channel.

[robotastic]

@rosecitytransit I just pulled in the changes from the Master branch - it will work for P25 systems now. I think this has a lot of what you are proposing. Calls timeout based on the last Voice Frame that was received. If a different Talkgroup appears on a Voice Channel, the Call will also stop.

I haven't seen any special TDUs or DATA frames on the Voice Channel that designate when a single SU has stop transmitting vs when the Control Channel has stopped allocating a Voice Channel for a Talkgroup. I have been using a SmartNet system for testing though... I will also look at my local P25 system today.

[rosecitytransit]

Is Unit to Group the standard type of call? Is Group the same thing as a Talkgroup? The weird thing about this diagram is that the Call Terminator comes from the RFSS, but that might just be because it is relaying a voice channel terminator from an SU?

Without having the document, my guess would be "yes" to all those questions.

It does look like GRANTS and UPDATES should sort of be treated the same... I think the trick will be not removing a Call when a Recorder has timed out, while the TG is still getting updates on the Control Channel:

This is on the list of things I want to propose or fix, and I thought was mentioned in an issue or PR a long time ago but can't find it now.

the solution for this could just be to wait until both the Recorder has timed out based on the Traffic Channel and the Call has timed out based on UPDATEs on the Control Channel.

I would think no more data on the traffic channel would be the definitive guide, especially with the decoding being somewhat delayed vs the control channel.

I have been using a SmartNet system for testing though

I've never played with (low-level) SmartNet stuff as the systems around here have long been since replaced, but I thought I've read it's pretty messy to deal with. In any case, as I've said, I think you can reference allocation based on there being something (anything) coming in on the voice channel and monitoring for that stopping.

[robotastic]

It does look like GRANTS and UPDATES should sort of be treated the same... I think the trick will be not removing a Call when a Recorder has timed out, while the TG is still getting updates on the Control Channel:

This is on the list of things I want to propose or fix, and I thought was mentioned in an issue or PR a long time ago but can't find it now.

Yep - I have switched back and forth from having it this.

the solution for this could just be to wait until both the Recorder has timed out based on the Traffic Channel and the Call has timed out based on UPDATEs on the Control Channel.

I would think no more data on the traffic channel would be the definitive guide, especially with the decoding being somewhat delayed vs the control channel.

The potential challenge here is that I am seeing the Control Channel continue to send UPDATEs after the Voice Channel has timed out. This would cause some thrashing, because the initial Call for the TG would be closed and then a new, additional Call would be started up based on the UPDATE.

One solution, as mentioned above, would be to only remove a Call when both the Voice and Control Channels have timed out from not receiving a message for the Call.

I have been using a SmartNet system for testing though

I've never played with (low-level) SmartNet stuff as the systems around here have long been since replaced, but I thought I've read it's pretty messy to deal with. In any case, as I've said, I think you can reference allocation based on there being something (anything) coming in on the voice channel and monitoring for that stopping.

yep - that is what it is doing now. Actually, it is based on the last Voice Frame coming in on the channel... but I could switch that to include any data packets too.

[rosecitytransit]

Also, I'm experiencing weirdness using my trimet4.5 branch. I'm ending calls based on last_voice_frame https://github.com/rosecitytransit/trunk-recorder/blob/6beb1fb508e5955c66257b751095c2d3af1a5f01/lib/op25_repeater/lib/p25_frame_assembler_impl.cc#L188-L190 or when (recorder->since_last_write() > config.call_timeout) and 1 of the 4 systems I'm recording from is working fine, on another one, on another, last_voice_frame doesn't seem to be working but recorder->since_last_write() is and on another neither are and I think lots of calls are never closing, only closing when there's a frequency overlap. Check out the log excerpt: http://www.rosecitytransit.org/files/05-08-2023_0047_00.log.gz

I may try switching to either my tdus-to-tsink approach or try your new_call_mgmt again.

[rosecitytransit]

One solution, as mentioned above, would be to only remove a Call when both the Voice and Control Channels have timed out from not receiving a message for the Call.

Or just wait until a few seconds after the voice channel time out. But wouldn't the delay in voice processing help here?

Actually, it is based on the last Voice Frame coming in on the channel... but I could switch that to include any data packets too.

By "data packets" do you mean TDUs, etc in the hang time following transmissions? Because you'd need to watch for those to know whether the call has ended or not, to wait for any further transmissions. If you're only looking for audio, there could be late update messages for the hang time.