Add support for BTech DM1701 / Retevis RT84

[hmatuschek]

The radio is very similar to other TyT/Retevis radios e.g. TyT MD-UV390. To this end, it is very likely that only the codeplug must be verified. As the binary codeplug can be extracted from the CPS file, I do not need a radio at hand. However, at least the USB VID/PID pair should be verified. Better, I get a wireshark capture of a codeplug read, write and a call-sign DB write.

Verify codeplug:

• [x] Timestamp
• [x] General settings.
• [x] Menu settings
• [x] Button settings
• [x] Messages
• [x] Encryption
• [x] Emergency systems
• [x] Group lists
• [x] Zones
• [x] Scan lists
• [x] VFO Channels
• [x] Channel encoding
• [x] DMR Contacts
• [x] Test implementation

Call-sign database:

The radio supports a call-sign database. It is, however, hard to reverse engineer this DB format without the radio at hand. To this end, I need a wireshark capture of a DB upload to the radio.

• [x] Get wireshark capture
• [x] Verify/implement DB format
• [ ] Test implementation

[hmatuschek]

@kimballen, Started implementation of the DM1701 in the branch of the same name. However, I need some information to actually enable the implementation. This concerns how the radio identifies itself. Could you send be the output of the command

dmrconf --verbose detect

when the radio is connected to the computer? If I am correct, the device will use the same interface as many TyT and Retevis devices and thus an error should mention some unknown TyT/Retevis device.

[hmatuschek]

@kimballen, I've verified the DM-1701 codeplug and fixed some minor issues. All that is left to do is to test the implementation in the dm1701 branch.

[AndresPlazaR2]

I just recently acquired a DM-1701. Let me know of you need anything to help implement the DB upload.

[hmatuschek]

There are a few things to be done. First it would be great if you could test the current implementation. That is, if all the settings are programmed/read properly and you are able to configure the device using qdmr. This was kind of a blind implementation. I do not own the device, so I cannot test it. If everything works, I can update the feature matrix at https://dm3mat.darc.de/qdmr/#dev.

It would be great if you could program some call-sign DBs onto the radio and capture the USB traffic. The latter is possible under Windows using wireshark (https://dm3mat.darc.de/qdmr/manual/ch08.html#reverseEngineeringProtocol). Start with only 1 and 2 entries in the DB and then program a full one. This way I might be able to reverse engineer the database format.

[AndresPlazaR2]

I tested qdmr with my DM-1701, and these are my comments from the matrix (for firmware version 2.003):

• FM Channels: can configure them in qdmr and they work correctly in the radio.
• DMR Channels: can configure them in qdmr and they work correctly in the radio.
• DMR Contacts: can configure them in qdmr and they work correctly in the radio.
• Group Lists: can configure them in qdmr and they work correctly in the radio.
• Zones: can configure them in qdmr and they work correctly in the radio.
• Scan Lists: Although I configured them in qdmr, and there is no error uploading them, they don't seem to be configured/working in the radio.
• VOX: haven't tried this yet. Any specific functionality that needs to be tested here?
• CTCSS: whatever is selected for channels, in qdmr, seems to be configured properly and working in the radio.
• DCS: haven't tried this yet.

I will capture the call-sign DB programing using wireshark and post it here; probably I will do that over the weekend.

Thanks!

[hmatuschek]

Thanks for testing it. I'll have a look at the scan list issue. I'll open a new issue for that. Concerning the VOX, just to test if enabling it, will work. Sometime, the VOX sensitivity/level must be set in the general settings.

[AndresPlazaR2]

Sorry for the delay: the Windows software for the radio was failing when trying to load the CSV file with the contacts, and I could not understand why... It turns out, you need Excel for it to work (I guess it uses some DLL from it). Disgusting. Anyways, attached are the Wireshark captures and the source CSV file for each one. Let me know if you need another test. I will give it a try to the VOX thing next (but it will be in a couple of days). dmrid_many.csv.gz dmrid_many.pcapng.gz dmrid_one.csv.gz dmrid_one.pcapng.gz dmrid_two.csv.gz dmrid_two.pcapng.gz

[hmatuschek]

Ok, thanks for the captures. I'll have a look at it.

[hmatuschek]

Ok, looks like the format other Retevis/TyT devices use for their call-sign DB. Except, that the index appears to be unused.

[hmatuschek]

Again, thanks for the captures. However, there appears to be an issue with the upload of the "many" contacts. The index is not used, because the data was imported incorrectly. Likely, because the fields are separated by spaces instead of comma. If you find time, could you recapture that. Maybe even one, where the entire 120k entries are used? E.g., the entire DMR-ID User DB.

[hmatuschek]

It appears to be a common TyT callsign DB, matching the memory addresses and element layout. To this end, I've added it in the dm1701 branch. If you like, you can test it there.

[AndresPlazaR2]

Attached are the CSV and PCAPNG files for the "whole" contacts DB transfer to the radio; let me know if that works.

I will compile and try that branch tomorrow; thanks!

[AndresPlazaR2]

It appears to be a common TyT callsign DB, matching the memory addresses and element layout. To this end, I've added it in the dm1701 branch. If you like, you can test it there.

OK, I compiled branch dm1701, and these are my observations:

• When Call-Sign DB is set to limit the number of DB entries (in this case, to 120k), whatever you select using the default DMR ID or a list of closest DMR IDs, it uploads the data and it seems fine in the radio (even with the error message below). This is an example debug output:

Debug in src/application.cc@680: Sort call-signs closest to ID=3196799. Debug in src/application.cc@694: Limit callsign DB entries to 120000. Debug in lib/tyt_radio.cc@71: Encode call-sign DB. Debug in src/application.cc@708: Start call-sign DB write... Debug in lib/tyt_radio.cc@249: Check alignment. Debug in lib/tyt_radio.cc@258: Erase memory section for call-sign DB. Debug in lib/tyt_radio.cc@264: Upload 1 elements. ERROR in lib/dfu_libusb.cc@238: Cannot get status: Input/Output Error. ERROR in lib/dfu_libusb.cc@200: Cannot write to device: No such device (it may have been disconnected). ERROR in lib/dfu_libusb.cc@431: Cannot leave DFU mode. Debug in src/application.cc@736: Write complete.

• When Call-Sign DB is set to NOT limit the number of DB entries, it crashes:

Debug in src/application.cc@680: Sort call-signs closest to ID=3196799. Debug in lib/tyt_radio.cc@71: Encode call-sign DB. Segmentation fault (core dumped)

[hmatuschek]

The error message in the first example is likely a false alarm. The radio reboots before I can close the connection. So qdmr fails at closing the connection to the device. Anyway, the latter is certainly a bug.

[AndresPlazaR2]

Regarding VOX: I have it to 1 as a default in settings. I changed it to 3 in one channel, save the codeplug and uploaded it, but it still shows it as 1 in that channel.

[hmatuschek]

The error message in the first example is likely a false alarm. The radio reboots before I can close the connection. So qdmr fails at closing the connection to the device. Anyway, the latter is certainly a bug.

Should be fixed in the dm1701 branch.

[hmatuschek]

Regarding VOX: I have it to 1 as a default in settings. I changed it to 3 in one channel, save the codeplug and uploaded it, but it still shows it as 1 in that channel.

The VOX sensitivity can only be set globally for the DM-1701. However, you can enable/disable it for each channel individually. So during encoding I can only check, if a particular channel has VOX enabled and during decoding, I use the global setting if the VOX is enabled. This, however, is a limitation of the DM-1701. Other radios allow for specifying a per-channel VOX sensitivity.

[AndresPlazaR2]

The error message in the first example is likely a false alarm. The radio reboots before I can close the connection. So qdmr fails at closing the connection to the device. Anyway, the latter is certainly a bug.

Should be fixed in the dm1701 branch.

Tested the changes in the branch. Observations:

• Whatever you use the default DMR ID or a list of prefixes to create the call-sign DB, without a limit in the number of entries, now it works and it does not crashes.
• When you load less entries than the limit (122197), it seems that the first entry is repeated several times at the end, to complete the max limit.

[hmatuschek]

Oh, that is strange. I hope, that I can reproduce it here without the radio at hand.

[AndresPlazaR2]

Regarding VOX: I have it to 1 as a default in settings. I changed it to 3 in one channel, save the codeplug and uploaded it, but it still shows it as 1 in that channel.

The VOX sensitivity can only be set globally for the DM-1701. However, you can enable/disable it for each channel individually. So during encoding I can only check, if a particular channel has VOX enabled and during decoding, I use the global setting if the VOX is enabled. This, however, is a limitation of the DM-1701. Other radios allow for specifying a per-channel VOX sensitivity.

Understood. I tested setting the VOX value in the Settings tab of qdmr, and that correctly sets the value for channels in the radio. But changing that value to anything other than off, or setting the vox value in tytExtension->menuSettings to true, does not activate VOX in the radio's main configuration.

[hmatuschek]

Ok, the binary encoded call-sign DB looks OK. That is, by calling dmrconf, you can encode the DB

dmrconf encode-db --radio=dm1701 --id=2621370 --limit=100 dm1701_callsign_db.dfu

You can then inspect the contents with

dmrconf info dm1701_callsign_db.dfu | less

Looks fine to me, the remaining memory is filled with 0xff.

[AndresPlazaR2]

Ok, the binary encoded call-sign DB looks OK. That is, by calling dmrconf, you can encode the DB

dmrconf encode-db --radio=dm1701 --id=2621370 --limit=100 dm1701_callsign_db.dfu

You can then inspect the contents with

dmrconf info dm1701_callsign_db.dfu | less

Looks fine to me, the remaining memory is filled with 0xff.

It may be a bug in the firmware of the radio, then?

[hmatuschek]

Maybe, or I misunderstood the format and the firmware gets confused by that.

[hmatuschek]

Ok, the VOX flag in the menu settings, should enable the VOX menu. I've just checked the bits with the manufacturer CPS. Now, I'll verify the VOX settings in the channel and main settings.

[AndresPlazaR2]

dmrconf encode-db --radio=dm1701 --id=2621370 --limit=100 dm1701_callsign_db.dfu

Oh, this is a different case than the one I was describing: that command only takes the 100 first entries given a default DMR ID; that works and only that number of entries are uploaded to the radio.

But I don't think, with dmrconf, there is a way to select a prefix(s) with less than 122K entries, but not giving it a limit of number of entries (you can do that in qdmr's Settings tab). This is the case that produces this behavior.

[hmatuschek]

Oh, that is interesting.

[hmatuschek]

Regarding VOX: I have it to 1 as a default in settings. I changed it to 3 in one channel, save the codeplug and uploaded it, but it still shows it as 1 in that channel.

The VOX sensitivity can only be set globally for the DM-1701. However, you can enable/disable it for each channel individually. So during encoding I can only check, if a particular channel has VOX enabled and during decoding, I use the global setting if the VOX is enabled. This, however, is a limitation of the DM-1701. Other radios allow for specifying a per-channel VOX sensitivity.

Understood. I tested setting the VOX value in the Settings tab of qdmr, and that correctly sets the value for channels in the radio. But changing that value to anything other than off, or setting the vox value in tytExtension->menuSettings to true, does not activate VOX in the radio's main configuration.

Double checked the addresses of the settings. Both in channels and the main settings.

[hmatuschek]

dmrconf encode-db --radio=dm1701 --id=2621370 --limit=100 dm1701_callsign_db.dfu

Oh, this is a different case than the one I was describing: that command only takes the 100 first entries given a default DMR ID; that works and only that number of entries are uploaded to the radio.

But I don't think, with dmrconf, there is a way to select a prefix(s) with less than 122K entries, but not giving it a limit of number of entries (you can do that in qdmr's Settings tab). This is the case that produces this behavior.

Oh, wait. Those prefixes do not select IDs, they define the priority those with these prefixes are selected when the DB is assembled. That is, the IDs with these prefixes are selected first, when the DB is filled to the max.

[AndresPlazaR2]

Oh, wait. Those prefixes do not select IDs, they define the priority those with these prefixes are selected when the DB is assembled. That is, the IDs with these prefixes are selected first, when the DB is filled to the max.

Interesting.

On Settings, Limit number of DB entries is not checked, Select using my DMR ID is not checked and Select using prefixes has 730 as a value. After uploading the DB to the radio, if I check the CSV contacts, there are 122197 entries, the first entry is for AA1JN, the second one is for another callsign, and if I go to the end of the list, the last entries are for AA1JN (but I have not checked how many, because it is not easy to jump in the list).

[hmatuschek]

Ok, could you try to use dmrconf to write the call-sign DB and check if it behaves properly? I need to narrow it down. Just call

dmrconf write-db --id=730

This should be the same result as with qdmr with the settings you mentioned. The encoded table looks different to mine.

[AndresPlazaR2]

dmrconf write-db --id=730

That command, for me, reproduces the same result but with different data: the first entry is KF6BFL, and that's the one that repeats at the end.

[AndresPlazaR2]

@hmatuschek , I'm still puzzled about the call-sign DB. It does not display correctly in the list of contacts, and snooping the USB data transferred (to upload the call-sign DB) from the original software and dmrconf look different:

• original software: the DMR ID is stored in ASCII, on as many bytes as needed. The field separator is 0x09 (tab).
• dmrconf: the DMR ID is stored in binary on 3 bytes (I mean the binary representation of those 3 bytes is the DMR ID), and little endian. It doesn't seem to be a clear field separator.

I wonder if this is the reason why the DB seems corrupted while looking at it in the radio.