Setting up Jabber4Linux

nicodds commented 1 month ago

Hello, thank you for this light in a land of darkness!

I'm trying to setup Jabber4Linux using the information available in the configuration of my Cisco Jabber. Unfortunately I'm not able to get it up and running. Is there any guide you can share?

All the best

schorschii commented 1 month ago

Hi,

The README.md contains some basic setup instructions. Can you please describe in detail where you are stuck?

nicodds commented 1 month ago

Thank you for your reply. Essentially I'm essentially stuck trying to input the correct values in Jabber4Linux startup dialog:

Screenshot from 2024-07-17 16-33-26

I'm guessing that as server I need to put the value in the last row in the image (configuration of Cisco Jabber installed on a virtual machine), while as port the default value (3804) I found in docs/CAPF Protocol Specification.md.

Neverthless, when I try to login, I get a timeout error, so I think there something that is wrong in the value I used. The same happens when I try the other entries in the Cisco Jabber Configuration.

Is there something I'm missing?

All the best, Domenico

schorschii commented 1 month ago

Normally, the server name is auto-discovered via DNS. But maybe you don't have a search domain set on your Linux machine or your organization does not have set the SRV records but configures Cisco Jabber via group policy. Does dig _cisco-uds._tcp.YOURDOMAIN.COM SRV find any server?

When clicking "Login", Jabber4Linux contacts the Cisco UDS API which uses port 8443 per default. Please try this port instead. The CAPF service on port 3804 cannot be used for logging in. In my environment, the UDS API is served by all servers (TFTP/CT/CCMCIP) visible in the Cisco Jabber, but this may vary per configuration. If port 8443 is not working, can you please check which ports are open on your Cisco servers using nmap servername.YOURDOMAIN.COM?

Please also click on the settings icon in Cisco Jabber -> Help -> Connection status. Is there an UDS (HTTPS) server listed? And which port is used?

nicodds commented 1 month ago

Hello Georg, thank you again for your support. I was able to find the UDS address in the dialog at the path "Cisco Jabber -> Help -> Connection status". It listens both on 443 and 8443.

I used these information in Jabber4Linux login dialog, but I got the error that the certificate cannot be verified, since it is self-signed.

I followed the instructions in the README ("SIP Transport Encryption (SIPS)"). Unfortunately, "Own Certificates" was empty and I was able to find a Cisco related certificate in "... Organization" (I don't know the exact name in the english localization), but it cannot be exported in PKCS #12.

So, I pointed my Firefox browser to the same address and I downloaded the pem certificate from the about:certificate shortcut. Then I moved the certificate file to "~/.config/jabber4linux/client-certificates" and a I tried back to login. Unfortunately, that doesn't work again, with the same error.

All the best, Domenico

nicodds commented 1 month ago

Hi again! Just for the record, I made some progress.

Digging the code and the web, I've found that with requests you can directly pass a custom certificate using the REQUESTS_CA_BUNDLE env variable.

So, in a terminal, I exported the variable with the path of the certificate and then I launched jabber4linux. I've got the following error:

(GUI ERROR DIALOG) HTTPSConnectionPool(host='long.host.name.it', port=443): Max retries exceeded with url: /cucm-uds/user/myuser (Caused by SSLError(SSLCertVerificationError(1, "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: Hostname mismatch, certificate is not valid for 'long.host.name.it'. (_ssl.c:1000)")))

But the certificate is taken directly from "long.host.name.it" :-(

####### UPDATE #######

I was able to further debug the issue. The DNS query you suggested pointed out that long.host.name.it is in reality written LONG.host.name.it and when I use openssl to check the hostname, it match with LONG.host.name.it. Neverthless, when I use LONG.host.name.it in the login dialog, it seems it gets lowercased and the certificate verification fails.

It seems this behavior originates from requests (or imported libraries). I tried to perform the GET directly from ipython using "LONG.host.name.it" and I get the same error as before.

Have you any idea?

Thanks in advance, Domenico

schorschii commented 1 month ago

Hi,

Good to see that we make some progress. Please do not mix up the SIPS client cert with the server cert(s), these are different things. To complete the confusion, there can be multiple server certs: one for the UDS API and one for SIPS. And since you said that "Own Certs" in the Windows cert store is empty, it seems that you don't have a client cert. We ignore this for now (maybe your Cisco Jabber is configured to not encrypt phone calls).

Let's solve the server cert problem. In my environment, the UDS API cert is "a real one", issued by a common CA so that every client trusts it by default, while the SIPS cert is self-signed. That's why I created the ability to trust specific self-signed SIPS certs by putting them into the server-certificates dir. The UDS connection did not honor these certs because it was not needed for my environment - but I added this functionality now with the last commit.

So please test out the new version from the master branch and put your server cert into ~/.config/jabber4linux/server-certificates. The certificate you got exported from Firefox should work (and it's maybe the same for UDS and SIPS in your case). If not, please get the certificates via openssl again:

openssl s_client -connect server.yourdomain.com:8443  # the UDS API cert
openssl s_client -connect server.yourdomain.com:5061  # the SIPS cert

From the command output, copy everything including -----BEGIN CERTIFICATE----- to -----END CERTIFICATE----- into a file, so that you then have 2 certificate files in ~/.config/jabber4linux/server-certificates (file extension does not matter).

Host names in certificates are not case sensitive, so that's not the problem here.

Kind regards

nicodds commented 1 month ago

Georg! I was able to login. Nevertheless, I had to make some small changes to the code in UDSWrapper.py to handle the case in which devices have no description (in case, I can send you a patch, but it is essentially a matter of two try... except blocks.

I made a few calls and they worked great. I had only a problem when calling or receiving calls from a colleague using a physical (cisco) phone. I attach the error logged:

.....
User-Agent: Cisco-CUCM11.5
Max-Forwards: 70
CSeq: 101 ACK
Allow-Events: presence
Session-ID: ********************************************
Content-Length: 0

Traceback (most recent call last):
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 120, in run
    self.handleSipMessage(header.strip(), '')
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 202, in handleSipMessage
    dstAddress, dstPort, payloadType, payloadTypeMap = self.parseSdpBody(body)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 496, in parseSdpBody
    for key, value in attrs['m'].items():
                      ~~~~~^^^^^
KeyError: 'm'
(GUI ERROR DIALOG) 'm'

If you are interested, I can further investigate in the next days.

All the best, Domenico

schorschii commented 1 month ago

Nice! Yes, please post your changes to UDSWrapper.py here so I can evaluate and implement it.

For the other problem, it would be great if you can provide the problematic SDP packet content here. The easiest would be to insert a print() here:

    def parseSdpBody(self, body):
        print(body)
        ...

nicodds commented 1 month ago

In UDSWrapper.py I had to handle the case of the empty tag (as in my case) when getting device information. I essentially modifiied two methods:

    def getDevices(self):
        url = f'https://{self.serverName}:{self.serverPort}/cucm-uds/user/{urllib.parse.quote(self.username)}/devices'
        with self.http_session.get(url, headers={'Authorization':self.basic_auth(self.username,self.password)}) as result:
            result.raise_for_status()
            if(self.debug): print(url, '::', result.text, "\n")
            document = minidom.parseString(result.text).documentElement
            values = []
            for item in document.getElementsByTagName('device'):
                try:
                    device_description = item.getElementsByTagName('description')[0].firstChild.data
                except AttributeError:
                    device_description = 'Unknown Description'
                values.append({
                    'id': item.getElementsByTagName('id')[0].firstChild.data,
                    'name': item.getElementsByTagName('name')[0].firstChild.data,
                    'type': item.getElementsByTagName('type')[0].firstChild.data,
                    'model': item.getElementsByTagName('model')[0].firstChild.data,
                    'description': device_description
                })
            return values

and

def getDevice(self, id):
        url = f'https://{self.serverName}:{self.serverPort}/cucm-uds/user/{urllib.parse.quote(self.username)}/device/{urllib.parse.quote(id)}'
        with self.http_session.get(url, headers={'Authorization':self.basic_auth(self.username,self.password)}) as result:
            result.raise_for_status()
            if(self.debug): print(url, '::', result.text, "\n")
            document = minidom.parseString(result.text).documentElement
            try:
                device_description = document.getElementsByTagName('description')[0].firstChild.data
            except AttributeError:
                device_description = 'Unknown Description'

            values = {
                'id': document.getElementsByTagName('id')[0].firstChild.data,
                'name': document.getElementsByTagName('name')[0].firstChild.data,
                'type': document.getElementsByTagName('type')[0].firstChild.data,
                'model': document.getElementsByTagName('model')[0].firstChild.data,
                'description': device_description,
                'deviceName': document.getElementsByTagName('name')[0].firstChild.data,
                'number': None,
                'contact': None,
                'callManagers': [],
                'deviceSecurityMode': '0',
                'certHash': None,
                'capfServers': [],
            }

## other code removed

Sorry for not being able to send you a full patch, but I'm setting up a new laptop while working and I've very few spare time. I also modified the code in SipHandler.py, I'll send you an example failing packet ASAP.

All the best, Domenico

nicodds commented 1 month ago

As promised, the details of the packets exchanged with the physical cisco phone (it seems the body is empty):

=== INCOMING SIP MESSAGE ===
INVITE sip:REDACTED@10.196.249.25:35526;transport=tcp SIP/2.0
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d4763b6f5f
From: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
To: <sip:REDACTED@10.199.245.133>
Date: Fri, 19 Jul 2024 08:46:29 GMT
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
Supported: timer,resource-priority,replaces
Min-SE:  3600
User-Agent: Cisco-CUCM11.5
Allow: INVITE, OPTIONS, INFO, BYE, CANCEL, ACK, PRACK, UPDATE, REFER, SUBSCRIBE, NOTIFY
CSeq: 101 INVITE
Expires: 180
Allow-Events: presence
Call-Info: <urn:x-cisco-remotecc:callinfo>; security= Unknown; orientation= from; gci= 3-14955303; isVoip; call-instance= 1
Send-Info: conference, x-cisco-conference
Alert-Info: <file://Bellcore-dr1/>
Session-ID: 20d2394000105000a00070f09651a8bd;remote=00000000000000000000000000000000
Remote-Party-ID: <sip:REDACTED@10.199.245.133;x-cisco-callback-number=REDACTED>;party=calling;screen=yes;privacy=off
Contact: <sip:REDACTED@10.199.245.133:5060;transport=tcp>;+u.sip!devicename.ccm.cisco.com="SEP70F09651A8BD"
Max-Forwards: 69
Content-Length: 0

=== OUTGOING SIP MESSAGE ===
SIP/2.0 100 Trying
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d4763b6f5f
From: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
To: <sip:REDACTED@10.199.245.133>;tag=e3350560df4a18ef735776f6-abcb96fd
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
Session-ID: 4e70422d1acffe994eef19f0687cf613;remote=20d2394000105000a00070f09651a8bd
Date: Fri, 19 Jul 2024 10:46:29 
CSeq: 101 INVITE
Server: Cisco-CSF
Contact: <sip:REDACTED@10.196.249.25:35526;transport=tcp>;+u.sip!devicename.ccm.cisco.com="CSFREDACTED"
Allow: ACK,BYE,CANCEL,INVITE,NOTIFY,OPTIONS,REFER,REGISTER,UPDATE,SUBSCRIBE,INFO
Supported: replaces,join,sdp-anat,norefersub,resource-priority,extended-refer,X-cisco-callinfo,X-cisco-serviceuri,X-cisco-escapecodes,X-cisco-service-control,X-cisco-srtp-fallback,X-cisco-monrec,X-cisco-config,X-cisco-sis-7.0.0,X-cisco-xsi-8.5.1
Allow-Events: kpml,dialog
Recv-Info: conference
Recv-Info: x-cisco-conference
Content-Length: 0

=== OUTGOING SIP MESSAGE ===
SIP/2.0 180 Ringing
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d4763b6f5f
From: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
To: <sip:REDACTED@10.199.245.133>;tag=e3350560df4a18ef735776f6-abcb96fd
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
Session-ID: 4e70422d1acffe994eef19f0687cf613;remote=20d2394000105000a00070f09651a8bd
Date: Fri, 19 Jul 2024 10:46:29 
CSeq: 101 INVITE
Server: Cisco-CSF
Contact: <sip:REDACTED@10.196.249.25:35526;transport=tcp>;+u.sip!devicename.ccm.cisco.com="CSFREDACTED"
Remote-Party-ID: "None" <sip:REDACTED@10.199.245.133>;party=called;id-type=subscriber;privacy=off;screen=yes
Allow: ACK,BYE,CANCEL,INVITE,NOTIFY,OPTIONS,REFER,REGISTER,UPDATE,SUBSCRIBE,INFO
Supported: replaces,join,sdp-anat,norefersub,resource-priority,extended-refer,X-cisco-callinfo,X-cisco-serviceuri,X-cisco-escapecodes,X-cisco-service-control,X-cisco-srtp-fallback,X-cisco-monrec,X-cisco-config,X-cisco-sis-7.0.0,X-cisco-xsi-8.5.1
Allow-Events: kpml,dialog
Content-Length: 0

:: using default ringtone output device  []
:: using default output device  None
=== OUTGOING SIP MESSAGE ===
SIP/2.0 200 OK
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d4763b6f5f
From: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
To: <sip:REDACTED@10.199.245.133>;tag=e3350560df4a18ef735776f6-abcb96fd
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
Session-ID: 4e70422d1acffe994eef19f0687cf613;remote=20d2394000105000a00070f09651a8bd
Date: Fri, 19 Jul 2024 10:46:33 
CSeq: 101 INVITE
Server: Cisco-CSF
Contact: <sip:REDACTED@10.196.249.25:35526;transport=tcp>;+u.sip!devicename.ccm.cisco.com="CSFREDACTED"
Remote-Party-ID: "None" <sip:REDACTED@10.199.245.133>;party=called;id-type=subscriber;privacy=off;screen=yes
Allow: ACK,BYE,CANCEL,INVITE,NOTIFY,OPTIONS,REFER,REGISTER,UPDATE,SUBSCRIBE,INFO
Supported: replaces,join,sdp-anat,norefersub,resource-priority,extended-refer,X-cisco-callinfo,X-cisco-serviceuri,X-cisco-escapecodes,X-cisco-service-control,X-cisco-srtp-fallback,X-cisco-monrec,X-cisco-config,X-cisco-sis-7.0.0,X-cisco-xsi-8.5.1
Allow-Events: kpml,dialog
Recv-Info: conference
Recv-Info: x-cisco-conference
Content-Type: application/sdp
Content-Disposition: session;handling=optional
Content-Length: 466

v=0
o=Cisco-SIPUA 22437 0 IN IP4 10.196.249.25
s=SIP Call
b=AS:4000
t=0 0
a=cisco-mari:v1
a=cisco-mari-rate
m=audio 33501 RTP/AVP 114 0 8 111 101
c=IN IP4 10.196.249.25
a=rtpmap:114 opus/48000/2
a=rtpmap:0 PCMU/8000
a=rtpmap:8 PCMA/8000
a=rtpmap:111 x-ulpfecuc/8000
a=extmap:14/sendrecv http://protocols.cisco.com/timestamp#100us
a=fmtp:111 max_esel=1420;m=8;max_n=32;FEC_ORDER=FEC_SRTP
a=rtpmap:101 telephone-event/8000
a=fmtp:101 0-16
a=sendrecv

:: opened UDP socket on port 33501 for incoming RTP stream
=== INCOMING SIP MESSAGE ===
ACK sip:REDACTED@10.196.249.25:35526;transport=tcp SIP/2.0
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d94e6a2679
From: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
To: <sip:REDACTED@10.199.245.133>;tag=e3350560df4a18ef735776f6-abcb96fd
Date: Fri, 19 Jul 2024 08:46:29 GMT
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
User-Agent: Cisco-CUCM11.5
Max-Forwards: 70
CSeq: 101 ACK
Allow-Events: presence
Session-ID: 20d2394000105000a00070f09651a8bd;remote=4e70422d1acffe994eef19f0687cf613
Content-Length: 0

Traceback (most recent call last):
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 120, in run
    self.handleSipMessage(header.strip(), '')
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 202, in handleSipMessage
    dstAddress, dstPort, payloadType, payloadTypeMap = self.parseSdpBody(body)
                                                       ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/share/jabber4linux/venv/lib/python3.12/site-packages/jabber4linux/SipHandler.py", line 498, in parseSdpBody
    for key, value in attrs['m'].items():
                      ~~~~~^^^^^
KeyError: 'm'
(GUI ERROR DIALOG) 'm'
=== OUTGOING SIP MESSAGE ===
BYE sip:REDACTED@10.199.245.133;transport=tcp SIP/2.0
Via: SIP/2.0/TCP 10.199.245.133:5060;branch=z9hG4bK4ea4d4763b6f5f
From: <sip:REDACTED@10.199.245.133>;tag=e3350560df4a18ef735776f6-abcb96fd
To: <sip:REDACTED@10.199.245.133>;tag=25772101~9f1775fa-0e9b-4036-aa72-02ca008dcf9d-59792824
Call-ID: 62b56600-69a127e5-4252b9-85f5c70a@10.199.245.133
Max-Forwards: 70
Session-ID: 4e70422d1acffe994eef19f0687cf613;remote=20d2394000105000a00070f09651a8bd
Date: Fri, 19 Jul 2024 10:46:37 
CSeq: 101 BYE
User-Agent: Cisco-CSF
Content-Length: 0

:: closed UDP socket for incoming RTP stream

schorschii commented 1 month ago

Thanks for the information. I added a commit to the master branch similar to your solution to make the device description optional.

Your SIP/SDP log is interesting. The calling phone never sends a SDP packet to you. It should ether do this in the INVITE message ("early offer") or in the ACK message ("late offer"). The only SDP visible is the outgoing SDP sent from Jabber4Linux. At the first sight, it looks like a firmware bug of the Cisco phone. Can you tell me which model the physical phone is? (we are using CP-7841 here and they are working)

nicodds commented 1 month ago

Hello Georg, the phones used in my company are the Cisco IP Phone 7821 (https://www.cisco.com/c/en/us/products/collaboration-endpoints/unified-ip-phone-7821/index.html). From what I've seen, the firmware is really old; latest update (on the phone checked) is 2017. I'll try to perform an update with a colleague and report back to you the results.

schorschii commented 1 month ago

If the firmware update does not make any difference, maybe you can send me a Wireshark recording from the original Cisco Jabber establishing a call with the problematic phone. Since you don't have a client certificate, the SIP messages should be visible unencrypted. After starting Wireshark on the correct interface, just type in "sip" as filter. Now when calling, you should see something like this: Unbenannt

From the file menu, you can save it as .pcapng and send the file to me.

schorschii commented 1 month ago

Update for public reference:

Wireshark trace showed that the phone uses G729 codec only. PCMA/PCMU seams to be disabled by administrator.
Advertising G729 support in J4L brought the phone to send a SDP back.
Implementing G729 codec
- Favorite solution is to use the libbcg729-0 from Ubuntu repos, similar like it's already implemented for the Opus codec (libopus0 Ubuntu package).
- The difference is that Python bindings for libopus already exists with the "opuslib" PyPI package. For libbcg729, we have do make the Python bindings ourselves.
- https://github.com/AlexIII/g729a-python is our fallback solution. It has examples and looks easy to use but has the disadvantage that another native C library must be compiled and shipped with the Python code.

schorschii commented 1 month ago

Update: I did a first attempt with libbcg729 Python bindings (see linked commit in g729 branch). Decoding is working but encoding currently segfaults. There is something wrong with how I call the bcg729Encoder() function (my C knowledge is limited). I'm still trying to figure it out.

nicodds commented 1 month ago

Thank you for your work. I was poking with the library too, yesterday. Honestly, I see it really poorly documented, but anyway at least we have it.

I'll try to figure out the problem, but I was thinking that this may result in a second package that offer a fully featured python wrapper for libbcg729

schorschii commented 1 month ago

Success! Thanks to my colleague @jzmp the encoder is now working. Can you please test the g729 branch @nicodds ?

Yes, if everything works, we can publish the libbcg729 bindings in a separate package on PyPI similar to the opuslib.

nicodds commented 1 month ago

Great news! I've just tested with a colleague, it works pretty well.

I just noticed that occasionally the quality of the call is lower than with the softphone. Is this due to the codec that overall has a lower quality with respect to opus?

Anyway, thank you very much and thanks also to @jzmp !

schorschii commented 1 month ago

Great, thanks for your feedback. Yes, the quality of G729 is not as good as Opus, also because of the lower sample rate of 8kHz used in G729 Annex A. Opus is used with 48kHz even in VoIP applications.

The G729 bindings are now released as separate package on PyPI: https://pypi.org/project/g729lib/

And I released v0.6.0 of J4L. Can you please test the new release (Debian package)?

nicodds commented 1 month ago

And I released v0.6.0 of J4L. Can you please test the new release (Debian package)?

Just tested, it works smoothly!

schorschii commented 1 month ago

Thanks. I'm closing this issue since all problems are solved, ok?

nicodds commented 1 month ago

Great work, thank you!

schorschii / Jabber4Linux

Setting up Jabber4Linux #15