registration of landline number via voice failing

[fabbra]

I have an issue (very similar to #628) to register a Swiss landline number using signal-cli 0.10.5 saying NonNormalizedPhoneNumberException: StatusCode: 400:

signal-cli --verbose -a +41XXXXXXXXX register --voice --captcha "03A...kpA"

• my Swiss landline number is considered as valid by libphonenumber (Test site)
• LANG is set to en_US.UTF-8

The following is the verbose output when running on my raspberry Pi 4 (compiled signal-cli according to these instructions):

2022-04-21T13:15:41.945+0200 [main] INFO  LibSignal - [libsignal]: rust/bridge/jni/src/logging.rs:156: Initializing libsignal version:0.15.0
2022-04-21T13:15:41.957+0200 [main] DEBUG org.asamk.signal.util.IOUtils - XDG_DATA_HOME not set, falling back to home dir
Request verify error: StatusCode: 400
org.whispersystems.signalservice.api.push.exceptions.NonNormalizedPhoneNumberException: StatusCode: 400
        at org.whispersystems.signalservice.api.push.exceptions.NonNormalizedPhoneNumberException.forResponse(NonNormalizedPhoneNumberException.java:19)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket$VerificationCodeResponseHandler.handle(PushServiceSocket.java:2543)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.makeServiceRequest(PushServiceSocket.java:1709)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.makeServiceRequest(PushServiceSocket.java:1693)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.makeServiceBodyRequest(PushServiceSocket.java:1682)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.makeServiceRequest(PushServiceSocket.java:1618)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.makeServiceRequest(PushServiceSocket.java:1606)
        at org.whispersystems.signalservice.internal.push.PushServiceSocket.requestVoiceVerificationCode(PushServiceSocket.java:331)
        at org.whispersystems.signalservice.api.SignalServiceAccountManager.requestVoiceVerificationCode(SignalServiceAccountManager.java:265)
        at org.asamk.signal.manager.util.NumberVerificationUtils.requestVerificationCode(NumberVerificationUtils.java:28)
        at org.asamk.signal.manager.RegistrationManagerImpl.register(RegistrationManagerImpl.java:102)
        at org.asamk.signal.commands.RegisterCommand.register(RegisterCommand.java:66)
        at org.asamk.signal.commands.RegisterCommand.handleCommand(RegisterCommand.java:42)
        at org.asamk.signal.App.handleRegistrationCommand(App.java:245)
        at org.asamk.signal.App.init(App.java:205)
        at org.asamk.signal.Main.main(Main.java:58)

To be sure it is not a problem of the platform I did the same test on a Win64 machine resulting in the same error.

Any suggestions on how to fix this would be very much appreciated.

[AsamK]

Please copy the issue description here, instead of just linking to another comment.

The signal server complains that the phone number is not in its normalized form. On the libphonenumber page you could check the formatting section of your number is displayed differently there.

[fabbra]

The phone number passed as input to signal-cli is equivalent to the phone number in "E164 format" printed in "Formatting Results" by libphonenumber.

Do we agree that E164 is the right format to pass to signal-cli? All others contain spaces which signal-cli does not seem to appreciate that much.

[AsamK]

Yes, it should be in E164 format. This is the code that does the checking on the Signal-Server: https://github.com/signalapp/Signal-Server/blob/9b3a8897cdfab4e830b3caa7f5f300ed25fedea9/service/src/main/java/org/whispersystems/textsecuregcm/util/Util.java#L57 Maybe it's using a different libphonenumber version ...

[AsamK]

I've extended the error message for this case, it should now print the normalized phone number expected by the server. You can get a development build here for testing: https://github.com/AsamK/signal-cli/actions/runs/2367581221

[fabbra]

Thanks for the commit.

However, unfortunately this does not seem to provide much more information since Expected normalized is null:

2022-05-23T20:44:19.421+0200 [main] INFO  LibSignal - [libsignal]: rust/bridge/jni/src/logging.rs:156: Initializing libsignal version:0.17.0
2022-05-23T20:44:19.434+0200 [main] DEBUG org.asamk.signal.util.IOUtils - XDG_DATA_HOME not set, falling back to home dir
Failed to register: Phone number is not normalized (StatusCode: 400). Expected normalized: null
org.asamk.signal.manager.api.NonNormalizedPhoneNumberException: Phone number is not normalized (StatusCode: 400). Expected normalized: null
        at org.asamk.signal.manager.util.NumberVerificationUtils.requestVerificationCode(NumberVerificationUtils.java:47)
        at org.asamk.signal.manager.RegistrationManagerImpl.register(RegistrationManagerImpl.java:112)
        at org.asamk.signal.commands.RegisterCommand.register(RegisterCommand.java:67)
        at org.asamk.signal.commands.RegisterCommand.handleCommand(RegisterCommand.java:43)
        at org.asamk.signal.App.handleRegistrationCommand(App.java:245)
        at org.asamk.signal.App.init(App.java:205)
        at org.asamk.signal.Main.main(Main.java:58)

However, if I try with a fake(?) Swiss mobile number (e.g. +41791234567 and add add another zero +410791234567 it properly spits out the expected normalized number of +41791234567).

Failed to register: Phone number is not normalized (StatusCode: 400). Expected normalized: +41791234567
org.asamk.signal.manager.api.NonNormalizedPhoneNumberException: Phone number is not normalized (StatusCode: 400). Expected normalized: +41791234567

Edit: If I do the same (i.e., insert a 0 after the country code +41) with my Swiss landline number (e.g., +410xxxxxxxxx) as one would expect I get the message Expected normalized: +41xxxxxxxxx. If I then copy & paste this exact expected normalized number into the command line and re-execute the command I get the following (also shown at the top):

Failed to register: Phone number is not normalized (StatusCode: 400). Expected normalized: null
org.asamk.signal.manager.api.NonNormalizedPhoneNumberException: Phone number is not normalized (StatusCode: 400). Expected normalized: null

What to do?

[fabbra]

Could it be related to issued with LANG or LANGUAGE environment variables?

On my system LANG = en_US.UTF-8 but LANGUAGE is not set. Do I have to set the two of them or is only one of them sufficient?

Edit: Having also set LANGUAGE = en_US.UTF-8 did not help.

[AsamK]

Thanks for testing, so it looks like libsignal-service is parsing the server error incorrectly ... I'll take a look what else could trigger that. The LANG env variable shouldn't be an issue anymore, i added a workaround for that already.

[najamelan]

I run into the same issue, expected normalized: null.

[DrOteonu-Nanquanu]

For me the same, from the Netherlands. I tried with two different phone numbers, one mobile, one VOIP. I have been trying for more than 2 hours, to no avail.

signal-cli -a [E164 number as indicated on libphonenumber site] register --captcha [CAPTCHA obtained from signalcapthcas.org]

Language environment variable settings:

LANG=en_US.UTF-8 LANGUAGE=en_US:en

I checked both numbers with the libphonenumber site. They were indeed correct and in the correct E164 format. However, I always get a "Failed to register: Phone number is not normalized (StatusCode: 400). Expected normalized: null." error.

[AsamK]

I can't reproduce this with a German landline number.

From the server code, the 400 status code can only happen when the language or the number is wrong. https://github.com/signalapp/Signal-Server/blob/9b3a8897cdfab4e830b3caa7f5f300ed25fedea9/service/src/main/java/org/whispersystems/textsecuregcm/controllers/AccountController.java#L219

Have you tried registering the number with one of the mobile apps? If that works, it would rule out an issue with the number itself.

[DrOteonu-Nanquanu]

Have you tried registering the number with one of the mobile apps? If that works, it would rule out an issue with the number itself.

What do you mean with "one of the mobile apps"? I have entered the phone number in http://www.libphonenumber.appspot.com. The result was a positive validation:

Validation Results

Result from isPossibleNumber()
true
Result from isValidNumber()
true
Result from isValidNumberForRegion()
true
Phone Number region
NL
Result from getNumberType()
MOBILE

I used the E164 number that appeared in the same table:

Formatting Results

E164 format   [my number in E164 format, something like +31[number without spaces] ]
...

So I copied that number into my signal-cli command.

[AsamK]

What do you mean with "one of the mobile apps"?

One of the official clients: https://signal.org/de/download/

[DrOteonu-Nanquanu]

O, as a matter of fact, that mobile number is already used for another signal account (on my Android smartphone, using the standard mobile app indeed). So, yes, it works on the Signal android client.

However, isn't http://www.libphonenumber.appspot.com/ already a sufficient test? Or does it sometimes produce false positives?

[DrOteonu-Nanquanu]

Also strange, this is a very standard phone number from one of the largest mobile phone providers in the Netherlands (KPN)... I have never had trouble anyone reaching me on it... These numbers typically start with +316 (landcode NL +31, and then a 6). Couldn't be more standard and ordinary. So that is what amazed me.

Also, I just logged into my account at that provider, and it states that the number is working perfectly, that the prepaid account is filled etc.

[AsamK]

Checking with libphonenumber should be enough and as it works with the android app, I'm pretty sure the issue is not with the number. I've added some logging for the language parameter, to check if the issue is somehow with the way java parses the system language. Can you try again with this development build: https://github.com/AsamK/signal-cli/actions/runs/2448618420 And run signal-cli with the --verbose parameter.

[MatzFan]

Having the same issue here with the latest build and a valid number (I did a successful Twilio test voice verification on my landline using the same E164 format I'm passing to signal-cli). Saw no logging of issues with my system language. I even tried an older version (0.10.4.2) too and got the same error.

I can't reproduce this with a German landline number.

Has anyone else managed to register a number since this issue was raised - might the bug be elsewhere?

I'd be happy to invest some time in trying to fix this myself. Is it possible to build signal-cli along with a fork of the patched libsignal-service-java dependency locally so I can debug them together? Not used Gradle before, but building signal-cli was straightforward enough.

[AsamK]

Thanks for looking into it. If you have the libsignal-service-java repo locally, you can run ./gradlew publishToMavenLocal there to deploy it to the local maven repository ($HOME/.m2). After that you can build signal-cli and it will use the local libsignal-service-java library version.

[MatzFan]

@AsamK perfect, will do that.

[MatzFan]

so it looks like libsignal-service is parsing the server error incorrectly

Yes. I think I found the problem by printing the responseBody.string() for the error response in the VerificationCodeResponseHandler here. Lo and behold, although the error code was 400, the message was "too few parts". Interesting.

Turns out that error is thrown by the RecaptchaClient parseInputToken() method here. So nothing to do with the phone number provided. Currently it looks like the VerificationCodeResponseHandler logs the 400 response body, but it doesn't show up in debugging with --verbose. It also assumes a NonNormalizedPhoneNumberException for any case with a non-empty response body - hence the red herring.

Looks to me like the parseInputToken() method expects not just the captcha token but something like "signal-recaptcha-v2.6LfBXs0bAAAAAAjkDyyI1Lk5gBAUWfhI_bIyox5W.registration.03AG..rest-of-the-base64-token-here" I tried that with an expired captcha token and got a new error; 402 Payment Required. I'll try tomorrow with a valid capture token from an IP address in same location as my phone.

Has the server code changed - i.e. was it previously possible to register with a signal-cli arg --captcha = 03AG...? Anyhow, hope this helps.

[AsamK]

Thanks for analyzing! Yeah the whole captcha string needs to be provided there. The format of the captcha changed a while back, when they switched to a different recaptcha version. I guess I'll adapt libsignal-service to improve parsing of the BadRequest exception so that signal-cli can provide a better error message.

[AsamK]

Error message should be improved now. With 7bb690e58e56ab81ad85f811e438f8024c6cc495 this particular captcha issue should now print this server error: Failed to register: [400] Response: {"code":400,"message":"too few parts"} Not great but better than before.

[fabbra]

Thanks for investigating this in detail.

Adding "signal-recaptcha-v2.6LfBXs0bAAAAAAjkDyyI1Lk5gBAUWfhI_bIyox5W.registration." to the front of the captcha did the job for me.