Closed eric-brechemier closed 5 years ago
At this step, the account has been created but we need to take a few additional steps to build trust between us and the system. We will:
provide your regular phone number in international format
get some details about intended usage for this number:
opt out of "onboarding support" over the phone:
Click the button to request the verification code.
Your regular phone number may be a mobile phone or a landline. In order to verify the number, a code will be sent to this number. I used a landline, and in this case I got a 4-digit code read to me over the phone by a nice robot lady, who repeated the code distinctly a few times.
You can now answer a few questions to "customize your experience".
I chose to instead.
You get a welcome message, which you can close:
and a suggestion to which we'll pass.
Trial numbers are taken from a limited pool of numbers that are reused and may already be scored negatively or blacklisted with some recipients for some of their past usage. You do not want to use them for anything serious.
We will upgrade immediately instead. Let's click .
We can also simply navigate to the Upgrade section found in the menu on the left.
We have already completed step 1: Verify a Caller ID.
We will now register our credit card and top up.
Now that our account has been upgraded, we can choose a virtual phone number from the whole pool of numbers available in Twilio.
I have described separately how to choose a memorable phone number.
I will now describe the steps to subscribe to the chosen number.
At the bottom of the tab in the sidebar, you can find a link to . Click this link to access a list of Twilio products and services.
In this list, you can select the different items that you are going to use in this project, and that will appear as sections in your sidebar. To this end, you need to each item in turn, in the order that you want them in the sidebar.
I would suggest the following list of items, in this order:
The Messaging and Twilio Functions are optional as they are not yet used in this guide. Messaging allows to configure actions to do when SMS messages are sent and received. And Twilio Functions offer more advanced scripting capabilities than the older TwiML Bins that we will be using, which have the advantage of being much simpler.
We'll start our setup with .
In the section, we'll start by removing our regular number from the Verified Caller IDs. Our goal is to have a virtual mobile phone number that is used on its own, and does not advertise our regular phone number at any point. So let's just make sure that it does not happen by mistake:
search for numbers and choose a memorable phone numbers with Voice and SMS capabilities (more details here)
select the number of your choice, check its type (e.g. not "Tollfree") and the monthly fee (typically $1 or $2) then click the button at the end of its row.
check the capabilities of the number (Voice + SMS) and the local requirements, which depend on the type of number and the country:
click to confirm the subscription.
the subscription is now confirmed:
Depending on the regulatory requirements of the country of origin of the phone number, you may need to submit additional documents to prove your identity and your address. This is the case in France.
For a mobile phone number in France, as an individual, you need to provide:
If you provide separate documents, it is important that they all display your name in exactly the same way.
When I subscribed to my mobile phone number back in October 2018, I was not made aware of these requirements. I had only submitted my address, which gets validated by checking that the address exists, but does not require documents as proof. I assume that the document submission process was put in place at a later point. It is only on 11 April 2019 that I received an email asking me to review the actions needed to comply with regulatory requirements.
My passport displays my name and my own address. Thus I only need to upload this single document.
click the button
select the type of end user: business or individual
click the button
optionally, fill the details of the end user
click the button
click the button then select one or several supporting documents to upload. In my case, I selected a copy of my passport.
fill in details of the uploaded documents, e.g. for a French passport:
click the button
give a descriptive name to the bundle of documents: e.g. "Proof of Identity and Address in France"
click the button
provide your email address again, to receive a notification once the submitted documents have been approved:
click the button
check the above information. By submitting these documents, you declare that the information is accurate and you agree to their processing by Twilio, which is described here:
I declare that the information provided above is accurate. I acknowledge that Twilio will be processing the information provided above for the purpose of identity verification, and will be sharing this information with my local telecommunications providers or authorities where required by local law. I understand that Twilio phone numbers may be taken out of service inaccurate or false information.
We are required to collect this information to fulfill local regulatory obligations regarding identity verification for telephone numbers. We will safeguard and protect any data we collect, and we will keep your personal data safe and confidential. Please see Twilio's Privacy Statement for details about how we protect and secure your personal data.
click the button to complete the submission.
The documents have been submitted. The bundle is now listed in Identity Documents, with status pending:
Within two business days, you should get notified of the approval of the documents. You will then need to go back to Twilio settings to associate the verified bundle with the phone number in France.
I received two notifications by email, about 4 hours later (much shorter than the announced delay of 48 hours). The two notifications are very similar.
The first one is for the uploaded document, my passport:
We'd like to let you know that the supporting Document accompanying the end-user Identity request labeled ‘PASSPORT-(...).pdf’ on AC(...) has passed verification.
The second one is for the whole bundle:
We'd like to let you know that the supporting Document accompanying the end-user Identity request labeled ‘Proof of Identity and Address in France’ on AC(...) has passed verification.
They both end with the following instructions:
Once the phone number type application has passed, you will be able to purchase phone numbers that come with a regulatory requirement. PLEASE NOTE:
If you provided a supporting document to validate a phone number you already own, please ensure you pair that phone number with the correct Phone Number type application in the Console, under the phone number regulations tab.
Learn more about this process here : https://www.twilio.com/docs/phone-numbers/regulatory/getting-started/console If you have any questions, please contact our support team.
Thanks, The Twilio Phone Number Regulatory Compliance Team
We will associate both this validated address and the verified bundle of supporting documents with the mobile numbers that we purchased.
in the list, click on the blue link with the phone number to access its details:
in the phone number details, select the tab.
The tab states whether the number has regulatory requirements:
This phone number (+336(...)) has regulatory requirements to comply with local regulations. You must comply with the local country regulations, or you risk possible disruption of service. Learn more
then allows to select an address and a bundle of documents to fulfill regulatory requirements.
click the button.
Apparently, nothing more happens. But according to Twilio reference documentation, it now takes up to 24-48 hours for the mapping of the documents to the phone number to be validated by Twilio. You should then receive a notification to inform you that the number is in compliance with national regulations.
Several days have passed and I received no such notification, which does not bother me much, since the number was already in use and operational. My guess is that you may only get that notification when the validation is performed during the purchase, and required to enable usage of the phone number.
The SIP user represents yourself, when you are making and receiving phone calls from your computer. Here we will simply call it me
but you could choose any other username based on your name, e.g. jane
or jane.doe
. Alternatively, if you plan on using this phone number from multiple computers, you should create a separate user with the name of each machine instead, e.g. pc2012
and macbook2015
.
We are now leaving the Phone Numbers section to work in the Programmable Voice section.
The password part is tricky because of all the requirements:
I used a password of size S (20 characters) from my own password generator, which usually contains a combination of upper case letters, lower case letters, special characters and digits.
Click the button to create the new SIP user.
Twilio now displays the details of the new SIP user as a Credential List called "me" with a list of Credentials that contains a Credential User Name called "me" as well:
This login can be used at the same time on different devices sharing the same phone number. However, I prefer to use different credentials on each device, which you can do in the following way:
Click the button,
fill the form with the Username and Password for each device:
Twilio allows you to customize the first part of this domain, the subdomain that appears before .sip.us1.twilio.com
.
While you can choose the subdomain freely, it has to be unique among all Twilio users. To ensure that it is the case, we will use here the phone number that you purchased from Twilio as a subdomain.
For illustration purpose, we'll consider that you purchased the (fake) number +1 (202) 555-0162 in the examples below. Since +
and ` (space) characters are not allowed in the SIP subdomain, we will write the corresponding subdomain as
1-202-555-0162` instead.
The SIP Domain is the bridge between your computer and regular phones through the VOIP provider; it appears in:
me@1-202-555-0162.sip.us1.twilio.com
+13035550123@1-202-555-0162.sip.us1.twilio.com
to call the US phone number +1 (303) 555-0123.Note: remember that we consider here that you bought the number +1 202 555 0162. You will need to replace it with your Twilio phone number.
Start by copying your Twilio phone number, found in the list of Active Numbers in the section.
Leave the Phone Numbers section and open the section instead.
In the Properties of the New SIP Domain, enter your Twilio phone number, formatted in your preferred way in the Friendly Name, and reformatted with only digits and dashes in the SIP URI subdomain: remove the initial +
and replace each ` (space) with a
-` (dash). For example:
I have recently noticed that the presence and position of dashes to separate digits in a phone number can influence the interpretation of numbers by Twilio during their conversion to international phone numbers. I have reported this issue to Twilio for their consideration. In the meantime, I would recommend to remove all dashes from the phone number value used as a custom identifier in the SIP domain. While most phone numbers will be interpreted correctly, and do not require a change, some will unexpectedly be transformed from e.g. a UK phone number to a US phone number, resulting in unexpected errors when trying to make calls from SIP to regular phone numbers, due to an invalid caller id. Removing all dashes, the first part of the SIP domain URI in the above example would read: 12025550162
.
This is the list of users allowed to make calls to regular phones from your virtual number.
You can leave the Call Control Configuration unchanged for now. We will come back here after we have created the script that describes how outgoing calls are handled, further down the line.
Under Secure Media, click the button to encrypt SIP registration and calls, deterring eavesdropping:
then select the same SIP user in the CREDENTIAL LISTS:
This is the list of users allowed to receive calls from regular phones to your virtual number.
Finally, click the button to create the new SIP domain.
The new SIP domain now appears in the list of Voice SIP Domains:
At this step, you can configure the VOIP software phone on your computer to connect to Twilio with your new SIP user.
In our example, the address of your SIP user is me@1-202-555-0162.sip.us1.twilio.com
. Following the steps above, you have created a similar address where 1-202-555-0162
is replaced with your Twilio phone number.
Note that there is an extra .us1
compared to the domain displayed in Twilio dashboard. This is required according to Twilio documentation to indicate that the server used for SIP registration is located in their US1 data center (North America Virginia). Additional servers may be added in different regions in the future.
I have described the configuration of two software phones, Zoiper and Linphone, in a separate issue.
I illustrate below the main steps of the setup in Linphone, applied to our example:
Open Linphone > Preferences > Calls and Chat.
Under Calls, click the button.
Note: Twilio will neither accept unencrypted calls on a domain where encryption has been enabled, nor encrypted calls on a domain where it has not been enabled. Once encryption has been enabled for a SIP domain, you have to use it, otherwise all your calls will get rejected.
Let's now configure a script to make calls to regular phone numbers.
In the settings of the SIP Domain, we can configure the HTTP URL that will be triggered when a call is made from your computer. That URL is expected to return further instructions telling Twilio how to handle the call. These instructions are written in an XML format called TwiML, which stands for Twilio Markup Language.
For example, the default handler for outgoing and incoming phone calls, https://demo.twilio.com/welcome/voice/
, returns the following TwiML:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Say voice="alice">
Thanks for the call.
Configure your number's voice U R L to change this message.
</Say>
<Pause length="1"/>
<Say voice="alice">
Let us know if we can help you in any way during your development.
</Say>
</Response>
<Response>
element wraps the other instructions,<Say>
reads the provided text using audio synthesis,<Pause length="1" />
waits for the given duration in seconds.To instruct Twilio to connect a call from our Twilio number +1 (202) 555-0162
to the number +1 (303) 555-0123
, we would use the <Dial>
element instead:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Dial callerId="+12025550162">+13035550123</Dial>
</Response>
You can host these XML instructions as a static file, or create them dynamically using a script in the language of your choice. Twilio provides helper libraries in different languages to help you to generate valid TwiML.
A script can take advantage of the parameters provided by Twilio. When you configure the callback URL, you can choose the HTTP method used to trigger it, either GET
or POST
. The parameters will be provided accordingly, either in the URL (GET
) or in the body (POST
).
You can find in Twilio documentation the list of parameters sent specifically in SIP requests as well as additional parameters sent in all requests.
Instead of always calling a fixed phone number, like in the previous example, we want to call the actual number that was requested in the software phone. That number is wrapped in a SIP URI of the form sip:+13035550123@[your-sip-domain].sip.us1.twilio.com
that is sent to the HTTP callback in the parameter To
.
We can write a script to extract this phone number from the SIP URI.
This is a simple script, and I don't want to host it on my own server. Twilio actually offers a number of options to host scripts on their servers instead. The simplest one is TwiML Bins. Initially limited to static TwiML files, they were later extended to support Mustache templates.
Using Mustache templates, we can access the value of the To
parameter as {{To}}
.
But Mustache templates support only very limited programming logic:
This is not enough to extract the phone number from the SIP URI. To this end, Twilio defined a custom Mustache filter called e164
. For example, {{#e164}}sip:+13035550123@[your-sip-domain].sip.us1.twilio.com{{/e164}}
gets replaced with +13035550123
.
Using this mechanism, we can turn our static TwiML example into a dynamic script:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Dial callerId="+12025550162">{{#e164}}{{To}}{{/e164}}</Dial>
</Response>
To avoid hardcoding our Twilio phone number in the callerId
attribute, we can take advantage of the fact that we used our Twilio phone number as subdomain. I checked that the e164
filter can also be applied to obtain the international phone number +12025550162
from the SIP Domain 12025550162.sip.us1.twilio.com
, which allows us to write:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Dial callerId="{{#e164}}{{SipDomain}}{{/e164}}">{{#e164}}{{To}}{{/e164}}</Dial>
</Response>
The script to answer phone calls is now ready.
We will save it in a TwiML Bin on Twilio, then set it as callback URL for the SIP Domain.
Note that the behavior described by this script is limited to calling regular phones. If you want to keep the possibility to call other SIP phones, you will need to apply a different logic. You may consider that a phone number will be called when the username in To
starts with a +
and otherwise a SIP call is made to the whole SIP address instead. This kind of logic is too advanced for TwiML templates. You can consider using Twilio Functions instead. Also hosted on Twilio servers, they offer support for a number of programming languages.
Leave the Programmable Voice section and open the section instead.
Fill in the form to create the new TwiML Bin:
We will now return to the Programmable Voice section to select this TwiML Bin in the settings of the SIP Domain.
Click the link to access the details of the SIP Domain for your Twilio phone number:
Click the button to update the SIP Domain settings.
You can now call a regular phone number from your software phone. For example, in Linphone:
In case something turns wrong, you can find details to investigate what happened in the tab found next to Develop in the sidebar.
Navigate to the Error logs then click on any of the error messages in the list to see how the request was interpreted, and what was treated as invalid.
The time of the error is displayed in UTC; you need to take into account your time zone difference to find out when the error occured in your local time.
We now need to prepare a new TwiML script with instructions that indicate how we wish Twilio to handle an incoming call. We will create a TwiML Bin to this end, and we will set the URL of the TwiML Bin as handler for incoming calls in the settings of our Twilio phone number.
We will start with the following script, which forwards the incoming call to our SIP endpoint:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Dial>
<Sip>sip:me@1-202-555-0162.sip.us1.twilio.com;transport=tls;secure=true</Sip>
</Dial>
</Response>
Remember that you need to replace 1-202-555-0162
with the SIP subdomain that you created for your Twilio phone number. Here we consider that you purchased the number +1 (202) 555-0162 and that you created the SIP subdomain 1-202-555-0162
.
The first steps are very similar to the creation of the TwiML Bin to handle outgoing phone calls:
We will now go to the Phone Numbers section to configure this TwiML Bin as handler for incoming calls in the settings of our Twilio phone number.
In the list of Active Numbers, click on the blue link with the phone number to access its details:
Under Voice & Fax, next to the field A CALL COMES IN, select TwiML in the first list. A second list appears, which reads "Select a TwiML Bin". In the second list, select the TwiML Bin that you just created: Receiving Calls from Regular Phones to SIP.
Under Messaging, next to the filed A MESSAGE COMES IN, delete the default URL, https://demo.twilio.com/welcome/sms/reply/
, which answers all texts with a canned response:
Thanks for the message. Configure your number's SMS URL to change this message. Reply HELP for help.Reply STOP to unsubscribe.Msg&Data rates may apply.
Click the button to update the settings of the Twilio phone number.
You can now answer calls from regular phones on your computer software phone.
You can try it by calling your Twilio number from a regular phone, with your software phone, e.g. Linphone, open on your computer. After a few seconds, you should get a notification for the incoming call, that you can answer to share some important insights with yourself:
You can now make and receive calls using your new Twilio phone number.
I will now describe how to configure a voicemail in a separate issue.
You can leave your comments below.
Does it work with VOIP software phone on mobile system, like Linphone?
I am using Linphone on macOS, but I am positive that it should work just the same using VOIP software on a phone or tablet.
Thank you so much man! I had to get a number urgently for work, didn't had any time for creating new app. mate do you have similar giude for sms send/receive with linphone?
Hi @SuneelGiri! Glad to help! Sending and receiving SMS is sadly not straightforward. You can find a description of the workflows that I have configured to send and receive SMS messages by email using Pipedream instead.
Beware that receiving SMS messages coming from short code numbers through a Twilio number has severe limitations and needs to be requested explicitly to be activated by Twilio.
@eric-brechemier no worries, thank you!
@eric-brechemier mate have you tried linphone android? I tried the same config for the android, outgoing call works perfectly, incoming has some problems, like the app doesn't give me option to accept the call, it automatically accepts, then cut the call itself after some seconds.
@SuneelGiri I haven't tried Linphone Android but what you describe reminds me of a similar issue reported in March of this year by another user. They got in touch directly with the Linphone developers to report the issue. It might be worth checking the status with them.
@eric-brechemier no worries mate. I fixed it.
@SuneelGiri Cool. Anything you'd like to share?
@eric-brechemier At first I added the sip account then changed the settings. Later uninstalled the app. Installed it again, change the settings then added the sip account and it got fixed.
Hi @eric-brechemier, Can we setup multiple numbers to a single linphone app? Also multiple number sms through pipedream?
@SuneelGiri Yes, I am using this setup with three different numbers corresponding to three different countries. I have a separate project for each number in Twilio and separate input/output pipelines for each number in Pipedream.
In Linphone, I can receive phone calls from all three numbers and select which number I want to use to place calls using the drop-down list.
@eric-brechemier , thats awesome. Thanks mate!
Excellent work. Thank you. Still usable even though the interfaces have changed.
To instruct Twilio to connect a call from our Twilio number +1 (202) 555-0162 to the number +1 (303) 555-0123, we would use the
element instead:
What's the +1 (303) 555-0123
for? Is it a real, physical phone to which the calls made to your Twilio virtual phone +1 (202) 555-0162
will be redirected?
Do you mean that it isn't possible to receive calls directly to my Zoiper and Linphone app via Twilio SIP via a virtual number only? Meaning, without +1 (303) 555-0123
@eric-brechemier
In https://github.com/eric-brechemier/how-i-replaced-skype-with-twilio/issues/5#issuecomment-489185155
you haven't assigned any particular number. Nor is there any reference to a phone number in the "SIP Domain" either. There're only variables.
When you make a call from Linphone to an regular phone, how does Twilio know which of the virtual numbers, of which may have multiple, to use?
To instruct Twilio to connect a call from our Twilio number +1 (202) 555-0162 to the number +1 (303) 555-0123, we would use the element instead:
What's the +1 (303) 555-0123 for? Is it a real, physical phone to which the calls made to your Twilio virtual phone +1 (202) 555-0162 will be redirected?
Do you mean that it isn't possible to receive calls directly to my Zoiper and Linphone app via Twilio SIP via a virtual number only? Meaning, without +1 (303) 555-0123 @eric-brechemier
@ally9335 This section describes how to make calls from your Zoiper/Linphone app to a regular phone number.
In that example, +1 (202) 555-0162
stands for the Twilio Phone number that you purchased and +1 (303) 555-0123
stands for a regular phone number that you would use for testing. Using a fixed, static value in this way is just an intermediate step that you may try to check that everything is working so far. The more complex case for the general configuration of calls from your Zoiper/Linphone app to regular phone numbers is described afterwards:
(...) Using this mechanism, we can turn our static TwiML example into a dynamic script:
<?xml version="1.0" encoding="UTF-8"?> <Response> <Dial callerId="+12025550162">{{#e164}}{{To}}{{/e164}}</Dial> </Response>
(...)
The following section describes how to receive calls from a regular phone to your Zoiper/Linphone app.
In https://github.com/eric-brechemier/how-i-replaced-skype-with-twilio/issues/5#issuecomment-489185155
you haven't assigned any particular number. Nor is there any reference to a phone number in the "SIP Domain" either. There're only variables.
When you make a call from Linphone to an regular phone, how does Twilio know which of the virtual numbers, of which may have multiple, to use?
@ally9335 you can hardcode the phone number in the callerId
attribute. But to avoid that and use the same script with all my virtual phone numbers, I chose to use a particular convention for the SIP domain name, of the form 12025550162.sip.us1.twilio.com
, which allows me to extract the corresponding phone number from the SIP domain used to connect to Twilio SIP for that call:
To avoid hardcoding our Twilio phone number in the
callerId
attribute, we can take advantage of the fact that we used our Twilio phone number as subdomain. I checked that thee164
filter can also be applied to obtain the international phone number+12025550162
from the SIP Domain12025550162.sip.us1.twilio.com
, which allows us to write:<?xml version="1.0" encoding="UTF-8"?> <Response> <Dial callerId="{{#e164}}{{SipDomain}}{{/e164}}">{{#e164}}{{To}}{{/e164}}</Dial> </Response>
In the Zoiper/Linphone app, a single SIP user / virtual phone number is active at any given time. This is the virtual phone number with which calls will be made. All other virtual phone numbers are still able to receive calls, however, not just the active one.
I already had a Twilio account when I started the setup. Here I will create a completely new account to check all the steps needed from Zero to Twilio. Starting from an email address, a regular phone number and a credit card, we will end up with a new virtual phone number that can make and receive calls from a computer.