This article was originally published on the Twilio Blog.
In this article, you will learn how to develop a web API with .NET 6 to handle Twilio webhooks and deploy it to AWS Lambda. You will also learn how to save call recordings to AWS S3 as MP3 files.
Prerequisites
What are webhooks?
In today’s API-driven world, integrating applications is easier than ever. Most of the time, you can get the information you need from an external system’s API, but sometimes you want to be notified by the external system when something happens. That’s where webhooks come in. You register your own endpoint with the external system, and they post data to your endpoint when the event you’re looking for occurs.
Twilio Webhooks
The type of data you can expect from webhooks depends on the Twilio service. For the Twilio Voice API, there are several types of webhooks, three of which you’ll use in this tutorial:
Incoming voice call webhook, as the name implies, is where you handle the incoming calls. When you use a programmable voice service such as Twilio, this is the core functionality you would want to implement. If you don’t handle the incoming call, you hear three beeps when you call your Twilio number, and the call is terminated. The call doesn’t even appear in the logs. As you will see later in this article, when you implement a call handler, you can provide instructions to Twilio to record the call, play audio, and more. Some of these actions also have their own follow up webhooks. These instructions are implemented in TwiML (the Twilio Markup Language). TwiML is an XML-based markup language that has elements such as Say (read text to the caller), Dial (add another party to the call) and Record (record the caller’s voice). You will use Say and Record in your project later.
After a call has been completed (inbound or outbound), Twilio sends an HTTP request to your endpoint. This is called the status callback.
You could receive the recording status callback if you requested to record the call. Then, Twilio sends your endpoint a message with the recording status and a URL to access the recording file. You have to specify your webhook URL to handle this callback message.
!!!warning
By default, Recording URLs don’t require authentication, and recordings are not encrypted. However, you can require basic authentication to access the recordings and configure recordings to be encrypted in the voice settings (Voice → Settings → General).
!!!
In this tutorial you will interact with the Twilio Voice product, but many other products also use webhooks and you can apply the same technique for them as you will for Voice.
Now that you’ve learned about these three webhooks, let’s move on to the next section.
Set up AWS IAM User
You will need credentials to deploy your application to AWS from the command line. To create the credentials, follow the steps below:
First, go to the AWS IAM Users Dashboard and click the Add users button.
Enter the user name such as twilio-webhook-user and tick the Access key - Programmatic access checkbox:
Click the Next: Permissions button at the bottom right.
Then, select Attach existing policies directly and select AdministratorAccess:
Click the Next: Tags button at the bottom right. Tags are optional (and quite valuable information), and it’s a good practice to add descriptive tags to the resources you create. Since this is a demo project, you can skip this step and click the Next: Review button at the bottom.
Confirm your selection on the review page. It should look like this:
Then, click the Create user button.
In the final step of the user creation process, you should see your credentials for the first and the last time.
!!!warning
Take note of your Access key ID and Secret access key before you press the close button.
!!!
Now, open a terminal window and run the following command:
You should see a prompt for AWS Access Key ID. Copy and paste your access key ID and press enter.
Then, copy and paste your secret access key and press enter.
When prompted, type us-east-1 as the default region name and press enter.
!!!info
In this example, I will use the us-east-1 region. Regions are geographical locations where AWS have their data centers. It is a good practice to deploy as close to your customers as possible for production deployments to reduce latency. Since this is a demo project, you can use us-east-1 for convenience as it’s the default region in AWS Management Console. You can find more on AWS regions in this document: Regions and Availability Zones.
!!!
As the default output format, type json and press enter.
To confirm you have configured your AWS profile correctly, run the following command:
The output should look like this:
Now that you have set up your AWS credentials, you can move on to setting up the code.
Create an ASP.NET Core project for AWS Lambda
You can download the finished project from GitHub. However, this article will provide step-by-step instructions to set it up yourself.
Open a terminal and navigate to the directory that will be the root of your project.
You will use the Lambda ASP.NET Core Web API project template in the sample project. So, first, install Lambda templates by running the following command:
dotnet new -i Amazon.Lambda.Templates
You should see the results of a successful installation:
Take note of the short name of Lambda ASP.NET Core Web API: serverless.AspNetCoreWebAPI.
Then, run the following command to create the project:
dotnet new serverless.AspNetCoreWebAPI --name TwilioWebhookLambda.WebApi --output .
The command above will create a new project with the following file structure:
Note that the template creates a folder named src and puts the project in that folder. You can move the code to your root folder, but the rest of the article will use the default paths.
You will leverage a new AWS Lambda feature called Function URLs to make the function publicly available. For this to work with your API you need to install the Amazon.Lambda.AspNetCoreServer.Hosting NuGet package. In the terminal window, navigate to the project folder and run:
cd src/TwilioWebhookLambda.WebApi
dotnet add package Amazon.Lambda.AspNetCoreServer.Hosting
Then, open Startup.cs in your IDE, update the ConfigureServices method so that it looks like this:
```csharp hl_lines=”4”
public void ConfigureServices(IServiceCollection services)
{
services.AddControllers();
services.AddAWSLambdaHosting(LambdaEventSource.HttpApi);
}
Lambda Function URLs use HttpApi behind the scenes, so you need to use `LambdaEventSource.HttpApi` as the event source type.
You will need the Amazon Lambda Tools .NET tool to deploy the function via the command line. You can install it by running the command below:
```bash
dotnet tool install -g Amazon.Lambda.Tools
Amazon Lambda Tools, use the aws-lambda-tools-defaults.json file to get some details about the installation. Unfortunately, it doesn’t come with all the values it needs. For example, you can store the runtime and the function’s name in this file, so you don’t have to keep entering it whenever you deploy it from scratch.
Open the file and update it so that it looks like this:
{
"profile": "",
"region": "",
"configuration": "Release",
"function-runtime": "dotnet6",
"function-memory-size": 256,
"function-timeout": 30,
"function-handler": "TwilioWebhookLambda.WebApi",
"function-name": "TwilioWebhookLambda-WebApi",
"function-url-enable": true
}
If you don’t provide profile and region values, it uses the default profile and region in your AWS configuration. If you want to override the defaults, update those values as well.
Then, deploy the Lambda function by running the following command:
dotnet lambda deploy-function
Your Lambda function needs an IAM role to execute. The policies attached to this role determine the permissions of the function. By default, the Amazon Lambda Tools create a role on your behalf and attach it to the function.
During the deployment, it lists the existing roles along with an option to create a new role:
Select the Create new IAM Role option.
Give it a descriptive name, such as TwilioWebhookLambda-WebApi-Role, so that you can easily determine its purpose when you see it in your IAM dashboard.
The next step is to select the IAM policy. Your project will need Amazon S3 access to store call recordings. Also, having access to CloudWatch logs is always helpful. So choose 3 - AWSLambdaExecute from the list:
!!!info
As a best practice, you should develop custom policies to grant only the minimum required permissions.
!!!
After the deployment has finished, you should see the successful deployment message:
The publicly available URL shown above is only created because you enabled the Function URL feature in the aws-lambda-tools-defaults.json file.
"function-url-enable": true
Without this feature, you wouldn’t be able to use a Lambda function as a webhook handler.
Now open that URL in a browser and you should see the default GET / endpoint result:
The API works like any other API. This template comes with a sample controller called ValuesController. Test the controller by appending /api/values to your function URL:
You should see an array of strings (value1 and value2) displayed on your browser.
You just deployed your ASP.NET Core web API to Lambda and made it publicly available. Great job!
Receive Incoming Calls
The Twilio .NET SDK and the helper library for ASP.NET make it easier to build Twilio applications. In this tutorial, you’ll use the SDK to generate TwiML and the helper library to respond to webhook requests. Add the SDK and helper library via NuGet:
dotnet add package Twilio
dotnet add package Twilio.AspNet.Core
Under the Controllers folder, add a new file called IncomingCallController.cs and replace its contents with the following code:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Core;
using Twilio.TwiML;
namespace TwilioWebhookLambda.WebApi.Controllers;
[ApiController]
[Route("api/[controller]")]
public class IncomingCallController : TwilioController
{
[HttpPost]
public TwiMLResult Index()
{
var response = new VoiceResponse();
response.Say("Hello. Please leave a message after the beep.");
return TwiML(response);
}
}
In the terminal, deploy the updated function:
dotnet lambda deploy-function
You should get a successful update message on your screen:
At this point, you have a publicly available endpoint but Twilio is not aware of it yet.
Go to the Twilio console. Select your account, and then click Phone Numbers → Manage → Active Numbers on the left pane. (If Phone Numbers isn’t on the left pane, click on Explore Products and then on Phone Numbers.)
!!!warning
You don’t permanently own Twilio numbers; instead, you lease them until you release them. If you release a number after a 10-day grace period, it is returned to the number pool.
!!!
Click on the phone number you want to use for your project and scroll down to the Voice section.
Under the “A Call Comes In” label, set the dropdown to Webhook, the text field next to it to your Lambda Function URL suffixed with the /IncomingCall path, the next dropdown to HTTP POST, and click Save. It should look like this:
To test, call your Twilio number, and you should hear the message “Hello. Please leave a message after the beep.”. It doesn’t actually wait for the message, but at least you know you have implemented an incoming voice webhook. Your code is executed when your Twilio number receives a call.
In the next section, you will handle the second webhook type: Call Status Updates.
Receive Call Status Updates
Create a new file in the Controllers folder called CallStatusChangeController.cs with the code below:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Core;
namespace TwilioWebhookLambda.WebApi.Controllers;
[ApiController]
[Route("api/[controller]")]
public class CallStatusChangeController : TwilioController
{
private readonly ILogger<CallStatusChangeController> _logger;
public CallStatusChangeController(ILogger<CallStatusChangeController> logger)
{
_logger = logger;
}
[HttpPost]
public async Task Index()
{
var form = await Request.ReadFormAsync();
var to = form["To"];
var callStatus = form["CallStatus"];
var fromCountry = form["FromCountry"];
var duration = form["Duration"];
_logger.LogInformation(
"Message to {to} changed to {callStatus}. (from country: {fromCountry}, duration: {duration})",
to, callStatus, fromCountry, duration);
}
}
This code logs some of the values posted to your webhook.
Go back to the active number configuration in the Twilio Console and update the “Call Status Changes” field with your Lambda Function URL suffixed with /CallStatusChange, as shown below:
Save your configuration, and then deploy your project again using dotnet lambda deploy-function.
Now call your Twilio number again, and after the call has been completed, you should see the callback logs in CloudWatch:
You received this message when the call status changed to “completed”.
You can also use the Twilio console to view all call logs: Click Monitor → Calls on the left pane.
Locate the call in the list and click the Call SID link to view the details.
In the Request Inspector section, you can see all the callbacks with their requests and responses in detail:
Next, you will look into the third and final type of voice webhook: Recording Status Updates.
Receive Recording Status Updates
!!!warning
Before you record anything, please make sure to read this article: Legal Considerations with Recording Voice and Video Communications.
!!!
Create a new controller called RecordingStatusChangeController and replace its contents with the code below:
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Core;
namespace TwilioWebhookLambda.WebApi.Controllers;
[ApiController]
[Route("api/[controller]")]
public class RecordingStatusChangeController : TwilioController
{
private readonly ILogger<RecordingStatusChangeController> _logger;
public RecordingStatusChangeController(ILogger<RecordingStatusChangeController> logger)
{
_logger = logger;
}
[HttpPost]
public async Task Index()
{
var form = await Request.ReadFormAsync();
var callSid = form["CallSid"];
var recordingStatus = form["RecordingStatus"];
var recordingUrl = form["RecordingUrl"];
_logger.LogInformation(
"Recording status changed to {recordingStatus} for call {callSid}. Recording is available at {recordingUrl}"
,recordingStatus, callSid, recordingUrl);
}
}
Similar to the status change handler, this code only logs some request details. Once you’ve seen all webhooks are working fine, you will update the implementation with more meaningful code.
You also need to modify the IncomingCallController and replace the code in the Index method as below:
var response = new VoiceResponse();
response.Say("Hello. Please leave a message after the beep.");
response.Record(
timeout: 10,
recordingStatusCallback: new Uri("/api/RecordingStatusChange", UriKind.Relative)
);
return TwiML(response);
Now you’re telling Twilio that you’d like to record the phone call. You are also specifying the webhook URL that will receive the recording status update. Unlike the other webhook types, there is no field in the Twilio console to set the recording status callback.
Deploy this update and call your number again. This time you should be able to leave a message after the beep. Once you’ve done that, check your CloudWatch logs, and you should see two status updates: One for the call status and one for the recording status:
As you can see in the logs, the recording URLs are public by default, but the recordings have long random names, so they cannot be iterated through and downloaded by unauthorized parties. To increase the security of the recordings, you can enable Enforce HTTP Auth on Media URLs and Voice Recording Encryption options in Voice Settings in your account.
Save Recording MP3 files to an Amazon S3 Bucket
Now let’s see how you can retrieve the recording file and upload it to an Amazon S3 bucket from your ASP.NET Core project.
!!!info
As of May 2022, Twilio has a built-in feature to store recordings in an Amazon S3 bucket. In this article, however, you will use a different approach and upload the MP3 files programmatically from your Lambda function.
!!!
First, you will need an S3 bucket to store the files. To create the bucket, go to AWS Management Console and search for S3:
Then, click the link to go to the S3 service dashboard.
Click Create Bucket button:
Give it a descriptive and globally unique name, accept all the defaults and click the Create Bucket button at the bottom of the screen.
You should see your bucket in the bucket list:
!!!info
Amazon S3 bucket names are global. If somebody else created a bucket named my-twilio-call-recordings, you can not also use that name. You can find more bucket naming rules in AWS Documentation.
!!!
In your application, you need to install the AWS SDK packages to talk to the Amazon S3 API.
In the terminal, run the following command:
dotnet add package AWSSDK.S3
Update the RecordingStatusChangeController code as below:
using Amazon.S3;
using Amazon.S3.Transfer;
using Microsoft.AspNetCore.Mvc;
using Twilio.AspNet.Core;
namespace TwilioWebhookLambda.WebApi.Controllers;
[ApiController]
[Route("api/[controller]")]
public class RecordingStatusChangeController : TwilioController
{
[HttpPost]
public async Task Index()
{
string recordingUrl = Request.Form["RecordingUrl"];
string fileName = $"{recordingUrl.Substring(recordingUrl.LastIndexOf("/") + 1)}.mp3";
string bucketName = "my-twilio-call-recordings";
using HttpClient client = new HttpClient(); // use HttpClient factory in production
using HttpResponseMessage response = await client.GetAsync($"{recordingUrl}.mp3");
using Stream recordingFileStream = await response.Content.ReadAsStreamAsync();
using var s3Client = new AmazonS3Client();
using var transferUtility = new TransferUtility(s3Client);
await transferUtility.UploadAsync(recordingFileStream, bucketName, fileName);
}
}
Make sure to set the bucketName with your bucket’s name.
In this code, when you receive the recording URL, you extract the recording file name, retrieve the file content via a stream and pass the stream to S3 TransferUtility which uploads it to the Amazon S3 bucket as {fileName}.mp3.
!!!info
By default, the recording URL doesn’t have a file extension. If you call the URL as is, Twilio returns the WAV version of the recording. To get the MP3 version, you need to append .mp3 to the URL as shown in client.GetAsync call.
!!!
During the setup process, you didn’t explicitly tell AWS that your Lambda function should have access to your S3 bucket. So you might be wondering how you have permission to do that. The reason is you chose AWSLambdaExecute policy to be attached to your function’s role. So if you go to the IAM dashboard and search AWSLambdaExecute, you should see the policy’s permissions are defined like this:
You can see that this policy has permission to put objects into all S3 buckets. Since this is a demo project, I decided to keep things simple. However, in production, I’d recommend writing your own policy and giving the minimum required permissions, such as using the names of the resources instead of using wildcards. You can read more on that here: IAM Best Practices: Apply least-privilege permissions
Deploy the final version of the API and call your number again.
A short while after you’ve completed the call, you should see the recording in your bucket:
!!!warning
As the focus of this article is using AWS Lambda to respond to Twilio webhooks, securing your endpoint is not covered in this article. To learn more about Webhooks Security, you can read this article: Webhooks Security.
!!!
Conclusion
Congratulations! You covered three types of webhooks for the Twilio Voice service and implemented handlers for all of them. In addition, you managed to download recordings to your own storage. Later on, you can download or move the files to cold storage using Amazon S3 Glacier. The possibilities are endless when you can manage all of this programmatically. For example, you could use Amazon Transcribe to transcribe the call recording to text, or you could use Twilio’s transcribe attribute on the record-verb.
If you didn’t follow along and implement the project, don’t worry. You can always download the final project from my GitHub repository and experiment on your own.
If you enjoyed playing with call recordings and webhooks using Twilio API, I’d recommend you take a look at these articles as well:
