Many cloud applications rely on backends or web apps to trigger external services. But the challenge has always been the reliability and performance concerns that abound when there is an overprovisioned service in high-traffic seasons.
Lambda has an event source mapping feature that allows you to process items from AWS services queues that don’t invoke Lambda services directly. It then becomes possible to set queues as event sources that get your lambda functions triggered by messages. These queues can help control your data processing rate to increase or reduce computing capacity per the data volume.
This article walks you through the steps of creating Lambda functions that connect with SQS queue events. We’ll begin with the basics of both Lambda and SQS and later get into the tutorial.
What is AWS Lambda?
The AWS Lambda service offered by AWS lets you run code and provision resources without server management requirements. The event-driven cloud computing services run code when triggered by events – you can use it for any computing task, from processing data streams or serving website pages.
AWS Lambda features include automated administration, auto-scaling, and fault tolerance.
What is AWS SQS?
Amazon SQS lets you integrate various software components and systems with scalability, security, and high availability benefits. The hosted solution on AWS eliminates the need to manage your service infrastructure manually. It excels in the in-transit storage of messages between apps and micro-services –it makes life easier regarding message-oriented AWS software systems.
Users can connect with SQS using an API provided by AWS. It can send requests to available queues by other services. It can receive messages that are pending in the queue. It also allows for the deletion of queue messages following proper processing.
Lambda + SQS
You can have a lambda function as part of the SQS queue. It would become a consumer of the queue using it as an event source. It then polls this queue-invoking function with events that contain queue messages.
Lambda will read the messages in batches and invoke functions for each batch. The functions invoked are within a specific payload threshold set by Lambda.
Lambda sets an invocation payload size quota that is also a threshold for invoking a function. It is 6MB for each response and request.
SQS + Lambda Tutorial
Step 1: Create an SQS Queue
Sign in to your AWS Management Console.
Navigate to the SQS service by searching for "SQS" in the search bar.
Click "Create Queue."
Choose "Standard Queue" for this tutorial and name your queue, such as "MySQSQueue."
Leave the rest of the settings as default and click "Create Queue."
Step 2: Create a Lambda Function
Navigate to the AWS Lambda service by searching for "Lambda" in the search bar.
Click "Create Function."
Choose "Author from scratch" and give your function a name, such as "MySQSLambdaFunction."
Choose a runtime, such as "Python 3.8."
Under "Function code," you can write your code inline or upload a .zip file containing your code. For this tutorial, let's use the following inline code to process messages from the SQS queue:
Under "Execution role," choose "Create a new role with basic Lambda permissions."
Click "Create Function."
Step 3: Grant Lambda Access to SQS
In the Lambda function's "Configuration" tab, click "Permissions."
Click on the role name under "Execution role."
Click "Attach policies."
Search for "AWSLambdaSQSQueueExecutionRole" and select it.
Click "Attach policy."
Step 4: Configure Lambda Trigger
Go back to your Lambda function's "Configuration" tab.
Click "Add Trigger."
Choose "SQS" from the trigger list.
In the "SQS Queue" field, select the SQS queue you created earlier (e.g., "MySQSQueue").
Set the "Batch size" to a value between 1 and 10. For this tutorial, set it to 5.
Click "Add."
Step 5: Test Your Setup
Go back to the SQS service in the AWS Management Console.
Select your queue (e.g., "MySQSQueue") and click "Send and Receive Messages."
Type a test message in the "Send a message" section and click "Send Message."
Your Lambda function should automatically trigger when messages are added to the queue. To verify this, navigate to the Lambda function's "Monitoring" tab and check the "Invocations" metric.
Now you have successfully set up an AWS SQS and Lambda integration. You can now send messages to your SQS queue, and your Lambda function will automatically process them. This highly scalable and cost-effective serverless architecture allows you to focus on your application's core functionality.