#4 Creating Secrets for My Lambda End Point

Print Friendly, PDF & Email

#4 Creating Secrets for My Lambda End Point

Last time we create the core of our Rest API Lambda, we now want to connect to our RDS MySQL database. However we don’t want to embed the connection details, passwords, port information inside our code for security purposes. To get around this problem AWS has a service call Secrets Management, and that’s what were going to cover in this Blog.

AWS Secrets Manager Create New Secret

To store a new AWS Secret, login to the AWS Services Console and navigate to the Secrets Manager Dashboard, then select Store a New Secret.

AWS Secrets Manager Dashboard

The first step is to select the Secret Type, since we are going to connect to our RDS MySQL Database we will select ‘Credentials for RDS Database’ for the secret type. Within the RDS MySQL Database we will create a new user and give it a password, this will be the same credentials stored within the secret so for our DataModel we will create a User called DataMode_User and give it a password. Write this information down for later use.

Secrets Manager – Secret Type

Once you ahve defined the User Name and Password (this well use when we create the Database User), we will use the Default Encryption and then select the Database Instance we want to connect to, then press Next.

Secrets Manager – Database Instance Selection

Give the Secret a Name, this will be the Name that we will use within our C# code to ask the Secret Manager for the Database Connection information.

Secret Manager – Secret Name

There are some optional settings but we wont need to change anything else for this example, select Next to continue.

Secrets Manager – Optional Settings

This example wont cover secret rotation, so select Next to continue.

Secrets Manager – Configure Rotation

The last page is the Review and Store Secret page, at the bottom is some sample code showing you how to retrieve your secret, we are going to use Secret Caching which is simpler than this example, select Store to create the Secret.

Secrets Manager – Review and Store

We can now see we have a stored secret.

Stored Secret

VPC End Point

To allow our code to read the secret we have to create an VPC Endpoint for the Secrets Manager. To do this navigate to the VPC dashboard and then select Endpoints, from the left-hand menu bar.

VPC Dashboard

Within the Endpoint Dashboard select the Create Endpoint button.

Create Endpoint

For the Endpoint definition select AWS Service, then type in ‘Secret’ into the search bar and press enter. This will filter the AWS Services list down to only services that contain the word Secret. Select the Secrets Manager service.

Endpoint – Select AWS Service

So we don’t have to mess with route tables, we will create the end point in the same VPC and use the same subnets as the RDS MySQL Database.

Endpoint – VPC’s and Subnets

Scroll down to the bottom of the window and select Create Endpoint to complete the creation.

Endpoint – Create Endpoint

MySQL RDS Database User

To keep our data access secure we will create a new user in the database that has specific access, to do this within your Database Developer (I’m using Heidi SQL) go to the Manage Users and Privilege’s.

Manage Users and Authentication

using the User Name and Password we created for the AWS Secret, create a new User and give that user privilege’s to the DataModel_Table specifically Select, Delete, Insert, and Update. Then select Save.

Adding New Database User

Secrets Manager Code

Were going to change our project so we can retrieve and display the Secret Information we saved in the Secrets Manager, then we will create a connection string to connect to the database.

Serverless Template

To publish our Lambda to the same VPC and Sub Nets as the RDS MySQL Database and the Secrets Manager End Point we need to add this information to the Serverless Template. Between the Handler and the Runtime sections add the following VPC, Security Group ID and Sub Net ID information. You can go back to the Endpoint configuration in the AWS VPC Endpoint Dashboard to retrieve the information that pertains to you.

 "Handler": "MyFirstLambdaProject::MyFirstLambdaProject.LambdaEntryPoint::FunctionHandlerAsync",
        "VpcConfig":{
            "SecurityGroupIds":["sg-xxxxxxxx"],
            "SubnetIds":["subnet-aaaaaaaa","subnet-bbbbbbbb","subnet-cccccccc"]
        },
        "Runtime": "dotnetcore3.1"

Secrets Manager Class

Lets write some code that will allow us to retrieve the secret, the first thing will be to add a Project Folder called ‘Secregts’ and a class called ‘SecrtesManager’, to our Visual Studio Project.

Secrets Manager Class

We need to add a NuGet package called ‘AWSSDK.SecretsManager.Caching’ to our project to allow us to write the Secrets manager code.

Secrets Manager Caching NuGet Package

We also need to add a NuGet package called ‘Newtonsoft.Json’ to our project to allow us to Deserialize, the Secrets Manager Json string.

Newtonsoft Json NuGet Package

Within the ‘SecretsManager’ class we will add the following code.

using Amazon.SecretsManager.Extensions.Caching;
using Newtonsoft.Json;

using System.Collections.Generic;
using System.Threading.Tasks;

namespace MyFirstLambdaProject.Secrets
{
    public class SecretsManager
    {
        private static SecretsManagerCache cache = new SecretsManagerCache();

        private static async Task<string> GetSecret(string name)
        {
            return await cache.GetSecretString(name);
        }

        public static Dictionary<string, string> GetSecretFromSecretManager(string iSecretName)
        {
            string secret = GetSecret(iSecretName).Result;
            return JsonConvert.DeserializeObject<Dictionary<string, string>>(secret);
        }
    }
}

Values Controller

So to prove that we can get the secret we will temporarily modify the Values Controller ‘Get’ method, to call the ‘GetSecretFromSecretManager’ method. We will then republish the project to Lambda and use Postman to call the API end point, to print our secrets.

So lets change the Get method as shown below, we need to pass into the ‘GetSecretFromSecretManager’ method the name of the Secret which was ‘DataModel’, you may have named it differently. If you don’t remember go back to the Secrets Manager Dashboard in the AWS Console and double check.

        [HttpGet("/v1/DataModel")]
        public IActionResult Get()
        {
            try
            {
                //return Ok(this._IDataModelProcessor.GetDataModel());

                return Ok(SecretsManager.GetSecretFromSecretManager("DataModel"));
            }
            catch (Exception ex)
            {
                return BadRequest(ex.Message);
            }
        }

Adding the Secrets Manager Policy to The Lambda

Navigate to the Lambda Dashboard inside the AWS console, and select the Lambda function.

Lambda Function

Scroll down and select the Configuration tab.

Lambda Configuration

Select the link to the Execution Role.

Execution Role

To allow the Lambda function the capability to read the Secret we must add the policy ‘SecretsManagerReadWrite‘.

Secrets Manager Read Write Policy

To do this select the Attach Policies button, then key into the text filter ‘secret’ and press enter. Then check the policy to add and select Attach Policy. With this policy attached the lambda function will be able to read our secret.

Attach Policy

Testing With Postman

The first thing we must do is republish the project to Lambda, to do this right mouse click on the project and select Publish to AWS Lambda from the contextual menu.

Publish to AWS Lambda

Once the publish is complete, copy the AWS Serverless END point URL.

AWS Serverless URL

Open postman and paste the AWS Serverless END point URL into a new Tab and set the Request Type to Get. Then add the endpoint URL and select the Send button.

You should get a response which will show you the secrets stored in the secrets manager.

Obviously this is a security breach, so I would recommend that you revert the Get method back to the original code and republish to Lambda, however we are going to secure our end point using an API key. Later on we will revert the code back.

Get Secrets in Postman

Adding API Key’s

If anybody was able to get your AWS Serverless URL, and End Point URL they could run thousands of requests and create a sizable bill. So let’s make sure that does not happen.

So lets first login to the AWS Console and Navigate to the API Gateway dashboard, then select the API Endpoint for our project.

API Gateway

So we can now see the individual end points, that were created by the Serverless Template. We now need to create a usage plan, API Key and apply it to each of the end points. To do this select the Usage Plans link in the left-hand menu bar.

Usage Plans

Then select the Create button.

Create Usage Plan

Since this is for testing and learning purposes, lets keep the Throttling and Quota numbers low. For a real production deployment you will want to think about these values, since if you exceed Quota then the Lambda will become inaccessible. Then select Next.

Throttling and Quota

Next we will add an API Stage, to do this select Add API Stage, choose the API then the Stage. Then Select the Checkmark to apply the ‘Method Throttling’ and select Next.

Associate API Stages

In the next section select Create API Key and add to Usage Plan, you can always use an existing API also.

Create API Key and Add to Usage Plan

In the popup window, key in a ‘Name’ for the API key and select Save.

Create API Key

We now have a competed Usage Plan, select Done to finish the definition.

Completed Usage Plan

Next for Each API End Point we have to enable the usage of the API key. To do this return back to the API Gateway dashboard, then select the API Endpoint for our project. Then select one of the end points and select the Method Request link.

Delete Method Request

In the method Request Settings, change the API Key Required value from False to True and select the Checkmark to approve the change.

Repeat this for all the API end points.

Method Request Settings

Next we must Deploy the API Endpoint. To do this select one of the API endpoint’s and from the Actions dropdown menu select Deploy API

Deploy API

In the popup window select the Deployment Stage and then select Done.

Repeat this for all the API end points.

Deploy API

Now we need to know what the API key is so we can test if the end point is protected or not. To get the API key, select API keys from the left-hand side bar menu, then select the new API Key and in the information panel select Show API Key. Now you can copy the API key.

API Key

Re-Testing With Postman

Let’s go back to Postman and rerun the Get Request, we should now get a Forbidden message in the response. This is because we must send the API Key in the request header.

Forbidden

To add the API key in the request header, select on the Authorizations tab and change the Type to ‘API Key’. The Key parameter must be named ‘x-api-key’ for the value paste in the copied API Key. We can re test by selecting Send, we should again get back the secret information again.

We are now secure.

Correct Return With API Key