Implementing Custom Resource and CFN Response

As most of the enterprises use Cloudformation to provision resources and also in the CI/CD, pipeline, but what about the resources that are not supported by Cloudformation, and what  if you want to implement the logic that cloudformation does not support inbuilt. Here AWS Cloudformation custom resources can help you fulfill above objective. 

Before reading this blog if you want to know more about custom resources please visit https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-custom-resources.html. 

The actual implementation of  Custom Resource uses the CFN-response module which is available in the ZipFile property in the template or for code in buckets, you must write your own functions to send responses. cfn-response is a micro package which uses some helpers script to talk with CloudFormation during Custom Resource creation/updation/deletion for sending responses to CloudFormation and providing exceptions. 

If we are using AWS Lambda based custom resources then, we’ll simply be writing Lambda code that accepts events that look like request objects, do some logic to fulfill the promises defined by the property’s object, then returns a response object by putting the object into S3. Before we begin with the solution of the above problems lets understand what are the components important to know in the cfn-response with the help of this Lambda code snippet 

AWSTemplateFormatVersion: “2010-09-09”[Text Wrapping Break]Resources:[Text Wrapping Break]  CustResource:[Text Wrapping Break]    Type: “Custom::Summer”[Text Wrapping Break]    Properties:[Text Wrapping Break]      ServiceToken: “arn:aws:lambda:<REGION-CODE>:<ACCOUNT-NUMBER>:function:<FUNCTION-NAME>”[Text Wrapping Break]      Input: 2[Text Wrapping Break]Outputs:[Text Wrapping Break]  Sum:[Text Wrapping Break]    Value: !GetAtt CustResource.Data

ZipFile: |[Text Wrapping Break]  import json[Text Wrapping Break]  import cfnresponse[Text Wrapping Break]  def handler(event, context):[Text Wrapping Break]    responseValue = int(event[‘ResourceProperties’][‘Input’]) * 5[Text Wrapping Break]    responseData = {}[Text Wrapping Break]    responseData[‘Data’] = responseValue[Text Wrapping Break]    cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData, “CustomResourcePhysicalID”) 

Now, the CustResource.Data value is 10 => 2*5 

Here the most important fields  we need to consider is responseData. 

responseData: The data field of a custom resource response object. The data is a list of name-value pairs. This is the field where you get to respond to the CFT stack event with the value(s) after your custom logic in the lambda. 

Now I am going to talk more about applying custom logic with custom resource which will resolve the problems in your CI-CD pipeline when using Cloudformation : 

1. You can implement the custom resource if you need to get some value every time when you create/update the CFT  and you cannot pass that variable as parameter in the stack as it can change frequently and use that dynamic variable as input to another resource in the same or other CFN stack in the pipeline. 

In such Conditions you can implement the custom resource and trigger a lambda to get 

that value using the responseData[ ] and pass that available into another resource using !GetAtt <CUSTOM-RESOURCE-NAME>.<RESPONSE-OBJECT-ITEM>. In the above code snippet you can see the value is gathered and consumed by implementing the same logic. 

1. If you want the script to be triggered during the first-run( creation time)  only that modifies or adds value to the resources created in the same stack, and should not be triggered during every CFT update. 

 For this condition we need to understand that whenever we create, update, or delete a custom resource, AWS CloudFormation sends a request to the specified service token. And according to RequestType we can implement our logic, to achieve our objective we need to choose “Create” as RequestType. And the important thing in this implementation is now you will have to remove the section of the DynamoDB table used for creation in the cft template( as it will be created using custom resource). 

# updating DynamoDB table when RequestType is Create [Text Wrapping Break]if event[‘RequestType’] == ‘Create’ : 

with table.batch_writer() as batch: 

batch.put_item(Item= {‘name’:’new-name’,’date’:’2020-01- 01′}) 

cfnresponse.send(event, context, cfnresponse. SUCCESS, responseData)  

 else : 

cfnresponse.send(event, context, cfnresponse. SUCCESS, responseData) 

 

Here the dynamoDB table gets updates with new items only during the first run ( during creation of the CFT stack ). And during updation we are just sending success signals and not implementing any logic. And again such logic can be applied when you are dealing with CFT update ‘RequestType’.  

 

1. If you want to modify some of the resource’s properties and it has been created using CFT but the updation of those resources requires replacement of that resource ( as updation will destroy and re-create a new resource with the same physical name) and want to modify the resource anyhow. One of the example of this can be Dynamodb Renaming TableName or updating LocalSecondaryIndex in the DynamoDB will create a new Dynamodb Table, deleting the existing table will delete all the items on the fly with CFT. 

            

In such condition we need to create a custom resource in our CFT with the create RequestType and we can have the logic written in the lambda script to create a backup of the dynamodb table, delete the dynamodb table and restore the dynamodb table with the same Table name and finally the responseData can be the name of the new restored table, which the other resource can use as input. 

 

 AWSTemplateFormatVersion: “2010-09-09”[Text Wrapping Break]Resources:[Text Wrapping Break]  CustResource:[Text Wrapping Break]    Type: “Custom::BackupRestoreDynamoDB”[Text Wrapping Break]    Properties:[Text Wrapping Break]      ServiceToken: “arn:aws:lambda:<REGION-CODE>:<ACCOUNT-NUMBER>:function:<FUNCTION-NAME>”[Text Wrapping Break]      DynamodbName: “myDynamoDBTable”[Text Wrapping Break]Outputs:[Text Wrapping Break]  Sum:[Text Wrapping Break]    Value: !GetAtt BackupRestoreDynamoDB.TableName 

  

import json[Text Wrapping Break]import uuid[Text Wrapping Break]import boto3[Text Wrapping Break]from botocore.vendored import requests[Text Wrapping Break]import cfnresponse[Text Wrapping Break][Text Wrapping Break]client = boto3.client(‘dynamodb‘)[Text Wrapping Break][Text Wrapping Break][Text Wrapping Break]def backup_restore_dynamodb(event):[Text Wrapping Break]   [Text Wrapping Break]    TableName = event[‘ResourceProperties‘].get(‘DynamodbName‘)[Text Wrapping Break][Text Wrapping Break]    backup_response = client.create_backup([Text Wrapping Break]    TableName= TableName,[Text Wrapping Break]    BackupName= TableName)[Text Wrapping Break][Text Wrapping Break]    BackupARN = backup_response[‘BackupDetails‘].get(‘BackupArn‘)[Text Wrapping Break]   [Text Wrapping Break]    response = client.delete_table([Text Wrapping Break]    TableName= TableName )[Text Wrapping Break][Text Wrapping Break]    restore_response = client.restore_table_from_backup([Text Wrapping Break]    TargetTableName= TableName,[Text Wrapping Break]    BackupArn= BackupARN)[Text Wrapping Break][Text Wrapping Break]    newTableName = restore_response[‘TableDescription‘].get(‘TableName‘)[Text Wrapping Break][Text Wrapping Break]    return newTableName[Text Wrapping Break][Text Wrapping Break]# execution begins here[Text Wrapping Break]def lambda_handler(event, context):[Text Wrapping Break][Text Wrapping Break]    print(json.dumps(event))     # logging the event for debugging purpose[Text Wrapping Break][Text Wrapping Break]    try:[Text Wrapping Break]        if event[‘RequestType‘] == ‘Create’:[Text Wrapping Break]            # place the code that you would want to run when the Request type is Create[Text Wrapping Break]            newTableName = backup_restore_dynamodb(event)[Text Wrapping Break]            responseData = {}[Text Wrapping Break]            responseData[‘TableName‘] = newTableName[Text Wrapping Break]            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData) [Text Wrapping Break]     [Text Wrapping Break]        elif event[‘RequestType‘] == ‘Delete’:[Text Wrapping Break]            # place the code that you would want to run when the Request type is Delete[Text Wrapping Break]            send_response(event,‘SUCCESS‘)[Text Wrapping Break]        elif event[‘RequestType‘] == ‘Update’:[Text Wrapping Break]            # place the code that you would want to run when the Request type is Update[Text Wrapping Break]            newTableName = backup_restore_dynamodb(event)[Text Wrapping Break]            responseData = {}[Text Wrapping Break]            responseData[‘TableName‘] = newTableName[Text Wrapping Break]            cfnresponse.send(event, context, cfnresponse.SUCCESS, responseData) [Text Wrapping Break][Text Wrapping Break]    # If any errors in the above section of the code, catch the error and send the siganl to the CFN stack about the error.[Text Wrapping Break]    except Exception as e:[Text Wrapping Break]        # Failure occurred[Text Wrapping Break]        # Sending an exception based response[Text Wrapping Break]        cfnresponse.send(event, context, cfnresponse.FAILED)  
 

   

Along with this blog I would also like to share some of the lesson learned by me during my practices with custom resources 

• In correct use of cfn-response or missing to add cfn-response in the lambda script will cause Cloudformation stack to stuck in two conditions DELETE_IN_PROGRESS and CREATE_IN_PROGRESS.  Solution to this can be found out in https://aws.amazon.com/premiumsupport/knowledge-center/cloudformation-lambda-resource-delete/ 
• We must set a timeout value On our Lambda function to respond to AWS CloudFormation with an error when a function is about to time out. A timer can help prevent delays for custom resources. 
• AWS CloudFormation will send your function a Create, Update, or Delete event depending on the stack action, these actions must be handled differently, so be sure that there are no unintended behaviors when any of the three event types is received and also logic to handle exceptions must be considered. 

 

if event[‘RequestType’] == ‘Create’:[Text Wrapping Break]  groupId = creation_of_security_group(event) [Text Wrapping Break]  send_response(event,‘SUCCESS’,physicalResourceId=groupId)[Text Wrapping Break]elif event[‘RequestType’] == ‘Delete’: [Text Wrapping Break]  response = ec2.delete_security_group( GroupId=event[‘PhysicalResourceId’], DryRun=False ) [Text Wrapping Break]  send_response(event,‘SUCCESS’)[Text Wrapping Break]elif event[‘RequestType’] == ‘Update’: [Text Wrapping Break]  groupId = creation_of_security_group(event) [Text Wrapping Break]  send_response(event,‘SUCCESS’,physicalResourceId=groupId)[Text Wrapping Break]

 

And that’s all..!! Hope you found it useful. Keep following our blogs for more interesting articles @ https://www.cloudworkmates.com/blogs/