How to Implement VOD Hosting on AWS

Video has become one of the most popular media of consuming information across the globe. Whether it's entertainment, education or broadcast media industry, having a self managed video hosting or serverless, cloud based video hosting is a must. There are many video platforms like Youtube, Vimeo, Dailymotion and you name it.

In this tutorial, we will be creating a completely automated workflow where video processing, conversion, making it stream ready and sending user/server a notification about completion will be done using several AWS services like S3, Lambda, MediaConvert, CloudWatch, SNS Notification etc.

1. Create an input S3 bucket

At first, we will create an S3 bucket where videos will be uploaded before conversion. Let's name the bucket vodapp-input (choose any name you prefer):

• In the AWS Management Console choose Services then select S3 under Storage.
• Choose Create Bucket.
• Provide a unique name for your bucket. For our case, vodapp-input .
• Select the Region you prefer to store your video. Videos from this bucket won't be served to end user, still choose the nearest bucket for better uploading speed.
• Click on Create and don't copy settings from any previous bucket.

2. Create an output S3 bucket

Now, we will create an S3 bucket where the converted and processed videos will be stored and served to end user (unless we use AWS Cloudfront CDN). Let's name the bucket vodapp-output:

• In the AWS Management Console choose Services then select S3 under Storage.
• Choose Create Bucket.
• Provide a unique name for your bucket. For our case, vodapp-input .
• Select the region which is nearest to your app's end user.
• Click on Create and don't copy settings from any previous bucket.
• Go to S3 Console and choose the bucket we just created. Select Properties > Static web hosting > Use this bucket to host website.
• Enter index.html in the Index document box and Save.
• In the Permissions tab, scroll down to Block public access (bucket settings), uncheck Block all public access.

In the same tab, scroll to Cross-origin resource sharing (CORS) and enter the JSON configuration given below:

[
  {
      "AllowedHeaders": [
          "*"
      ],
      "AllowedMethods": [
          "GET"
      ],
      "AllowedOrigins": [
          "https://yoursite.com", // replace this with your app's web address
      ],
      "ExposeHeaders": [],
      "MaxAgeSeconds": 3000
  }
]

3. Create an IAM Role for MediaConvert

At this step, we will be creating an IAM (Identity Access Management) Role for AWS Elemental MediaConvert so that, it can access the S3 bucket files and generate CloudWatch events throughout the media conversion process. Let's follow these steps:

• In the AWS management console, navigate to Services > IAM.
• In the left menu, click on Roles and then Create Role.

Select AWS Service as Role type and for use case choose MediaConvert, then click Next.

• Click Next:Review, Enter MediaConvertRole as Role name (or you can choose as you wish).
• Click on Create Role.
• Enter MediaConvertRole in the filter box on Roles page, and select the role we just created.
• An ARN ID will be shown, keep it noted for later.

4. Create an IAM Role for Lambda function

Lambda functions interact with other AWS services through IAM role. To let our Lambda function interact with MediaConvert and other services, we have to create an IAM role.

• In the AWS management console, navigate to Services > IAM.
• In the left menu, click on Roles and then Create Role.
• Select AWS Service as Role type and for use case choose Lambda, then click Next.
• In the filter box, type AWSLambdaBasicExecutionRole and check the corresponding box. then click Next:Review.
• Role name: VODLambdaRole
• Select Create Role.
• Enter VODLambdaRole in the filter box on Roles page, and select the role we just created.
• Now, navigate to Permissions tab and expand Add Inline Policies, then select JSON tab.

Paste the following JSON policy in the Policy Document Box:

{
  "Version": "2012-10-17",
  "Statement": [
      {
          "Action": [
              "logs:CreateLogGroup",
              "logs:CreateLogStream",
              "logs:PutLogEvents"
          ],
          "Resource": "*",
          "Effect": "Allow",
          "Sid": "Logging"
      },
      {
          "Action": [
              "iam:PassRole"
          ],
          "Resource": [
              "MediaConvertRoleARN" // replace with the ARN we saved earlier
          ],
          "Effect": "Allow",
          "Sid": "PassRole"
      },
      {
          "Action": [
              "mediaconvert:*"
          ],
          "Resource": [
              "*"
          ],
          "Effect": "Allow",
          "Sid": "MediaConvertService"
      },
      {
          "Action": [
              "s3:*"
          ],
          "Resource": [
              "*"
          ],
          "Effect": "Allow",
          "Sid": "S3Service"
      }
  ]
}

• Replace MediaConvertRoleARN this with the ARN we saved in Step 3.
• Click Review Policy and enter name of the policy as VODLambdaPolicy.
• Click on Create Policy.

5. Create a Lambda function for video conversion

Whenever we upload (putObject event of S3) a video file in the vodapp-input bucket, we want MediaConvert to automatically get triggered to start video conversion. A Lambda function can trigger this whenever putObject event occurs in the S3 bucket and also the MediaConvert Python SDK can be used to customize and set video conversion parameters.

• In AWS Management Console, navigate to Services > Lambda.
• Select Create function.
• Select Author From Scratch and enter VODLambdaConvert as Function Name.
• Select Python 3.8 as the Runtime from the dropdown list.
• In the Existing Role dropdown, select VODLambdaRole (we have created this role in previous steps).

Create a file in your computer, named convert.py and paste the following code:

#!/usr/bin/env python

import glob
import json
import os
import uuid
import boto3
import datetime
import random
from urllib.parse import urlparse
import logging

from botocore.client import ClientError

logger = logging.getLogger()
logger.setLevel(logging.INFO)

S3 = boto3.resource('s3')

def handler(event, context):
  '''
  Watchfolder handler - this lambda is triggered when video objects are uploaded to the 
  SourceS3Bucket/inputs folder.

  It will look for two sets of file inputs:
      SourceS3Bucket/inputs/SourceS3Key:
          the input video to be converted

      SourceS3Bucket/jobs/*.json:
          job settings for MediaConvert jobs to be run against the input video. If 
          there are no settings files in the jobs folder, then the Default job will be run 
          from the job.json file in lambda environment. 

  Ouput paths stored in outputGroup['OutputGroupSettings']['DashIsoGroupSettings']['Destination']
  are constructed from the name of the job settings files as follows:

      s3://<MediaBucket>/<basename(job settings filename)>/<basename(input)>/<Destination value from job settings file>

  '''

  assetID = str(uuid.uuid4())
  sourceS3Bucket = event['Records'][0]['s3']['bucket']['name']
  sourceS3Key = event['Records'][0]['s3']['object']['key']
  sourceS3 = 's3://'+ sourceS3Bucket + '/' + sourceS3Key
  destinationS3 = 's3://' + os.environ['DestinationBucket']
  mediaConvertRole = os.environ['MediaConvertRole']
  application = os.environ['Application']
  region = os.environ['AWS_DEFAULT_REGION']
  statusCode = 200
  jobs = []
  job = {}

  # Use MediaConvert SDK UserMetadata to tag jobs with the assetID 
  # Events from MediaConvert will have the assetID in UserMedata
  jobMetadata = {}
  jobMetadata['assetID'] = assetID
  jobMetadata['application'] = application
  jobMetadata['input'] = sourceS3

  try:    

      # Build a list of jobs to run against the input.  Use the settings files in WatchFolder/jobs
      # if any exist.  Otherwise, use the default job.

      jobInput = {}
      # Iterates through all the objects in jobs folder of the WatchFolder bucket, doing the pagination for you. Each obj
      # contains a jobSettings JSON
      bucket = S3.Bucket(sourceS3Bucket)
      for obj in bucket.objects.filter(Prefix='jobs/'):
          if obj.key != "jobs/":
              jobInput = {}
              jobInput['filename'] = obj.key
              logger.info('jobInput: %s', jobInput['filename'])

              jobInput['settings'] = json.loads(obj.get()['Body'].read())
              logger.info(json.dumps(jobInput['settings'])) 

              jobs.append(jobInput)

      # Use Default job settings in the lambda zip file in the current working directory
      if not jobs:

          with open('job.json') as json_data:
              jobInput['filename'] = 'Default'
              logger.info('jobInput: %s', jobInput['filename'])

              jobInput['settings'] = json.load(json_data)
              logger.info(json.dumps(jobInput['settings']))

              jobs.append(jobInput)

      # get the account-specific mediaconvert endpoint for this region
      mediaconvert_client = boto3.client('mediaconvert', region_name=region)
      endpoints = mediaconvert_client.describe_endpoints()

      # add the account-specific endpoint to the client session 
      client = boto3.client('mediaconvert', region_name=region, endpoint_url=endpoints['Endpoints'][0]['Url'], verify=False)

      for j in jobs:
          jobSettings = j['settings']
          jobFilename = j['filename']

          # Save the name of the settings file in the job userMetadata
          jobMetadata['settings'] = jobFilename

          # Update the job settings with the source video from the S3 event 
          jobSettings['Inputs'][0]['FileInput'] = sourceS3

          # Update the job settings with the destination paths for converted videos.  We want to replace the
          # destination bucket of the output paths in the job settings, but keep the rest of the
          # path
          destinationS3 = 's3://' + os.environ['DestinationBucket'] + '/' \
              + os.path.splitext(os.path.basename(sourceS3Key))[0] + '/' \
              + os.path.splitext(os.path.basename(jobFilename))[0]                 

          for outputGroup in jobSettings['OutputGroups']:

              logger.info("outputGroup['OutputGroupSettings']['Type'] == %s", outputGroup['OutputGroupSettings']['Type']) 

              if outputGroup['OutputGroupSettings']['Type'] == 'FILE_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['FileGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['FileGroupSettings']['Destination'] = destinationS3+templateDestinationKey

              elif outputGroup['OutputGroupSettings']['Type'] == 'HLS_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['HlsGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['HlsGroupSettings']['Destination'] = destinationS3+templateDestinationKey

              elif outputGroup['OutputGroupSettings']['Type'] == 'DASH_ISO_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['DashIsoGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['DashIsoGroupSettings']['Destination'] = destinationS3+templateDestinationKey

              elif outputGroup['OutputGroupSettings']['Type'] == 'DASH_ISO_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['DashIsoGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['DashIsoGroupSettings']['Destination'] = destinationS3+templateDestinationKey

              elif outputGroup['OutputGroupSettings']['Type'] == 'MS_SMOOTH_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['MsSmoothGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['MsSmoothGroupSettings']['Destination'] = destinationS3+templateDestinationKey

              elif outputGroup['OutputGroupSettings']['Type'] == 'CMAF_GROUP_SETTINGS':
                  templateDestination = outputGroup['OutputGroupSettings']['CmafGroupSettings']['Destination']
                  templateDestinationKey = urlparse(templateDestination).path
                  logger.info("templateDestinationKey == %s", templateDestinationKey)
                  outputGroup['OutputGroupSettings']['CmafGroupSettings']['Destination'] = destinationS3+templateDestinationKey
              else:
                  logger.error("Exception: Unknown Output Group Type %s", outputGroup['OutputGroupSettings']['Type'])
                  statusCode = 500

          logger.info(json.dumps(jobSettings))

          # Convert the video using AWS Elemental MediaConvert
          job = client.create_job(Role=mediaConvertRole, UserMetadata=jobMetadata, Settings=jobSettings)

  except Exception as e:
      logger.error('Exception: %s', e)
      statusCode = 500
      raise

  finally:
      return {
          'statusCode': statusCode,
          'body': json.dumps(job, indent=4, sort_keys=True, default=str),
          'headers': {'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*'}
      }

In the same folder of your computer, create a file named job.json and paste the following code:

{
  "OutputGroups": [
    {
      "CustomName": "HLS",
      "Name": "Apple HLS",
      "Outputs": [
        {
          "ContainerSettings": {
            "Container": "M3U8",
            "M3u8Settings": {
              "AudioFramesPerPes": 4,
              "PcrControl": "PCR_EVERY_PES_PACKET",
              "PmtPid": 480,
              "PrivateMetadataPid": 503,
              "ProgramNumber": 1,
              "PatInterval": 0,
              "PmtInterval": 0,
              "Scte35Source": "NONE",
              "TimedMetadata": "NONE",
              "VideoPid": 481,
              "AudioPids": [
                482,
                483,
                484,
                485,
                486,
                487,
                488,
                489,
                490,
                491,
                492
              ]
            }
          },
          "VideoDescription": {
            "Width": 640,
            "ScalingBehavior": "DEFAULT",
            "Height": 360,
            "TimecodeInsertion": "DISABLED",
            "AntiAlias": "ENABLED",
            "Sharpness": 50,
            "CodecSettings": {
              "Codec": "H_264",
              "H264Settings": {
                "InterlaceMode": "PROGRESSIVE",
                "NumberReferenceFrames": 3,
                "Syntax": "DEFAULT",
                "Softness": 0,
                "GopClosedCadence": 1,
                "GopSize": 90,
                "Slices": 1,
                "GopBReference": "DISABLED",
                "SlowPal": "DISABLED",
                "SpatialAdaptiveQuantization": "ENABLED",
                "TemporalAdaptiveQuantization": "ENABLED",
                "FlickerAdaptiveQuantization": "DISABLED",
                "EntropyEncoding": "CABAC",
                "Bitrate": 1000000,
                "FramerateControl": "INITIALIZE_FROM_SOURCE",
                "RateControlMode": "CBR",
                "CodecProfile": "MAIN",
                "Telecine": "NONE",
                "MinIInterval": 0,
                "AdaptiveQuantization": "HIGH",
                "CodecLevel": "AUTO",
                "FieldEncoding": "PAFF",
                "SceneChangeDetect": "ENABLED",
                "QualityTuningLevel": "SINGLE_PASS",
                "FramerateConversionAlgorithm": "DUPLICATE_DROP",
                "UnregisteredSeiTimecode": "DISABLED",
                "GopSizeUnits": "FRAMES",
                "ParControl": "INITIALIZE_FROM_SOURCE",
                "NumberBFramesBetweenReferenceFrames": 2,
                "RepeatPps": "DISABLED"
              }
            },
            "AfdSignaling": "NONE",
            "DropFrameTimecode": "ENABLED",
            "RespondToAfd": "NONE",
            "ColorMetadata": "INSERT"
          },
          "AudioDescriptions": [
            {
              "AudioTypeControl": "FOLLOW_INPUT",
              "CodecSettings": {
                "Codec": "AAC",
                "AacSettings": {
                  "AudioDescriptionBroadcasterMix": "NORMAL",
                  "Bitrate": 96000,
                  "RateControlMode": "CBR",
                  "CodecProfile": "LC",
                  "CodingMode": "CODING_MODE_2_0",
                  "RawFormat": "NONE",
                  "SampleRate": 48000,
                  "Specification": "MPEG4"
                }
              },
              "LanguageCodeControl": "FOLLOW_INPUT"
            }
          ],
          "OutputSettings": {
            "HlsSettings": {
              "AudioGroupId": "program_audio",
              "AudioRenditionSets": "program_audio",
              "SegmentModifier": "$dt$",
              "IFrameOnlyManifest": "EXCLUDE"
            }
          },
          "NameModifier": "_360"
        },
        {
          "ContainerSettings": {
            "Container": "M3U8",
            "M3u8Settings": {
              "AudioFramesPerPes": 4,
              "PcrControl": "PCR_EVERY_PES_PACKET",
              "PmtPid": 480,
              "PrivateMetadataPid": 503,
              "ProgramNumber": 1,
              "PatInterval": 0,
              "PmtInterval": 0,
              "Scte35Source": "NONE",
              "Scte35Pid": 500,
              "TimedMetadata": "NONE",
              "TimedMetadataPid": 502,
              "VideoPid": 481,
              "AudioPids": [
                482,
                483,
                484,
                485,
                486,
                487,
                488,
                489,
                490,
                491,
                492
              ]
            }
          },
          "VideoDescription": {
            "Width": 960,
            "ScalingBehavior": "DEFAULT",
            "Height": 540,
            "TimecodeInsertion": "DISABLED",
            "AntiAlias": "ENABLED",
            "Sharpness": 50,
            "CodecSettings": {
              "Codec": "H_264",
              "H264Settings": {
                "InterlaceMode": "PROGRESSIVE",
                "NumberReferenceFrames": 3,
                "Syntax": "DEFAULT",
                "Softness": 0,
                "GopClosedCadence": 1,
                "GopSize": 90,
                "Slices": 1,
                "GopBReference": "DISABLED",
                "SlowPal": "DISABLED",
                "SpatialAdaptiveQuantization": "ENABLED",
                "TemporalAdaptiveQuantization": "ENABLED",
                "FlickerAdaptiveQuantization": "DISABLED",
                "EntropyEncoding": "CABAC",
                "Bitrate": 2000000,
                "FramerateControl": "INITIALIZE_FROM_SOURCE",
                "RateControlMode": "CBR",
                "CodecProfile": "MAIN",
                "Telecine": "NONE",
                "MinIInterval": 0,
                "AdaptiveQuantization": "HIGH",
                "CodecLevel": "AUTO",
                "FieldEncoding": "PAFF",
                "SceneChangeDetect": "ENABLED",
                "QualityTuningLevel": "SINGLE_PASS",
                "FramerateConversionAlgorithm": "DUPLICATE_DROP",
                "UnregisteredSeiTimecode": "DISABLED",
                "GopSizeUnits": "FRAMES",
                "ParControl": "INITIALIZE_FROM_SOURCE",
                "NumberBFramesBetweenReferenceFrames": 2,
                "RepeatPps": "DISABLED"
              }
            },
            "AfdSignaling": "NONE",
            "DropFrameTimecode": "ENABLED",
            "RespondToAfd": "NONE",
            "ColorMetadata": "INSERT"
          },
          "AudioDescriptions": [
            {
              "AudioTypeControl": "FOLLOW_INPUT",
              "CodecSettings": {
                "Codec": "AAC",
                "AacSettings": {
                  "AudioDescriptionBroadcasterMix": "NORMAL",
                  "Bitrate": 96000,
                  "RateControlMode": "CBR",
                  "CodecProfile": "LC",
                  "CodingMode": "CODING_MODE_2_0",
                  "RawFormat": "NONE",
                  "SampleRate": 48000,
                  "Specification": "MPEG4"
                }
              },
              "LanguageCodeControl": "FOLLOW_INPUT"
            }
          ],
          "OutputSettings": {
            "HlsSettings": {
              "AudioGroupId": "program_audio",
              "AudioRenditionSets": "program_audio",
              "SegmentModifier": "$dt$",
              "IFrameOnlyManifest": "EXCLUDE"
            }
          },
          "NameModifier": "_540"
        },
        {
          "ContainerSettings": {
            "Container": "M3U8",
            "M3u8Settings": {
              "AudioFramesPerPes": 4,
              "PcrControl": "PCR_EVERY_PES_PACKET",
              "PmtPid": 480,
              "PrivateMetadataPid": 503,
              "ProgramNumber": 1,
              "PatInterval": 0,
              "PmtInterval": 0,
              "Scte35Source": "NONE",
              "Scte35Pid": 500,
              "TimedMetadata": "NONE",
              "TimedMetadataPid": 502,
              "VideoPid": 481,
              "AudioPids": [
                482,
                483,
                484,
                485,
                486,
                487,
                488,
                489,
                490,
                491,
                492
              ]
            }
          },
          "VideoDescription": {
            "Width": 1280,
            "ScalingBehavior": "DEFAULT",
            "Height": 720,
            "TimecodeInsertion": "DISABLED",
            "AntiAlias": "ENABLED",
            "Sharpness": 50,
            "CodecSettings": {
              "Codec": "H_264",
              "H264Settings": {
                "InterlaceMode": "PROGRESSIVE",
                "NumberReferenceFrames": 3,
                "Syntax": "DEFAULT",
                "Softness": 0,
                "GopClosedCadence": 1,
                "GopSize": 90,
                "Slices": 1,
                "GopBReference": "DISABLED",
                "SlowPal": "DISABLED",
                "SpatialAdaptiveQuantization": "ENABLED",
                "TemporalAdaptiveQuantization": "ENABLED",
                "FlickerAdaptiveQuantization": "DISABLED",
                "EntropyEncoding": "CABAC",
                "Bitrate": 3000000,
                "FramerateControl": "INITIALIZE_FROM_SOURCE",
                "RateControlMode": "CBR",
                "CodecProfile": "MAIN",
                "Telecine": "NONE",
                "MinIInterval": 0,
                "AdaptiveQuantization": "HIGH",
                "CodecLevel": "AUTO",
                "FieldEncoding": "PAFF",
                "SceneChangeDetect": "ENABLED",
                "QualityTuningLevel": "SINGLE_PASS",
                "FramerateConversionAlgorithm": "DUPLICATE_DROP",
                "UnregisteredSeiTimecode": "DISABLED",
                "GopSizeUnits": "FRAMES",
                "ParControl": "INITIALIZE_FROM_SOURCE",
                "NumberBFramesBetweenReferenceFrames": 2,
                "RepeatPps": "DISABLED"
              }
            },
            "AfdSignaling": "NONE",
            "DropFrameTimecode": "ENABLED",
            "RespondToAfd": "NONE",
            "ColorMetadata": "INSERT"
          },
          "AudioDescriptions": [
            {
              "AudioTypeControl": "FOLLOW_INPUT",
              "CodecSettings": {
                "Codec": "AAC",
                "AacSettings": {
                  "AudioDescriptionBroadcasterMix": "NORMAL",
                  "Bitrate": 96000,
                  "RateControlMode": "CBR",
                  "CodecProfile": "LC",
                  "CodingMode": "CODING_MODE_2_0",
                  "RawFormat": "NONE",
                  "SampleRate": 48000,
                  "Specification": "MPEG4"
                }
              },
              "LanguageCodeControl": "FOLLOW_INPUT"
            }
          ],
          "OutputSettings": {
            "HlsSettings": {
              "AudioGroupId": "program_audio",
              "AudioRenditionSets": "program_audio",
              "SegmentModifier": "$dt$",
              "IFrameOnlyManifest": "EXCLUDE"
            }
          },
          "NameModifier": "_720"
        }
      ],
      "OutputGroupSettings": {
        "Type": "HLS_GROUP_SETTINGS",
        "HlsGroupSettings": {
          "ManifestDurationFormat": "INTEGER",
          "SegmentLength": 10,
          "TimedMetadataId3Period": 10,
          "CaptionLanguageSetting": "OMIT",
          "Destination": "s3://<MEDIABUCKET>/HLS/",
          "DestinationSettings": {
            "S3Settings": {
              "AccessControl": {
                "CannedAcl": "PUBLIC_READ"
              }
            }
          },
          "TimedMetadataId3Frame": "PRIV",
          "CodecSpecification": "RFC_4281",
          "OutputSelection": "MANIFESTS_AND_SEGMENTS",
          "ProgramDateTimePeriod": 600,
          "MinSegmentLength": 0,
          "DirectoryStructure": "SINGLE_DIRECTORY",
          "ProgramDateTime": "EXCLUDE",
          "SegmentControl": "SEGMENTED_FILES",
          "ManifestCompression": "NONE",
          "ClientCache": "ENABLED",
          "StreamInfResolution": "INCLUDE"
        }
      }
    },
    {
      "CustomName": "MP4",
      "Name": "File Group",
      "Outputs": [
        {
          "ContainerSettings": {
            "Container": "MP4",
            "Mp4Settings": {
              "CslgAtom": "INCLUDE",
              "FreeSpaceBox": "EXCLUDE",
              "MoovPlacement": "PROGRESSIVE_DOWNLOAD"
            }
          },
          "VideoDescription": {
            "Width": 1280,
            "ScalingBehavior": "DEFAULT",
            "Height": 720,
            "TimecodeInsertion": "DISABLED",
            "AntiAlias": "ENABLED",
            "Sharpness": 50,
            "CodecSettings": {
              "Codec": "H_264",
              "H264Settings": {
                "InterlaceMode": "PROGRESSIVE",
                "NumberReferenceFrames": 3,
                "Syntax": "DEFAULT",
                "Softness": 0,
                "GopClosedCadence": 1,
                "GopSize": 90,
                "Slices": 1,
                "GopBReference": "DISABLED",
                "SlowPal": "DISABLED",
                "SpatialAdaptiveQuantization": "ENABLED",
                "TemporalAdaptiveQuantization": "ENABLED",
                "FlickerAdaptiveQuantization": "DISABLED",
                "EntropyEncoding": "CABAC",
                "Bitrate": 3000000,
                "FramerateControl": "INITIALIZE_FROM_SOURCE",
                "RateControlMode": "CBR",
                "CodecProfile": "MAIN",
                "Telecine": "NONE",
                "MinIInterval": 0,
                "AdaptiveQuantization": "HIGH",
                "CodecLevel": "AUTO",
                "FieldEncoding": "PAFF",
                "SceneChangeDetect": "ENABLED",
                "QualityTuningLevel": "SINGLE_PASS",
                "FramerateConversionAlgorithm": "DUPLICATE_DROP",
                "UnregisteredSeiTimecode": "DISABLED",
                "GopSizeUnits": "FRAMES",
                "ParControl": "INITIALIZE_FROM_SOURCE",
                "NumberBFramesBetweenReferenceFrames": 2,
                "RepeatPps": "DISABLED"
              }
            },
            "AfdSignaling": "NONE",
            "DropFrameTimecode": "ENABLED",
            "RespondToAfd": "NONE",
            "ColorMetadata": "INSERT"
          },
          "AudioDescriptions": [
            {
              "AudioTypeControl": "FOLLOW_INPUT",
              "CodecSettings": {
                "Codec": "AAC",
                "AacSettings": {
                  "AudioDescriptionBroadcasterMix": "NORMAL",
                  "Bitrate": 96000,
                  "RateControlMode": "CBR",
                  "CodecProfile": "LC",
                  "CodingMode": "CODING_MODE_2_0",
                  "RawFormat": "NONE",
                  "SampleRate": 48000,
                  "Specification": "MPEG4"
                }
              },
              "LanguageCodeControl": "FOLLOW_INPUT"
            }
          ]
        }
      ],
      "OutputGroupSettings": {
        "Type": "FILE_GROUP_SETTINGS",
        "FileGroupSettings": {
          "Destination": "s3://<MEDIABUCKET>/MP4/",
          "DestinationSettings": {
            "S3Settings": {
              "AccessControl": {
                "CannedAcl": "PUBLIC_READ"
              }
            }
          }
        }
      }
    },
    {
      "CustomName": "Thumbnails",
      "Name": "File Group",
      "Outputs": [
        {
          "ContainerSettings": {
            "Container": "RAW"
          },
          "VideoDescription": {
            "Width": 1280,
            "ScalingBehavior": "DEFAULT",
            "Height": 720,
            "TimecodeInsertion": "DISABLED",
            "AntiAlias": "ENABLED",
            "Sharpness": 50,
            "CodecSettings": {
              "Codec": "FRAME_CAPTURE",
              "FrameCaptureSettings": {
                "FramerateNumerator": 1,
                "FramerateDenominator": 5,
                "MaxCaptures": 500,
                "Quality": 80
              }
            },
            "AfdSignaling": "NONE",
            "DropFrameTimecode": "ENABLED",
            "RespondToAfd": "NONE",
            "ColorMetadata": "INSERT"
          }
        }
      ],
      "OutputGroupSettings": {
        "Type": "FILE_GROUP_SETTINGS",
        "FileGroupSettings": {
          "Destination": "s3://<MEDIABUCKET>/Thumbnails/",
          "DestinationSettings": {
            "S3Settings": {
              "AccessControl": {
                "CannedAcl": "PUBLIC_READ"
              }
            }
          }
        }
      }
    }
  ],
  "AdAvailOffset": 0,
  "Inputs": [
    {
      "AudioSelectors": {
        "Audio Selector 1": {
          "Offset": 0,
          "DefaultSelection": "DEFAULT",
          "ProgramSelection": 1
        }
      },
      "VideoSelector": {
        "ColorSpace": "FOLLOW"
      },
      "FilterEnable": "AUTO",
      "PsiControl": "USE_PSI",
      "FilterStrength": 0,
      "DeblockFilter": "DISABLED",
      "DenoiseFilter": "DISABLED",
      "TimecodeSource": "EMBEDDED",
      "FileInput": "s3://rodeolabz-us-west-2/vodconsole/VANLIFE.m2ts"
    }
  ]
}

• Now, Zip the convert.py and job.json together as a single ZIP file lambda.zip.

In VODLambdaConvert's configuration tab's Function code panel:

• Select Upload a .zip file in Code Entry Type.
• Click Upload and select the ZIP file we created earlier.
• Enter convert.handler for the Handler field.

Now, we have to set some Environment variables so that convert.py can fetch the necessary values from them. In the Environment Variables panel:

• DestinationBucket: vodapp-ouput
• MediaConvertRole:
• Application: VOD

In the Basic Settings panel, enter the following value for Timeout:

• Timeout: 2 min
• Now Save the settings.

If you want to test the Lambda function, you can follow this tutorial.

6. Create an S3 Event Trigger for the Lambda converter function

We will add putItem event trigger in our input S3 bucket, so that whenever a file gets uploaded, this event gets triggered. We want our converter Lambda function to run whenever a file gets uploaded in the input S3 bucket (vodapp-input). In the Lambda function VODLambdaConvert we created earlier, navigate to Configuration > Designer section:

• In Add triggers, click on S3.
• Scroll down to the Configure triggers section:
• Select vodapp-input (the bucket where videos will be uploaded).
• In Event type, select All object create events.
• Enter input/ in the Prefix section (or any folder you want).
• Now, keeping all other settings in default state, click on Add.