On this page

Saving to AWS S3

Introduction

This tutorial is designed to guide you through saving files to Amazon Web Services (AWS) S3, a widely-used cloud storage service offering scalability, data availability, security, and performance.

By the end of this tutorial, you'll be able to set up a destination object for saving files to AWS S3, understanding the URL structure, credentials, access permissions, and IAM policy requirements.

Supported Protocols

Protocol	Description
s3://	AWS S3 Storage URL

1
Creating a Bucket in AWS S3

Log into your AWS Management Console.
Navigate to the S3 service and create a new bucket.
Select a region (e.g., us-east-1 or us-west-2) and configure the bucket settings.

For more detailed information and a list of AWS S3 regions, refer to the official Amazon S3 Documentation.

2
Obtaining AWS S3 Credentials:

In the AWS Console, go to your account settings.
Access the Security Credentials section.
Find or create your Access Key ID and Secret Access Key.

3
Saving Output to AWS S3

In order to save your outputs to your AWS S3 Storage, define your destination object using the following structure.

Destination Object Structure

"destination": {
  "url": "s3://s3.[region].amazonaws.com/bucket/path",
  "key": "access_key_id",
  "secret": "access_key_secret",
  "permissions": "permissions",
  "storage_class": "storage_class"
}

Attributes of AWS S3 Destination Object

Attribute	Description
url	Your AWS S3 URL.
key	Your AWS S3 Access Key ID.
secret	Your AWS S3 Secret Access Key.
permissions	Parameter specifies object access permissions. For AWS possible values are: ‘private', ‘public-read', 'authenticated-read', 'bucket-owner-read' and others described in Access Control List Overview. Please note: If your credentials don't have the necessary permissions to set the ACL for objects in the bucket, the output might save successfully to the bucket, but the ACL setting operation for the object could fail.
storage_class	Can specify REDUCED_REDUNDANCY value in order to lower your storage costs for noncritical, reproducible data. See Reduced redundancy storage description.

URL Structure for AWS

The url used for your AWS S3 Storage destination object consists of region, bucket and path if applicable. Check the examples for single file and ABR formats like HLS and DASH below:

s3://s3.[region].amazonaws.com/[bucket]/[path]

When saving ABR outputs like HLS or DASH, please specify the path to a folder where the contents of the HLS or DASH stream should be located:

s3://s3.[region].amazonaws.com/[bucket]/folder

Components of URL

region	is one of the AWS S3 regions, e.g. us-east-1 or us-west-2.
bucket	Your AWS S3 bucket name.
path	The path within your bucket where the file will be saved.

Destination Object Example

"destination": {
  "url": "s3://s3.us-east-1.amazonaws.com/my-bucket/folder/output.mp4",
  "key": "myAccessKeyId",
  "secret": "mySecretAccessKey",
  "permissions": "public-read",
  "storage_class": "REDUCED_REDUNDANCY"
}

"destination": {
  "url": "s3://s3.us-east-1.amazonaws.com/my-bucket/folder",
  "key": "myAccessKeyId",
  "secret": "mySecretAccessKey",
  "permissions": "public-read",
  "storage_class": "REDUCED_REDUNDANCY"
}

Considerations

Access Permissions: Choose appropriate permissions such as 'private', 'public-read', etc.
Storage Class: Use 'REDUCED_REDUNDANCY' for noncritical, reproducible data to lower storage costs.
Security: Ensure your IAM policies and credentials are securely managed.
Connectivity Issues: Verify the bucket name, region, and path in the URL.
Authentication Errors: Double-check the accuracy of your access key and secret key.
Explore AWS S3’s advanced features like lifecycle policies, versioning, and cross-region replication.