AWS S3

Describes the functionality of the AWS S3 integration and instructions for use

Overview

The AWS S3 integration allows for data deliveries from Live Data to be shared via S3 for consumption by customers. This provides a secure system for sharing data for those customers that use AWS.

Live Data supports 3 methods of configuring an S3 integration - users can configure any number of these integrations to support the delivery of different datasets to different endpoints.

  • Use your AWS role to access data in a bucket owned by Live Data
  • Grant a Live Data AWS role direct access to a bucket in your AWS account
  • Create a cross-account AWS role in your account and permit Live Data to assume the role

All methods are fully supported by AWS and follow best practices for cross-account access to data in S3 buckets.

Configuration Instructions

Use your AWS role to access data in a bucket owned by Live Data

One of the simplest to configure, in this paradigm you simply provide Live Data your AWS role ARN and we grant that role access to a scoped folder in a bucket in our account. You can then access data in that bucket using your role.

The ARN of the Live Data bucket is: arn:aws:s3:::ldt-customer-deliveries and your data will be in a folder in that bucket that is the same as your Live Data Org ID. This value can be found in the URL of the Integrations page in Live Data and follows the form o_<8-characters>, such as o_8abe35ab.

Steps

  1. Log into gotlivedata.com and click on the Integrations page

  2. Find the AWS S3 card and press Configure

  3. Select Live Data S3 Bucket

  4. Provide a name for this integration (this can be anything)

  5. Enter the ARN of the role in your AWS account that you will use to access the data

  6. Press Save

Verification

To verify that you have access to the data, you can try a simple aws s3 ls command. First find your Org ID by inspecting the URL for the Integrations page and locating the o_<8-chars> value, such as o_8abe35ab. Then run the following command using the AWS CLI:

aws s3 ls s3://ldt-customer-deliveries/<org_id>

If the command completes with no errors, your integration setup is completed.

If you get an error message containing Access Denied, please contact Live Data support.

If you get an error message containing The specified bucket does not existthen double-check the bucket name and path in your command and try again.

Grant a Live Data AWS role direct access to a bucket in your AWS account

With this approach, you create bucket policies to allow Live Data to write directly to a bucket in your account. This approach may be preferred in situations where you want to trigger automation when a file is created in your bucket.

Steps

  1. In AWS, navigate to the bucket you want to grant Live Data access to

  2. Click the Permissions tab and edit the bucket policy to allow the Live Data role to access the bucket as required. At a minimum, the Live Data role must have ListBucket permissions on the bucket and PutObjecton the objects in the bucket. See the example policy below, but also reference AWS documentation and your own security team/standards to ensure these permissions are set up as required. The Live Data role arn is: arn:aws:iam::100811426269:role/customer-delivery-handler

  3. Log into gotlivedata.com and click on the Integrations page

  4. Find the AWS S3 card and press Configure

  5. Select Your S3 Bucket

  6. Select Grant Access to Live Data Role

  7. Provide a name for this integration (this can be anything

  8. Enter the bucket identifier for the bucket in your account. This can be the S3 URI, the ARN, or simply the bucket name.

  9. Press Save

Sample bucket policy

{
	"Version": "2012-10-17",
	"Statement": [
        {
            "Sid": "LiveDataBucket",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::100811426269:role/customer-delivery-handler"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<your-bucket-name>"
        },
        {
            "Sid": "LiveDataObject",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::100811426269:role/customer-delivery-handler"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<your-bucket-name>/*"
        }
    ]
}

Create a cross-account AWS role in your account and permit Live Data to assume the role

This approach uses the AWS cross-account role paradigm to allow Live Data access to your account. The most complex approach of the supported methods, it may be preferred under higher security standards.

Note: although an External ID is optional in AWS, it is required for the Live Data integration

Steps

  1. In AWS create the cross-account role using the Live Data account ID 730335578765. When creating the cross-account role, make sure to create and note the External ID value for added security.

  2. Grant the cross-account role access to the S3 bucket as desired. The Sample bucket policy below can be used as a reference for the minimum required permissions.

  3. Log into gotlivedata.com and click on the Integrations page

  4. Find the AWS S3 card and press Configure

  5. Select Your S3 Bucket

  6. Select Create a Cross-Account Role for Live Data to Use

  7. Provide a name for this integration (this can be anything)

  8. Enter the bucket identifier for the bucket in your account. This can be the S3 URI, the ARN, or simply the bucket name.

  9. Enter the role ARN for the cross-account role that you created for this purpose

  10. Enter the external ID value that you noted when creating the cross-account role

  11. Press Save

Sample bucket policy

{
	"Version": "2012-10-17",
	"Statement": [
        {
            "Sid": "LiveDataBucket",
            "Effect": "Allow",
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::<your-bucket-name>"
        },
        {
            "Sid": "LiveDataObject",
            "Effect": "Allow",
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::<your-bucket-name>/*"
        }
    ]
}