Module 4

All About AWS Storage

Module 7 | AWS Cloud Foundations

Types of Storage

S3 Buckets

• Amazon S3 stores massive (unlimited) amounts of unstructured data

• .Amazon S3 stores data files as objects in a bucket that you define.

• Five TB is the maximum file size of a single object

• .Objects have a globally unique URL (universal namespace).

• Objects have a key, version ID, value, metadata, and sub-resources

Use cases

• Media hosting (Netflix, Spotify, etc)

• Static websites

Options for uploading

• AWS Management Console

• AWS CLI

• AWS SDK

  • Use AWS SDKs to upload objects programmatically.

• AWS S3 Rest API's

  • Send a PUT request to upload data in a single operation

S3 Bucket Feature: Multi-Part Upload

• Improve throughput.

• Recover quickly from any network issues.

• Pause and resume object uploads.

• Begin an upload before you know the final object size

S3 Transfer Acceleration

You have an S3 bucket that is Canada Central. It has a web portal that allows for media upload. However, we have users around the world. The traffic will have to be routed a poop ton.

Solution is to use an AWS Edge Location to reduce latency.

• Provides fast and secure transfers of files over long distances.

• Optimizes transfer speeds from across the world into S3 buckets.

• Uses globally distributed edge locations in CloudFront.

• Improves speed by 50–500 percent on average for cross-country transfer of larger objects

AWS Transfer Family

Allows you to use other protocols like SFTP, FTPS, or SSH to transfer files and data rather than just using HTTPS

Is used to transfer files into and out of Amazon S3 storage or Amazon Elastic File System (Amazon EFS) file systems over the following protocols

Benefits

• Scaling in real time

• You don't need to modify your applications or run any file transfer protocol infrastructure.

• With Transfer Family, you use native AWS services for processing, analytics, reporting, auditing, and archival functions with your data in durable Amazon S3 storage

• Transfer Family is a managed elastic file system (with Amazon EFS) for use with AWS Cloud services and on-premises resources.

• Transfer Family is a managed, serverless file transfer workflow service that you can use to set up, run, automate, and monitor file uploads.

• You pay for only the use of the service, and there are no upfront costs.

Use Cases

Object Storage Classes

Amazon S3 Lifecycle

Move files across all the different tiers based on access frequency and time

Create a policy -> Data will automatically transfer to a different storage class without any changes to your application

AWS Versioning

• Protects objects from accidental overwrites and deletes

• Amazon S3 generates a new version ID and adds this newer version of the object to the bucket

• The original version remains in the bucket

Cross-Origin Resource Sharing

Essentially allows you to use resources (pictures) from your S3 bucket on different websites/domains

This is configured by creating a CORS policy that allows for GET requests for other domain resources. This policy is written in XML

When Amazon S3 receives a preflight request from a browser, it evaluates the CORS configuration for the bucket and uses a CORS rule that matches the incoming browser request to allow a cross-origin request.

Data Consistency Model

• Is consistent for all new and existing objects in all Regions

• Provides read-after-write consistency for all GET, LIST, PUT, and operations on objects in S3 buckets

• Offers an advantage for big data workloads•Simplifies the migration of on-premises analytics workloads

Encryptions with S3 Buckets

• Server-side encryption

  • Amazon S3 encrypts objects before it saves the objects to disk and decrypts the objects when you download them.

    Enable this feature by selecting the default encryption option on the bucket

• Client-side encryption

  • Encrypt data on the client side and upload the encrypted data to Amazon S3.

  • In this case, you manage the encryption process

Tools for protecting S3 Buckets

Configuring Access

S3 Buckets Best Practices