Amazon S3 action in AWS IoT

Rohith Kumar -

Hello People. This article discusses about Amazon S3 action in AWS IoT. What is Amazon S3? Amazon S3 is an object storage service on the cloud with unparalleled industry-leading scalability, data availability, security and performance.

Amazon S3 is one of the Amazon legacy services launched in 2006.  Object-based storage is an architecture that manages unstructured data in the form of object blobs along with metadata. It is opposite to other storage architectures like file-systems which treats data as a file hierarchy ( used in Operating Systems ) and block storage which manages data as blocks within sectors and tracks (used in physical storage devices like Hard Disk).

The basic storage unit of S3 is a bucket having a unique user-assigned key called bucket-name. Files or objects can be stored within these buckets. Buckets and its content can be managed by the GUI ( AWS console ) or programmatically via language-specific SDK or Amazon S3 REST Endpoints.

Talking about the file size, each file in the S3 bucket can be up to 5 Terabytes in size with up to 2 Kilobytes of metadata. Files larger than 5 Terabytes can be stored as multi-part files. But the logic of breaking down while storing in S3 and consolidating the file parts after they are retrieved from S3 needs to be handled by the consumer.

Amazon S3 action in AWS IoT

Types of S3 Storage Classes

  • S3 Standard
  • S3 Standard-IA
  • S3 Intelligent-Tiering
  • S3 One Zone-IA
  • S3 Glacier
  • S3 Glacier Deep Archive
  • S3 Outposts

S3 Standard

S3 Standard is the default storage class if none of the storage class is specified during upload. It is ideal for frequently accessed data because it provides low latency and high availability. It has a wide range of use cases from cloud applications and web services, websites hosting, big data analytics, mobile gaming, and content distribution. It is the most expensive storage class among all others.

Are you looking to start your business in the electric vehicle industry? We provide software development, web application development, mobile application development, charging stations management app, electric vehicle fleet management software development, cyber security and all software services. Please check our home page here https://iwheels.co/

https://iwheels.co/

Ok. Let's get back to the article.

Key Points:

  • High Availability and low latency
  • Data is stored in multiple locations. So it is resilient against events that affect an entire Availability Zone
  • The durability of 99.999999999% and availability of 99.99% availability over a given year
  • Most expensive storage class among all others.

S3 Standard-IA

S3 Standard-Infrequent Access is optimized for long-lived and less frequently accessed data but requires rapid access whenever required. Similar to S3 Standard, it also offers high durability, low latency, and high throughput but has a low per GB storage price and per GB retrieval fee. The S3 Standard-IA is ideal for backups, long-term storage, and as a data store for disaster recovery

Key Points:

  • High Availability and Low Latency (Same as S3 Standard)
  • Offers greater availability and resiliency than the OneZone-IA storage.
  • The durability of 99.999999999% and availability of 99.99% availability over a given year
  • Less expensive than S3 Standard storage but you will be charged a retrieval fee hence suitable for infrequently accessed data.

S3 Intelligent-Tiering

S3 Intelligent-Tiering optimizes costs by automatically moving data to the most cost-effective access tier, without performance impact or operational overhead. It moves objects that have not been accessed for 30 consecutive days to the infrequent access tier. If the object is accessed then it is automatically moved back to the frequent access tier. No retrieval fees or additional tiering fees are using the S3 Intelligent-Tiering storage class. It is ideal for storing long-lived data where the access patterns are unknown.

Key Points:

  • Low latency and high throughput performance
  • Automatically moves the data between two access tiers. (Infrequent Access and Frequent Access)
  • The durability of 99.999999999% and availability of 99.99% availability over a given year
  • Small monthly monitoring and auto-tiering fee

S3 One Zone-IA

S3 One Zone- Infrequent Access is for the data that is accessed less frequently but available for millisecond access. Since the other S3 storage class store data in a minimum of 3 Availability Zones (AZ), S3 One Zone-IA stores data in only one AZ which makes the costs 20% lesser than the S3 Standard-IA. It offers the same high durability, high throughput, and low latency. It can be considered as a good choice for storing secondary backup copies or easily re-creatable data if an AZ fails.

Key Points:

  • Low Latency and High throughput performance
  • The durability of 99.999999999% and availability of 99.5% availability over a given year
  • Data will be lost if the Availability Zone where the data is stored is destroyed.
  • Suitable for larger objects greater than 128 KB kept for at least 30 days (charged minimum for 30 days)

S3 Glacier

S3 Glacier is a low-cost storage class for data archiving where data access is infrequent. It provides a configurable retrieval time for the data from minutes to hours. This storage class uses a very low-cost Glacier storage service but the objects are still managed through S3.

Key Points:

  • Low-cost design for long-term archiving
  • Data will be available in case of entire Availability Zone destruction
  • The durability of 99.999999999% and availability of 99.9% availability over a given year
  • It has a minimum storage duration period of 90 days.

S3 Glacier Deep Archive

The S3 Glacier Deep Archive provides the lowest-cost storage class and supports long-term retention and digital preservation for data that may be accessed only once or twice in a year. It is ideal for those industries which store data for 5-10 years or longer like healthcare, finance, etc. It can also be used for backup and disaster recovery.

Key Points:

  • Lowest cost storage option in S3
  • The durability of 99.999999999% and availability of 99.9% availability over a given year
  • Retrieval costs can be reduced by using bulk retrieval
  • It has a minimum storage duration period of 180 days

S3 Outposts

S3 on Outposts provides object storage to our on-premises AWS outposts environment. S3 on Outposts makes it easy to store, retrieve, secure, control access, tag, and report on the data. It is ideal for workloads with local data residency requirements, and to satisfy demanding performance needs by keeping data close to on-premises.

Key Points:

  • S3 Object compatibility and bucket management is through S3 SDK
  • For durable and redundant storage of data on Outposts
  • S3 on Outposts will give users 48TB or 96TB of S3 storage capacity, with up 100 buckets on each Outpost.

S3 action in AWS IoT

The S3 action writes the data from an MQTT message to an Amazon Simple Storage Service (Amazon S3) bucket.

Requirements

This rule action has some requirements as below:

An IAM role that AWS IoT can assume to perform the s3:PutObject operation.

Go to AWS IoT console, you can choose or create a role to allow AWS IoT to perform this rule action.

  • If you use an AWS Key Management Service (AWS KMS) customer-managed AWS KMS key (KMS key) to encrypt data at rest in Amazon S3, the service must have permission to use the AWS KMS key on the caller's behalf.

Parameters

When you create an AWS IoT rule with this action, you must specify the following information:

bucket

The Amazon S3 bucket to which to write data.

Supports substitution templates: API and AWS CLI onlycannedacl

(Optional) The Amazon S3 canned ACL that controls access to the object identified by the object key.

Supports substitution templates: Nokey

The path to the file where the data is written.

Consider an example where this parameter is ${topic()}/${timestamp()} and the rule receives a message where the topic is some/topic. If the current timestamp is 1460685389, then this action writes the data to a file called 1460685389 in the some/topic folder of the S3 bucket.Note

If you use a static key, AWS IoT overwrites a single file each time the rule invokes. We recommend that you use the message timestamp or another unique message identifier so that a new file is saved in Amazon S3 for each message received.

Supports substitution templates: YesroleArn

The IAM role that allows access to the Amazon S3 bucket.

Supports substitution templates: No

Hope this article on Amazon S3 action in AWS IoT is useful to you. Please read about Tata power EV charging stations in Andhra Pradesh