A Primer on Edge Analytics : Using an AWS envoirnment example

On Edge Analytics will drive Industry 4.0 networks

On-edge analytics are the newest class of analytics in the IoT but few of us know that they have been the oldest class of analytics in the industrial sector. In fact, industries started developing analytics on the controller way before the IoT hype. As analytics needs evolve, the notion that everything should be cloud-based has started to become less important and many vendors have started deploying analytics on the edge, also often known as the fog.

Examples of instances where Edge analytics should be leveraged in Industry 4.0 scenario are:

  • If you have high volume of sampling data, such as the vibration of a turbine
  • If you have a policy restriction regarding sending data outside a perimeter
  • If you require low or minimal latency

Note that Analytics on the controller is different from edge analytics

In real time envoirnments, it is rare that cloud-based or edge-based analytics are allowed to perform actions on the controller in the industrial sector. On the contrary, the preference is that analytics that are optimized on the controller carry out operations that could impact production. This is due to issues of security and a lack of robustness. Moreover, these operations had already been made before the revolution of the IoT and therefore there are already widely tested technologies based on on-controller analytics.

Normally, these analytics work on a Real-time Operating System (RTOS), which is closer to the edge device, very far from the high latency of the cloud.

AWS and Google, however, are working to deliver an RTOS that works closely with an Edge device as well.

So now let us walk through a simplified architecture of Edge Analytics to understand how it works.

A simplified Edge Analytics architecture leveraging AWS IoT Greengrass

The following diagram is the simplified architecture for the I-IoT using AWS . We will start with this very high level view and the build a more detailed architecture.

Capture.JPG

If the above architecture, even after being simplified looks complex, don’t panic. We will simplify this architecture and will help you develop an in depth understanding. The Architecture at a high level consists of the steps shown in the figure below:

  • Device Management
  • Data Acquisition
  • Data Storage
  • Data Processing
  • Execution

We will now explore each of these in detail:

Device Management

As many IoT deployments consist of hundreds of thousands to millions of devices, it is essential to track, monitor, and manage connected device fleets. You need to ensure your IoT devices work properly and securely after they have been deployed. You also need to secure access to your devices, monitor health, detect and remotely troubleshoot problems, and manage software and firmware updates.

Capture

AWS IoT Device Management makes it easy to securely onboard, organize, monitor, and remotely manage IoT devices at scale. With AWS IoT Device Management, you can register your connected devices individually or in bulk, and easily manage permissions so that devices remain secure. You can also organize your devices, monitor and troubleshoot device functionality, query the state of any IoT device in your fleet, and send firmware updates over-the-air (OTA). AWS IoT Device Management is agnostic to device type and OS, so you can manage devices from constrained microcontrollers to connected cars all with the same service. AWS IoT Device Management allows you to scale your fleets and reduce the cost and effort of managing large and diverse IoT device deployments.

Data Accquisition

Data Accquisition originates on the edge device but it is captured using AWS Greengrass ??

Capture

AWS Greengrass

AWS IoT Greengrass is software that lets you run local compute, messaging, data caching, sync, and ML inference capabilities on connected devices in a secure way. With AWS IoT Greengrass, connected devices can run AWS Lambda functions, execute predictions based on machine learning models, keep device data in sync, and communicate with other devices securely – even when not connected to the Internet.

AWS IoT Greengrass seamlessly extends AWS to devices so they can act locally on the data they generate, while still using the cloud for management, analytics, and durable storage. With AWS IoT Greengrass, you can use familiar languages and programming models to create your device software in the cloud, and then deploy it to your devices. AWS IoT Greengrass can be programmed to filter device data and only transmit necessary information back to the cloud.

Data Storage

Capture.JPG

AWS IoT Core is a managed cloud service that lets connected devices easily and securely interact with cloud applications and other devices. AWS IoT Core can support billions of devices and trillions of messages, and can process and route those messages to AWS endpoints and to other devices reliably and securely. With AWS IoT Core, your applications can keep track of and communicate with all your devices, all the time, even when they aren’t connected.

AWS IoT Core also makes it easy to use AWS services like AWS Lambda, Amazon Kinesis, Amazon S3, Amazon SageMaker, Amazon DynamoDB, Amazon CloudWatch, AWS CloudTrail, and Amazon QuickSight, to build IoT applications that gather, process, analyze and act on data generated by connected devices, without having to manage any infrastructure.

Other components of AWS that are not specific to the IoT are listed as follows:

  • AWS QuickSight : Amazon QuickSight is a fast, cloud-powered business intelligence service that makes it easy to deliver insights to everyone in your organization.As a fully managed service, QuickSight lets you easily create and publish interactive dashboards that include ML Insights. Dashboards can then be accessed from any device, and embedded into your applications, portals, and websites.                               .
  • AWS SageMaker : Amazon SageMaker provides every developer and data scientist with the ability to build, train, and deploy machine learning models quickly. Amazon SageMaker is a fully-managed service that covers the entire machine learning workflow to label and prepare your data, choose an algorithm, train the model, tune and optimize it for deployment, make predictions, and take action. Your models get to production faster with much less effort and lower cost.                   .
  • AWS Athena: Amazon Athena is an interactive query service that makes it easy to analyze data in Amazon S3 using standard SQL. Athena is serverless, so there is no infrastructure to manage, and you pay only for the queries that you run.                      .
  • AWS DynamoDB : Amazon DynamoDB is a key-value and document database that delivers single-digit millisecond performance at any scale. It’s a fully managed, multiregion, multimaster, durable database with built-in security, backup and restore, and in-memory caching for internet-scale applications. DynamoDB can handle more than 10 trillion requests per day and can support peaks of more than 20 million requests per second.                                                                                                 .
  • AWS Lambda : AWS Lambda lets you run code without provisioning or managing servers. You pay only for the compute time you consume – there is no charge when your code is not running.  With Lambda, you can run code for virtually any type of application or backend service – all with zero administration. Just upload your code and Lambda takes care of everything required to run and scale your code with high availability. You can set up your code to automatically trigger from other AWS services or call it directly from any web or mobile app.                                                .
  • AWS S3: Amazon Simple Storage Service (Amazon S3) is an object storage service that offers industry-leading scalability, data availability, security, and performance. This means customers of all sizes and industries can use it to store and protect any amount of data for a range of use cases, such as websites, mobile applications, backup and restore, archive, enterprise applications, IoT devices, and big data analytics. Amazon S3 provides easy-to-use management features so you can organize your data and configure finely-tuned access controls to meet your specific business, organizational, and compliance requirements. Amazon S3 is designed for 99.999999999% (11 9’s) of durability, and stores data for millions of applications for companies all around the world.                                                                                               .
  • AWS Machine Learning (ML) analytics:You can choose from pre-trained AI services for computer vision, language, recommendations, and forecasting; Amazon SageMaker to quickly build, train and deploy machine learning models at scale; or build custom models with support for all the popular open-source frameworks.

IoT Core is the hub to which the devices send data using several protocols. The data can be stored in DynamoDB for time-series data, or S3 for object data. AWS allows us to perform this storing action by enabling simple rules from IoT Core. In parallel, we can process data using AWS Lambda, a serverless platform, or using IoT Analytics.

Results can either be stored on S3 or Elasticsearch, or directly connected to QuickSight or Kibana for fast visualizing. AWS also allows you to export data from DynamoDB to S3 with a few clicks. We can use AWS ML, SageMaker, or Athena for batch processing. Finally, AWS allows us to carry out actions on the edge by deploying AWS Lambda on the on-premises component Greengrass.

————————————————————————————————————

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s