A Platform for Sensor Data

Democratizing the ability to build intelligence on the edge partially solves the problem. The real power lies in the ability to collect and process data from millions and millions of sensor devices and take decisions, often in real time. For instance, adjusting the temperature of a thermostat based on usage patterns observed over the last six months. Building a platform that has the ability to handle extremely large amounts of data, make sense of it in real time and to have it scale to exponentially increasing amounts of data is a complex problem. The advent of cloud computing and the increasing adoption of Big Data technology is enabling the ability to process sensor data at scale.

Why Cloud?

An effective sensor driven application relies on collecting sensor data in a centralized location and processing this data at scale. This data can exponentially grow, which means that compute and storage on demand is a necessity. A Case in Point :Working with the same smart metering example outlined earlier. Meter readings can be extracted and bills can be made automatically, which is fairly useful but only scratches the surface of the overall problem. A lot of interesting insights can be gained from the data from millions of meters can be processed to unearth usage patterns by time-of-the-day or by geolocations. The ability to do this requires the right amount of computing power and software platforms designed to handle such data volumes. Today, cloud solutions providers such as Amazon Web Services, Rackspace, and others allow you to take small instances initially and grow into massively scalable data center infrastructure.The Cloud enables democratization of infrastructure platform that makes the Internet of Things (IoT) to become viable and feasible. You do not need to work with a capital expenditure intensive model and buy all the hardware that you will possibly need. The cloud enables the ability to build out an IoT solution that starts of with connecting a limited number of devices that requires limited compute infrastructure. This can then keep scaling up as the number of devices keeps increasing.

Why Big Data?

The cloud provides the infrastructure to handle an exponentially increasing amount of sensor data. Big Data platforms allow you to effectively process this data, at scale on commodity hardware.

A Case in Point : 

You have a sensor and it is sending a single data point every minute (which is a very conservative estimate). This adds up to 1440 data points every minute. Now assume that you have a million sensors pushing data every minute. This is more than a billion data points to be processed even with a fairly conservative data push rate. Big Data technologies consist of systems that can process data in a highly scalable, resilient and distributable manner (the Apache Hadoop ecosystem) and systems that can store increasing amounts of unstructured data efficiently (NoSQL databases like Cassandra or MongoDB). These systems are designed to scale on commodity hardware and provide redundancy, resiliency and provide an increasing number of tools and frameworks to simplify processing and storage of data.You can always opt for building systems using traditional database technology and they will work for most situations. However, it is an expensive challenge to scale such a stack for exponentially increasing amounts of data. You will require specialized hardware and sophisticated scaling strategies. You can start with a small cluster of commodity servers hosted on the cloud and scale up based on demand. Very often, you can also employ strategies like auto-scale up and scale down to ensure that you only pay for processing that you need. Small development teams can build extremely scalable systems. Technology is now truly democratized.

About Altizon Systems

Altizon is world’s first Industrial Internet platform company focused on this space and provides hardware appliances and a big-data platform to kick-start your IoT story. To know how Altizon’s product can help your business, please write to us at sales@altizon.com. To test drive our IoT platform for free, click here for our flagship Datonis^® platform