Data Storage and Analytics

Data Storage and Analytics

Sensor data needs to be stored and analysed. In some cases these two activities are separate, in others they are performed by the same system.

Data is usually received via the MQTT protocol but HTTP(S) is also suitable in situations requiring less throughput. This means a system needs to accept MQTT or HTTP requests and store the data in a database. Traditional databases can be used but some systems use more specialist databases, such as Cassandra, that are optimised for storing serial data.

A very large number of IoT platforms are available. Of particular note are AWS IoT Analytics and Google Cloud IoT Core being provided by the top cloud providers, ThingSpeak due to its in-built MATLAB analytics and ThingsBoard an open source platform you can host yourself. The very large number of IoT providers means many won’t survive so factor this and due diligence into choosing a platform.

Most systems provide configurable dashboards where you can display charts, dials and stats via a web interface. Some provide custom processing and triggering of alerts to people and other systems. There are also standalone systems such as Grafana that provide for the visual aspects.

SMEs have smaller and simpler needs and in many cases should avoid getting unnecessarily tied into large and complex IoT platforms with unknown future fees. Instead, consider hosting your own databases and/or self-hosted IoT platforms.

Analytics is very important as some organisations, especially larger ones, have been able to collect data but have found it more difficult to capture real value from analytics. Some of this comes down to having the right skills and the politics of adapting processes but, in part, it’s also about using the appropriate tools.

“Value is likely to accrue to the owners of scarce data, to players that aggregate data in unique ways … Organizations that are able to harness these capabilities effectively will be able to create significant value and differentiate themselves, while others will find themselves increasingly at a disadvantage.”
McKinsey The Age of Analytics: Competing in a Data-driven World


Featured Suppliers