Big Data Use-Case: Real-time Dispenser Maintenance

“Sensor” applications have great potential in the Big Data space. The fact that machines produce the most amount of data (aka sensor data), presents a natural application for the use of the Hadoop framework for collecting, storing, applying simple  analytics, and automating action.

The above picture presents a solution for a network of dispensers of food items. These may be installed in thousands of locations across a large geography. With “smart dispensers” the investment is high. Having any particular dispenser “down” due to maintenance issues can significantly impact ROI. On the flip side, dedicating people for checking and maintaining these dispensers could also significantly impact ROI.

This solution uses the Athena framework and Flume to incorporate data sensor agents for each dispenser, and then to efficiently collect, aggregate, and move all the dispenser log data into a Hadoop cluster. Then HBase is used as a column-oriented database store modeled after Google’ Bigtable: A Distributed Storage System for Structured Data. Lucene provides a full-featured text search engine library written entirely in Java that is suitable for nearly any application that requires full-text search.

The ultimate goal is to provide both ipad and iphone applications which enable remote maintenance personnel to quickly identify and respond to dispenser issues, in real-time. The logs for each machine can be browsed with similar functionality as Splunk in the datacenter in terms of being able to search and analyze logs seamlessly. The final outcome….a low-cost solution to enabling a cost-effective and real-time maintenance workforce.

This solutions could also be deployed in Amazon Web Services (Elastic MapReduce), making it easy to support geographical deployments worldwide.

Do you have any Big Data “sensor” applications?

Jim Kaskade

Jim Kaskade is a serial entrepreneur & enterprise software executive of over 36 years. He is the CEO of Conversica, a leader in Augmented Workforce solutions that help clients attract, acquire, and grow end-customers. He most recently successfully exited a PE-backed SaaS company, Janrain, in the digital identity security space. Prior to identity, he led a digital application business of over 7,000 people ($1B). Prior to that he led a big data & analytics business of over 1,000 ($250M). He was the CEO of a Big Data Cloud company ($50M); was an EIR at PARC (the Bell Labs of Silicon Valley) which resulted in a spinout of an AML AI company; led two separate private cloud software startups; founded of one of the most advanced digital video SaaS companies delivering online and wireless solutions to over 10,000 enterprises; and was involved with three semiconductor startups (two of which he founded, one of which he sold). He started his career engineering massively parallel processing datacenter applications. Jim has an Electrical and Computer Science Engineering degree from University of California, Santa Barbara, with an emphasis in semiconductor design and computer science; and an MBA from the University of San Diego with an emphasis in entrepreneurship and finance.