I recently attended the SAMENA Council Leaders Summit in Dubai to discuss the topic of Micro-Trends Analytics: A Secret to Successful Digital Transformation for Telecom Operators
The PDF copy of SAMENA Trends article is attached here and also a link to the LinkedIn blog post
A great article by SAHAR TORKAMANI and VOLKER LOHWEG on the topic of motif discovery. Here is the abstract:
“Last decades witness a huge growth in medical applications, genetic analysis, and in performance of manufacturing technologies and automatised production systems. A challenging task is to identify and diagnose the behavior of such systems, which aim to produce a product with desired quality. In order to control the state of the systems, various information is gathered from different types of sensors (optical, acoustic, chemical, electric, and thermal). Time series data are a set of real‐valued variables obtained chronologically. Data mining and machine learning help derive meaningful knowledge from time series. Such tasks include clustering, classification, anomaly detection and motif discovery. Motif discovery attempts to find meaningful, new, and unknown knowledge from data. Detection of motifs in a time series is beneficial for, e.g., discovery of rules or specific events in a signal. Motifs provide useful information for the user in order to model or analyze the data. Motif discovery is applied to various areas as telecommunication, medicine, web, motion‐capture, and sensor networks. This contribution provides a review of the existing publications in time series motif discovery along with advantages and disadvantages of existing approaches. Moreover, the research issues and missing points in this field are highlighted. The main objective of this focus article is to serve as a glossary for researchers in this field. WIREs Data Mining Knowl Discov 2017, 7:e1199. doi: 10.1002/widm.1199″
O’Reilly’s post “What lies ahead for data in 2018” cleary states the importance of time-series analytics.
Here is the full quote about the #1 trend:
1. New tools will make graphs and time series easier, leading to new use cases.
|Graphs and time series have been a crucial part of the explosion in big data. 2018 will see the emergence of a new generation of tools for storing and analyzing graphs and time series at large scale. These new analytic and visualization tools will help product groups devise new offerings, especially for use cases in security and fraud detection|
Trendalyze is generally available on the Microsoft Azure cloud platform distributed as a Virtual Machine Image (Ubuntu) and Docker container for Azure Container Services.
Integration has been provided for a number of the Azure services including:
- Azure BLOB Storage using WASB
- Azure Data Lake Store
- Azure HD Insights
- Azure IoT Hub
- Azure SQL Data Warehouse
- Power BI
The support for Azure is in addition to the support for Amazon Web Services (AWS) platform where the implementation supports Ubuntu Amazon Machine Image (AMI) distribution and Docker container for EC2 Container Services (ECS).
This is a nice blog post about “Three Ways Pattern Analytics Will Grow Your Business”. The article explains in a straightforward way what pattern analytics is and why it can be difficult to implement. Because the data is typically big and contains many variables, you will need some tools that automate the pattern discovery. By the way this is what Trendalyze provides.
According to the blog, the first way to grow the business is trough sensor analytics. The analysis of granular data will help you “find anomalies, commonalities and trends that reveal insights that otherwise would remain unnoticed”. You can use these insights to optimize operations and manage maintenance. The second way is to use pattern analysis to fortify your company against cyber attacks. Since all intrusion occurs in a sequential manner, pattern analytics allow you to catch any new ways of infiltrating your systems. Lastly, but perhaps most importantly, pattern analytics can help you grow sales. As the authors point out: “Pattern analytics on customer and sales data can therefore indicate market trends, customer interests, latent needs and reveal future sales trends. In addition, pattern analytics can show the top selling items based on geography and interest and news happening around the world.”
We will be interested to hear your stories about how pattern analytics improved your business.
Trendalyze and UCL get a C2N award to develop advanced analytic solutions for computer assisted surgery
MedCity @MedCityHQ is a collaboration between the Mayor of London and London’s three Academic Health Science Centres – Imperial College Academic Health Science Centre, King’s Health Partners, and UCL Partners. Launched in April 2014 to promote and grow the world-leading life sciences cluster of England’s greater south east, it is promoting life sciences investment, entrepreneurship and industry in the region.
Collaborate to Innovate @C2N_ERDF is an exciting MedCity project in the broad life sciences domain which launched in 2016, part-funded by the European Regional Development Fund. The project aims to promote knowledge transfer and commercialisation of innovations. C2N targets London based SMEs that have ambitions to embark on innovative projects in the applied research/clinical domain, or wish to bring new products and services to market.
Trendalyze and UCL have been successful in a C2N award to develop analytics solutions based on advanced data science and deep learning for computer assisted surgery.
Image-guided surgery (IGS) has the potential to enhance the localisation and navigation capabilities of the surgeon during Minimal Access Surgery (MAS). To address the significant challenge of deploying such a system in clinical practice it is important to understand its use in training scenarios with two key research goals:
- To test the computing algorithms for IGS in a manageable training environment and understand the workflow needs and trend changes in motion patterns for robotic radical prostatectomy;
- To understand how guidance information influences surgical performance and if it can potentially be used for developing advanced instrument-tissue motion modelling and skill evaluation.
This project will have potential for several potential exploitation opportunities in the IGS and broader computer assisted surgery (CAS) markets:
- Added value to surgical robotics companies who can use the analytical platform and research to guide their system by Trendalyze;
- Added value to medical device manufacturers by demonstrating skills changes with new devices in a new analytics product by Trendalyze, designed for bespoke analysis and deep learning on instrument motion data gathered from different sources (robots, optical trackers, sensors);
- New products can come from IGS work (and the broader CAS) as well as from the time series analysis and deep learning that exploit the workflow information to highlight parameters automatically at the right point of care. Such new analytics products can accelerate clinical innovation time to market and improve quality.
We will be speaking at IoT conference ThingsExpo in New York City. June 6-8.
Analytics for Motif Discovery and Deep Learning in Time Series Data Generated by IoT
IoT generates lots of temporal data. But how do you unlock its value? You need to discover patterns that are repeatable in vast quantities of data, understand their meaning, and implement scalable monitoring across multiple data streams in order to monetize the discoveries and insights. Motif discovery and deep learning platforms are emerging to visualize sensor data, to search for patterns and to build application that can monitor real time streams efficiently.
In his session at @ThingsExpo, Dave Watson, CTO and Co-Founder of Trendalyze, will discuss real world IoT projects from UK environmental monitoring using Mosquitto, Node-RED, Kafka, Spark, MLlib and R.
Dave Watson is CTO and Co-Founder of Trendalyze and works on developing the database search and analytics platform for various IoT projects. He holds number of UK and US patents and has led engineering in database middleware, OLAP and time series databases in the past.
Speaking Experience: Speaker at number industry events including NoSQL Now, MongoDB Europe, Gartner, IDC, TDWI, VLDB, DB/EXPO, Google, Bearingpoint, Fujitsu and IBM/Informix user groups. Holds first class honours degree in Computer Systems Engineering from University of Bristol. Holds patents in UK and USA.
Many algorithms have been developed in the academic world for Motif/time series analysis such as the popular dynamic time warping (DTW). However, most such algorithms have not been implemented to scale and as a result can not provide interactive low-latency response times needed for web applications when dealing with large data sets.
Parallel processing frameworks (such as map reduce) can improve the performance but require large (and expensive) clusters of servers to achieve significant improvements in response times. The expectation of users is often set by using OLAP tools or internet search engines where sub-second response times are the norm.
At Trendalyze Decisions, we have implemented our own patent-pending techniques that can accomplish low latency interactive response times when processing large number of time series for motif search, match and ranking.
This is illustrated below using data from NHS England comparing Trendalyze on a single node with Map Reduce on five nodes: