Why CEPs are (so) important
Complex Event Processing is about predicting and inferring events or patterns quickly, with the use of different continuously produced data sources.
There are many use cases for CEP technologies, typically those that require figuring out that some circumstances are happening and require immediate reaction. To mention a few:
– Finance: fraud detection, risk, etc.
– Smart cities, IoT, sensors, etc.
– Social media analytics
– Clickstream analytics
– Business Process Management (BPM) and Business Activity Monitor (BAM)
– Network and apps monitoring
In general, where there are operations, transactions or monitoring, a CEP is a key advantage. CEP solutions need to deal with a high throughput, have a very low latency and perform complex techniques :
– Event-pattern detection
– Event abstraction
– Event filtering
– Event aggregation and transformation
– Modeling event hierarchies
– Detecting relationships between events
– Abstracting event-driven processes
CEPs after Big Data
With the arrival of Big Data, CEP systems have evolved and let real-time insights emerge. Now it is possible to cope with extra (and massive) data sources that in many cases are ingested in real-time too. That is also, coping with continuous data streams.
In recent times Spark, with Spark Streaming, has been used for complex event processing (see ). But Spark has some limitations due to the micro batching processing schema .
Analyticmate is going one step further and integrating Apache Flink (with FlinkCEP library) to deal with such limitations (option A in the figure). Apache Flink offers a natural approach to deal with streaming dataflows: high throughput and low latency. In some cases it is needed a latency below 50 ms. So Spark Streaming is not an option  when real-time is needed (option B in the figure).
 Wikipedia, "Complex Event Processing," [Online]. Available: https://en.wikipedia.org/wiki/Complex_event_processing. [Accessed 16 06 2016].  M. Ganta, «Cloudera Engineering Blog,» 5 10 2015. [En línea]. Available: http://blog.cloudera.com/blog/2015/11/how-to-build-a-complex-event-processing-app-on-apache-spark-and-drools/. [Último acceso: 21 06 2016].  Apache Flink, «Flink,» [En línea]. Available: https://flink.apache.org/. [Último acceso: 21 06 2016].  Apache Spark, "Spark," [Online]. Available: http://spark.apache.org/docs/latest/configuration.html#spark-streaming. [Accessed 21 06 2016].