📄️ Eyer Algorithm Overview
Our machine-learning pipeline comprises a series of algorithms that contribute to building an alert.
📄️ Univariate anomaly detection
This part of the pipeline detects anomalies on a single time series. It consists of a training part (baseline creation) that periodically (every 24 hours) determines the normal behaviour of the time series and a detection (live anomaly detection) part that close to real-time determines if behaviour is different from the one observed in the past (an anomaly) is ongoing is.
📄️ Correlation and grouping of time series
To know how different resources work together we calculate correlation and group the resources based on correlations. This method is meant to be complementary and not substitutive to informed ways of grouping, like user-defined or transaction tracing.
📄️ ML training
From the moment data is connected through agents, it will take at least 7 days before Eyer will provide anomaly alerts to the end user. Onboarding is happening on a schedule, every week, the night between Saturday and Sunday. New environments will be onboarded on the first Sunday after they have been active for at least 6 days, and new metrics will be onboarded on the first Sunday after they have been producing enough data (the amount of data can vary depending on the unique behaviour of the metric, but an absolute minimum is 7 data points on average in the last 7 days). For new metrics added to a pre-existing environment, the first week of anomaly detection might produce false positives.
📄️ Alerts
Different anomalies on a single time series are grouped in an alert containing several nodes, to give a bigger context to each anomaly and also to reduce the number of alerts sent to the user. Time series that are in the same node are already considered related, so they will always be alerted together. To capture inter-node relations we are using the groups created by using correlations among nodes. Time series which belong to nodes in the same group are alerted together.
📄️ Boomi data collector structure
The data coming from the Influx Telegraf monitoring Atoms are structured into nodes and metrics.
📄️ Adjust the Telegraf sampling interval
Only do this if strictly necessary! Reducing the number of samples will have an impact on the correctness of the data transmitted to the Eyer anomaly detection.
📄️ Security - Boomi data collector
All the components used to collect Atom performance metrics are based on open source code.
📄️ F1 Performance tests anomaly detection algorithm
Test August 2024 - Cyclical data
📄️ Note on F1 performance testing of the core algorithm of Eyer
Abstract