Rules-based vs. advanced analytics – Do you have to choose?
Artificial intelligence. Advanced, predictive analytics. Machine learning. These terms are heard with increasing frequency in the compliance space as solution providers seek new ways to evolve technology offerings to get ahead of compliance challenges. But does this mean that rules-based testing should be completely abandoned in favor of newer technology, or become their poor second cousin? In a nutshell: no. In real-life implementations, this is simply not practical. Here are a few reasons why.
Let’s use a fairly simple use case to explain. Say you’re implementing a fraud analytics solution for health insurance claims to examine the relationship between the policyholder and their dependents. A rules-based analytic may flag dependents classified as spouses that are outside an age range of 18–75 years old, or children over 25. This seems reasonable and will have relatively few false-positives.
Advanced analytics, on the other hand, may use an anomaly detection model to flag outliers without specifically writing a rule. This is likely to detect spouses outside the typical age range that policyholders in the data set predominantly have spouses in. Therefore, if the similar policyholders have spouses aged 19–55 then a 70-year-old spouse may be flagged as an outlier as the model learns from the population and segmentation of its policyholders. In some situations, the anomaly detection model will identify fraudulent cases that would go undetected by rules-based analytics.
Advanced analytics models have many other upsides. You can use other features in the data set, such as the age of the policyholder (or the age difference) in the model. That may provide more accurate outliers and will be reflected in the outlier score. For example, the larger the age difference the more likely the policy is an outlier.
Sometimes the best way to explain the advantages is to say that a good model can tell you what you do not know. There is tremendous value in that. This seems great, so why should you hold on to rules-based testing as well?
What happens if you don’t have enough data to train a model or most of the transactions for a customer segment are in fact breaches? The model won’t accurately detect or predict non-compliant behavior. Rules-based analytics are also still useful in identifying non-compliant activity when you are clear what the compliance requirements are and they are not driven by behavior.
So what’s the best approach? As depicted below, it may be best to fuse the two approaches using a scoring engine. Then, based on the score, a decision is made to block the activity (possibly in real-time), investigate or continue monitoring.
This allows both approaches to work together, which may be better than just using one or the other. Applying rules and advanced models together ensures that false-positives will be much lower. If you have a workflow and case management platform, then it is a good idea to feed the results of investigations back into the learning models for continuous improvement.
There are varying views on this topic but I wanted to share what we’ve found that actually works in real implementations. In a straight shootout, behavioral and predictive analytics are sexier and promise better results but in the compliance space, simple rules are still relevant.
To learn more about the advanced analytics, workflows and case management offered in the CaseWare AML Compliance solution, watch our video now:
About Andrew Simpson:
Andrew Simpson has close to two decades of experience in the information systems audit and security business; specifically data analytics, interrogation and forensics. He is a regular contributor to various auditing conferences and is acknowledged as an expert on continuous controls monitoring and revenue assurance.
Connect: Andrew Simpson