Rules Engine vs Approval-Gated AI: When If-Then Logic Fails

Q: Can a rules engine handle fraud detection alone?

Yes, if fraud patterns are stable and simple. If fraud is 95% stolen cards + 5% friendly fraud, rules work: block mismatched zip codes, block high-value first purchases from new accounts. If fraud is 40% novel patterns (account takeover, velocity abuse, synthetic identity), rules fail. Fraud evolves; rules don't adapt automatically. Hybrid approach is safer: rules catch obvious cases, AI catches novel ones.

Q: How many decisions can one reviewer handle per hour?

3 - 5 decisions per hour if each requires reading order details, customer history, and payment info. 10 - 15 per hour if decisions are simple (approve/deny with minimal context). If queue depth exceeds 2 hours of work per reviewer, staffing is insufficient. A team of 2 can handle ~40 - 60 decisions per day; a team of 5 can handle ~200 - 300.

Q: What's the typical approval rate in a hybrid system?

Rules engine approves 85% - 95% of orders (low-risk, clear pattern). AI flags 5% - 15% for review. Humans approve 70% - 90% of flagged orders (context matters). Net approval rate: 92% - 98%. If net approval rate is 99%, system is under-catching fraud.

Rules Engine: Structure and Failure Modes

A rules engine operates on explicit conditions and fixed outcomes. If order value exceeds $500 AND customer has 3+ chargebacks, block the transaction. If cart abandonment occurs after 2 minutes AND user is returning, send SMS. These systems are fast, auditable, and repeatable.

Rules engines fail when conditions don't cover reality. A customer with a legitimate reason for a chargeback (friendly fraud, processor error) gets blocked. A bot abandons carts at 2 minutes; a human does at 45 minutes. New fraud patterns emerge that the ruleset doesn't address. The system becomes either too permissive (rules relax, fraud increases) or too restrictive (false positives spike, revenue drops).

Maintenance cost scales with complexity. Each new rule adds decision tree branches. Interactions between rules create unexpected outcomes. A rule that works in Q1 breaks in Q4 when customer behavior shifts seasonally. Rules engines require constant monitoring and adjustment - they are not set-and-forget.

Approval-Gated AI: Judgment Under Uncertainty

Approval-gated AI assigns a risk score or recommendation, then routes decisions to a human reviewer when confidence is below a threshold or risk is above a ceiling. An order scores 0.67 fraud probability - human reviews. A refund request from a 2-week-old account - human reviews. A $3,000 first purchase from a new IP - human reviews.

This model handles novelty. A customer's behavior doesn't match historical patterns, but a human can see context: they're calling from a business address, their email domain matches a Fortune 500 company, their IP is stable. The AI flagged it; the human approved it. No false positive, no lost revenue.

The cost is latency and human labor. Every flagged decision waits for review. If 8% of orders hit the approval queue and each takes 3 minutes to review, a team of 2 can handle ~480 orders per day. At 10,000 daily orders, the queue backs up. Approval-gated systems require staffing proportional to exception rate, and exceptions are often high-value or high-risk - the decisions that matter most.

Decision Matrix: When to Use Each

Choose a rules engine when false positives are cheap and exceptions are rare. Blocking a low-value order from a new customer costs $15 in lost revenue and minimal reputation damage. If 0.5% of orders trigger exceptions, the system is stable. Rules are simple: order value, customer age, payment method, geographic mismatch. The ruleset is reviewed quarterly and rarely changes.

Rules engine: High-volume, low-stakes decisions (fraud screening for $20 - $100 orders)
Rules engine: Deterministic inputs (payment method, account age, geographic data)
Rules engine: Stable patterns (repeat customer, known fraud signals, seasonal trends)
Approval-gated AI: High-stakes decisions (refunds, chargebacks, account suspension)
Approval-gated AI: Ambiguous inputs (customer intent, context, novel patterns)
Approval-gated AI: Low exception rate (2% - 5% of decisions require review)

Hybrid Approach: Layered Decision Logic

Most DTC operations use both. A rules engine handles 95% of orders: approve low-risk, block obvious fraud. The remaining 5% - ambiguous cases, high-value orders, edge cases - route to approval-gated AI. The AI scores them; a human reviews scores above 0.5 or below 0.3. This reduces human review load to 2% - 3% of orders while catching exceptions.

The sequence matters. Rules first (fast, cheap), then AI (accurate, contextual). If a rule catches it, no AI call needed. If a rule can't decide, AI scores it. If AI is uncertain, a human reviews. This is a funnel: each layer eliminates decisions the next layer doesn't need to see.

Operational Metrics and Thresholds

Track false positive rate (legitimate transactions blocked) and false negative rate (fraud approved). A rules engine might achieve 2% false positives and 5% false negatives. Approval-gated AI might achieve 0.5% false positives and 2% false negatives, but requires 8 hours of human review per 1,000 orders.

Calculate the cost of each outcome. False positive: lost revenue + customer churn risk. False negative: chargeback fee ($15 - $100) + fraud loss + operational overhead. If a false positive costs $50 and a false negative costs $200, the approval-gated AI is worth the labor cost if it reduces false negatives by >25%.

Set approval queue SLA. If a decision waits >4 hours for review, it's stale - customer context changes, fraud patterns evolve. If queue depth exceeds 2 hours of work, staffing is insufficient. Monitor queue depth daily; scale reviewers when depth trends upward.

Common Pitfalls

Over-reliance on rules. Teams build 50+ rules, each with exceptions. The system becomes unmaintainable. A rule that worked for 2 years suddenly breaks because a supplier changed payment processing. Rules should be <20 per decision type; beyond that, switch to AI.

Approval queues without SLA. Reviewers become a bottleneck. Decisions pile up. Customers wait for refunds. Fraud slips through because reviewers are overwhelmed. Set SLA first (e.g., approve/deny within 2 hours), then staff to meet it.

AI without explainability. A model scores an order 0.89 fraud probability. Why? The reviewer can't see the reasoning. They either rubber-stamp the AI or second-guess it. Use models that output feature importance or decision rules. A human needs to understand why the system flagged a decision.

Implementation Checklist

Before deploying either system, define the decision: What are we approving or blocking? What are the inputs? What's the cost of error? Document the baseline (current approval rate, current false positive rate). Measure against it.

Rules engine: List all conditions (order value, customer tenure, payment method, geographic mismatch). Test each rule against historical data. Measure false positive and false negative rates. Set review cadence (weekly, monthly, quarterly).
Approval-gated AI: Define approval threshold (e.g., flag if score >0.7 or <0.3). Set SLA for review (2 hours, 4 hours, 24 hours). Assign reviewers. Track queue depth, approval rate, and override rate (how often reviewers disagree with AI).
Hybrid: Route rules first. Measure pass-through rate (% approved by rules). Route remainder to AI. Measure AI flag rate. Calculate total review load. Adjust thresholds to hit target SLA.

Questions

FAQ

Can a rules engine handle fraud detection alone?

Yes, if fraud patterns are stable and simple. If fraud is 95% stolen cards + 5% friendly fraud, rules work: block mismatched zip codes, block high-value first purchases from new accounts. If fraud is 40% novel patterns (account takeover, velocity abuse, synthetic identity), rules fail. Fraud evolves; rules don't adapt automatically. Hybrid approach is safer: rules catch obvious cases, AI catches novel ones.

How many decisions can one reviewer handle per hour?

3 - 5 decisions per hour if each requires reading order details, customer history, and payment info. 10 - 15 per hour if decisions are simple (approve/deny with minimal context). If queue depth exceeds 2 hours of work per reviewer, staffing is insufficient. A team of 2 can handle ~40 - 60 decisions per day; a team of 5 can handle ~200 - 300.

What's the typical approval rate in a hybrid system?

Rules engine approves 85% - 95% of orders (low-risk, clear pattern). AI flags 5% - 15% for review. Humans approve 70% - 90% of flagged orders (context matters). Net approval rate: 92% - 98%. If net approval rate is <90%, rules are too strict or AI is too conservative. If >99%, system is under-catching fraud.

When should we switch from rules to AI?

When false positive rate exceeds 3% or ruleset exceeds 20 conditions. When new fraud patterns emerge faster than rules can be updated (monthly or more). When approval queue SLA is consistently missed. When cost of maintaining rules (engineering time, QA, monitoring) exceeds cost of AI + human review. Measure the switch: run AI in shadow mode for 2 weeks, compare outcomes to rules, calculate ROI.

Want this on your account?

Thirty minutes. Bring the number that keeps you up.

Book a demo