Rumman Firdos Research Identifies a Critical Vulnerability Shift in AI-Hardened Network Intrusion Detection Systems

New research introduces Vulnerability Shift, a framework for identifying uneven AI robustness across attack classes and a multi-epsilon approach to mitigate it.

BERLIN, GERMANY, September 8, 2026 /EINPresswire.com/ — New research by artificial intelligence and cybersecurity researcher Rumman Firdos examines a potential weakness in how adversarially hardened AI-based Network Intrusion Detection Systems (NIDS) are evaluated and introduces a framework for identifying and mitigating what the researchers describe as Vulnerability Shift.

The research, titled “Mitigating Vulnerability Shifts in Adversarially Hardened Network Intrusion Detection Systems via Multi-Epsilon Curriculum Learning,” investigates whether improving the overall adversarial robustness of an AI security model necessarily makes every category of network traffic and attack more robust.

The findings suggest that it may not.

Identifying an overlooked robustness problem

AI and machine learning are increasingly being incorporated into cybersecurity systems for intrusion detection, threat classification and automated security analysis. As these systems become more capable, researchers and security engineers are also developing adversarial training techniques intended to make models more resistant to attacks designed to manipulate their predictions.

However, conventional evaluations often emphasize aggregate metrics such as overall accuracy, macro-F1 or overall robustness.

The research identifies an important limitation in this approach: an AI security model can become more robust overall while robustness is distributed unevenly across individual attack classes.

The researchers refer to this phenomenon as Vulnerability Shift.

Instead of treating adversarial robustness as a single property of a model, the study examines how robustness changes at the class level. This distinction is particularly important in cybersecurity because an attacker does not necessarily need to compromise an entire AI system. A weakness concentrated in a specific attack category may provide a more targeted opportunity for evasion.

The research therefore proposes that adversarial robustness evaluations should examine not only whether a model becomes harder to attack, but also which classes become more or less vulnerable as a result of adversarial training.

Introducing Defense Budget Exhaustion and a Sensitivity Index

To explain the observed behavior, the research proposes the Defense Budget Exhaustion (DBE) hypothesis.

DBE describes a possible mechanism through which a model’s limited capacity to learn robust decision boundaries can result in robustness being allocated unevenly across different classes during adversarial training.

The paper presents DBE as a proposed hypothesis and mechanism for further investigation rather than as an established universal law.

The researchers also introduce a gradient-based Sensitivity Index (Ψ) to quantify class-level susceptibility to adversarial perturbations.

The reported analysis found a strong relationship between class-level sensitivity and F1-score degradation, with a Spearman correlation of ρ = 0.8833 and p = 0.00159.

This result supports the study’s investigation into whether classes exhibiting greater adversarial sensitivity are more likely to experience substantial performance degradation.

Testing adversarial attacks across datasets and architectures

The research evaluates three adversarial attack methods:

Fast Gradient Sign Method (FGSM)
Projected Gradient Descent (PGD)
Auto-PGD (APGD)

Experiments were conducted using the UNSW-NB15 and CIC-IDS2017 network intrusion datasets and evaluated across Multi-Layer Perceptron (MLP) and Long Short-Term Memory (LSTM) architectures.

The reported UNSW-NB15 results demonstrate the scale of the class-level differences.

For the Reconnaissance class, the research reports F1 degradation of 89.32% under FGSM, 96.07% under PGD and 96.75% under APGD.

By comparison, the reported F1 degradation for the Normal class was approximately 0.03–0.04%.

The contrast illustrates the central problem investigated by the study: adversarial effects can vary dramatically between classes even within the same AI-based intrusion detection system.

A proposed mitigation through Multi-Epsilon Adversarial Training

To address Vulnerability Shift, the research proposes Multi-Epsilon Adversarial Training (MEAT), a curriculum-based adversarial training approach that progressively exposes models to multiple perturbation strengths.

Rather than relying on a single adversarial perturbation budget, the approach uses progressively varied epsilon levels with the objective of encouraging more balanced robustness across different threat categories.

In the reported evaluation, MEAT produced a +0.60 F1 improvement for the Reconnaissance class under PGD.

The approach provides a potential direction for developers and researchers seeking to evaluate and mitigate class-specific weaknesses in adversarially trained cybersecurity models.

Implications for AI cybersecurity development

The research has broader implications for organizations developing AI-powered cybersecurity technologies.

As companies increasingly build machine-learning-based systems for intrusion detection, malware analysis, threat detection and other security applications, evaluating a model solely through aggregate performance can leave important questions unanswered.

A security AI system may achieve a strong overall robustness score while containing a substantially weaker class-level decision boundary.

The research does not claim that existing commercial AI cybersecurity products universally exhibit Vulnerability Shift. Instead, it identifies a research and evaluation gap that can be investigated when developing and validating adversarially hardened security systems.

The study suggests that future AI security evaluations should consider class-level robustness, adversarial sensitivity and robustness redistribution alongside conventional aggregate metrics.

Research contribution

The work contributes a framework for investigating adversarial robustness in network intrusion detection through four central concepts: Vulnerability Shift, the Defense Budget Exhaustion hypothesis, the Sensitivity Index (Ψ) and Multi-Epsilon Adversarial Training (MEAT).

Together, these concepts provide a methodology for examining where adversarial robustness is gained, where it may be lost, and how training strategies can potentially reduce uneven robustness across attack classes.

“AI security cannot be evaluated only by asking whether a model becomes more robust overall. We also need to understand where that robustness is being gained, where it is being lost, and why,” said Rumman Firdos. “Vulnerability Shift is an attempt to make that imbalance measurable and visible.”

The research contributes to ongoing work in adversarial machine learning, AI security, network intrusion detection, robust machine learning and cybersecurity engineering.

Research: Mitigating Vulnerability Shifts in Adversarially Hardened Network Intrusion Detection Systems via Multi-Epsilon Curriculum Learning

Researcher: Rumman Firdos

Website: RummanFirdos.com

Media Contact:
Rumman Firdos
AI & Cybersecurity Researcher
firdos@scalewidth.com

Rumman Firdos
Scalewidth
firdos@scalewidth.com
Visit us on social media:
LinkedIn

Media gallery