Reliable intrusion detection under dataset shift
Most ML IDS pipelines are optimized for benchmark scores, not operational reliability. The exact problem DriftGuard solves is performance collapse when traffic patterns, feature distributions, and attack mix change after deployment.
What Typical Research Reports
High in-distribution accuracy when training and testing are from the same source. This number rarely reflects production behavior.
What Security Teams Actually Face
Model drift, noisy alerts, and missed attacks in new networks. Analysts lose time on false positives while true incidents compete for attention.
How the complete project is organized
DriftGuard is structured as a reproducible research-to-deployment pipeline: datasets and preprocessing artifacts feed model training, trained checkpoints feed inference and risk scoring, and generated outputs drive dashboard and explainability workflows.
Core code modules: src/data_loader.py, src/features.py, src/train_iforest.py, src/train_autoencoder.py, src/train_lstm.py, src/inference.py, and src/risk_scoring.py.
Model artifacts: Trained checkpoints are persisted under models/ (Isolation Forest .pkl, Autoencoder and LSTM .keras) and reused for inference without retraining.
Experiment layer: Notebooks in notebooks/ and research/experiments/ cover feature engineering, model evaluation, SHAP analysis, false-positive analysis, and cross-dataset validation.
Operations layer: dashboard/app.py consumes final risk outputs, severity labels, and ground truth to provide SOC-facing KPIs, alert distribution, and top-risk event views.
Three detectors, one decision score
Each engine captures a different failure mode in network behavior. Their outputs are scaled to the same range and combined, so no single model dominates on its own.
Exactly how the pipeline runs
Input is tabular flow data. Output is per-flow prediction, risk score, and optional explanation artifact for analyst review.
1) Ingest and standardize features
Raw CSV traffic logs are transformed into a consistent 77-feature matrix. Missing/infinite values are repaired, scaling is applied, and arrays are persisted as .npy for reproducible training and fast inference.
2) Learn normal behavior from benign traffic
All models are trained only on benign CICIDS2017 records. This makes the system attack-type agnostic: it detects deviation from normality, including novel attacks not seen as labels during training.
- Isolation Forest: 100 trees, contamination 0.1
- Autoencoder: 50 epochs, batch 256, Adam / MSE
- LSTM: 30 epochs, sequence length 10, BiLSTM layers
3) Fuse model outputs into one risk score
Each model emits an anomaly score in a different numeric range. Scores are normalized to [0,1] and combined by weighted averaging:
risk = 0.3×IF + 0.4×AE + 0.3×LSTM
4) Explain high-risk alerts for analysts
For selected alerts, SHAP generates feature-attribution plots that show why the score is high. This converts model output into actionable context (which ports, flags, rates, or timing signals drove the alert).
5) Validate transferability before deployment
Generalization is tested on UNSW-NB15 (49 features padded to 77) and UGR16 (135 features aligned to 77). The same training scaler is reused to avoid leakage and simulate realistic deployment transfer.
- KS test per feature (p < 0.05 = significant shift)
- PSI per feature (> 0.2 = substantial shift)
Why three datasets are necessary
The system is intentionally evaluated across heterogeneous sources to answer one question: does detection survive environmental change, not just benchmark replay?
Results tied to the original problem
Generalization gain: The ensemble maintained materially better cross-dataset detection quality than any single model, reducing the failure risk of model-specific blind spots.
Feature stability insight: Protocol-level features (ports/flags) stayed more stable across environments than heavily derived throughput features, informing safer feature selection.
Shift is quantifiable: PSI and KS statistics identified distribution changes before deployment, enabling pre-launch risk assessment instead of reactive firefighting.
Operational relevance: Validation across CICIDS2017 → UNSW-NB15 → UGR16 demonstrates that the pipeline is designed for transfer, not only for one closed benchmark.
| Metric | Value |
|---|---|
| Inference Speed | ~2,380 samples / sec |
| Latency | 420 ms / 1,000 samples |
| Total Model Size | ~3 MB |
| Training Time | ~65 min on 2M+ samples |
| RAM (Training) | ~4 GB |
Stack
Core ML
Visualization & Dev
Dashboard Integration
The Streamlit dashboard (in dashboard/app.py) loads final outputs, computes severity bands from risk scores, and presents SOC-style KPIs:
- Total events processed
- High-risk alert count
- Detected attack count using available labels
- Severity distribution and top-risk event table
Run inference and consume outputs
Inference returns anomaly predictions and continuous risk scores. In operations, map high-risk outputs to analyst queues and attach SHAP explanations for investigation context.
from src.inference import AnomalyDetector
detector = AnomalyDetector()
detector.load_model('iforest', 'models/iforest_model.pkl', 'sklearn')
detector.load_model('autoencoder', 'models/autoencoder.keras', 'keras')
detector.load_model('lstm', 'models/lstm_autoencoder.keras', 'keras')
predictions = detector.predict(X_test)
risk_scores = detector.get_risk_scores(X_test)Use this sequence to reproduce the complete project workflow from setup to validation:
# 1) Environment setup (Windows)
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
# 2) Train all three models
python src/train_iforest.py
python src/train_autoencoder.py
python src/train_lstm.py
# 3) Evaluate / validate in notebooks
jupyter notebook notebooks/06_model_evaluation.ipynb
jupyter notebook notebooks/09_cross_dataset_validation.ipynb
# 4) Explainability analysis
jupyter notebook notebooks/07_explainability_shap.ipynb
# 5) Launch SOC dashboard
streamlit run dashboard/app.pymodels/, preprocessed arrays and scores in data/, and experiment figures/results under research/results/ and reports/figures/.
Where this system can still fail
DriftGuard improves cross-environment robustness, but it is still an anomaly-based IDS and has known constraints that matter for deployment planning:
Feature alignment tradeoff: Mapping 49 or 135 native features to the shared 77-feature space can discard source-specific signals.
Concept drift over time: Traffic baselines evolve, so periodic recalibration or retraining is still required for long-running deployments.
Threshold sensitivity: Severity cutoffs (0.4 / 0.7) balance recall and analyst workload; they should be tuned to each SOC risk appetite.
Current mode is mostly batch: Real-time packet/flow ingestion is planned (Zeek integration) and not yet the default production path.
What comes next
🔴 Zeek Real-Time Inference
Extend current batch pipeline into continuous packet-to-flow monitoring for SOC operations.
- Zeek integration for live packet feature extraction
- Streaming prediction pipeline, sub-second latency
- SIEM alert forwarding
- Live anomaly monitoring dashboard
🗺️ MITRE ATT&CK Mapping
Translate anomalous behavior into standardized ATT&CK tactics and techniques for response alignment.
- Map anomalies to techniques (e.g. T1046 Network Scanning)
- Tactic-level classification
- ATT&CK Navigator integration
- Automated playbook selection