Financial entities shall, after collecting and analysing all relevant information, submit a final report to the competent authority with details on the root cause and measures adopted to prevent recurrence.
Final report submitted to competent authorities within one month of a major ICT-related incident, containing complete root cause analysis, total impact assessment, and preventive measures as required by DORA Articles 19-20.
incident-final-reportGenerated example artifact using the default institution profile (COMMON availability, synthetic data only).
MAJOR ICT INCIDENT — FINAL REPORT
Nordvik Bank AG
Report Reference: RPT-INC-2025-0042-FINAL
Incident ID: INC-2025-0042
Submission Date: 19 February 2025 (within one-month deadline)
Submitted To: Swiss Financial Market Supervisory Authority (FINMA)
Classification: Confidential — Regulatory
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
EXECUTIVE SUMMARY
On 28 January 2025, Nordvik Bank AG experienced a major ICT-related incident
affecting the core banking transaction processing platform. The incident was caused
by database connection pool exhaustion on the Temenos T24 system under peak morning
transaction load, resulting in approximately 35% of payment transactions failing
over a 5-hour 13-minute window.
The incident affected 12,340 clients across Switzerland and Germany, with a total
estimated financial impact of EUR 138,200. No data loss, data breach, or
unauthorised access occurred. All regulatory reporting deadlines were met: initial
notification within 3.5 hours, intermediate report within 52 hours, and this final
report within 22 days.
Root cause analysis identified four contributing factors, and five preventive
measures have been defined with implementation timelines extending to Q2 2025.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
INCIDENT TIMELINE
Date/Time (CET) Event
─────────────────────────────────────────────────────────────────────────────
28 Jan 09:17 First automated alert — Splunk correlation rule
triggered on elevated T24 transaction timeout rate
28 Jan 09:20 Second alert — Dynatrace application monitoring
detected response time degradation on T24 API
28 Jan 09:25 SOC Analyst acknowledged alerts; triage initiated
28 Jan 09:30 Customer service centre briefed with talking points
28 Jan 09:35 Incident classified as High severity; IRT activated
28 Jan 09:45 Customer notifications issued (push + web banner)
28 Jan 10:00 Emergency connection pool increase (500 → 1000)
applied; partial improvement observed
28 Jan 10:15 Root cause identified as connection pool exhaustion
compounded by connection leak in batch processing module
28 Jan 10:45 Overflow traffic rerouted to secondary processing node;
containment achieved — failure rate dropped to < 2%
28 Jan 11:00 Incident reclassified as Major per taxonomy rule MIT-02
28 Jan 12:30 Permanent fix applied: connection leak patched, pool
size set to 1500 on primary node
28 Jan 12:48 Initial notification submitted to FINMA (3h 31m)
28 Jan 13:00 Load testing of primary node completed successfully
28 Jan 14:00 Traffic gradually restored to primary node
28 Jan 14:30 Full service restoration confirmed
28 Jan 18:00 Post-restoration monitoring period ended; no issues
30 Jan 13:15 Intermediate report submitted to FINMA (52h)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
COMPLETE IMPACT ASSESSMENT
Impact Dimension Final Assessment
─────────────────────────────────────────────────────────────────────────────
Clients Affected 12,340 (11,200 retail, 890 corporate, 250 wealth)
Total Duration 5 hours 13 minutes
Geographical Spread 2 EU member states (Switzerland, Germany)
Data Losses 0 records (confirmed by forensic audit)
Critical Services Impacted 1 (Core Banking Transaction Processing)
Total Financial Impact EUR 138,200
Financial Impact Breakdown:
Direct costs (response, vendor support, staffing): EUR 27,700
Lost transaction fee revenue: EUR 42,000
Customer goodwill compensation: EUR 18,500
Estimated reputational impact: EUR 50,000
Transaction Impact:
Total transactions attempted during incident window: 28,450
Transactions failed (timeout): 9,958 (35%)
Transactions successfully retried after resolution: 9,958 (100%)
Transactions incorrectly processed: 0
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
ROOT CAUSE ANALYSIS
Analysis Reference: RCA-INC-2025-0042
Analysis Date: 10 February 2025
Methodology: Combined 5 Whys and Fault Tree Analysis
Lead Analyst: Marcus Weber, Incident Manager
Root Cause:
Database connection pool exhaustion on the Temenos T24 core banking platform,
compounded by a connection leak in the batch payment processing module.
5 Whys Analysis:
Why 1: Why did transactions fail?
→ The T24 application server could not obtain database connections to process
incoming transaction requests.
Why 2: Why were database connections unavailable?
→ The connection pool (max 500) was fully consumed, with no connections being
returned to the pool in a timely manner.
Why 3: Why were connections not being returned?
→ The batch payment processing module (deployed in the December 2024 release)
contained a connection leak — connections opened for batch reconciliation
were not properly closed in the error handling path.
Why 4: Why was the connection leak not detected before production?
→ The batch module's integration tests did not include error-path scenarios
that would trigger the leak. The connection pool monitoring threshold (95%)
was too high to provide early warning.
Why 5: Why was the connection pool sized at 500?
→ The pool size was set during the June 2024 capacity planning exercise based
on Q1 2024 peak volumes. The 15% increase in digital banking adoption in
Q4 2024 was not reflected in a capacity review.
Contributing Factors:
1. Connection leak in batch processing module (December 2024 release)
2. Insufficient connection pool capacity for current transaction volumes
3. Monitoring threshold too high (95%) for timely proactive intervention
4. Capacity planning cycle did not account for Q4 2024 growth
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
RECOVERY ACTIONS
Action Date Completed Performed By
─────────────────────────────────────────────────────────────────────────────
Emergency connection pool increase 28 Jan 2025 IT Infrastructure
Traffic rerouted to secondary node 28 Jan 2025 IT Infrastructure
Connection leak hotfix deployed 28 Jan 2025 Application Support
Primary node restored and verified 28 Jan 2025 IT Infrastructure
Post-restoration monitoring (3h) 28 Jan 2025 SOC
Batch module regression test suite run 29 Jan 2025 QA Team
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
PREVENTIVE MEASURES
ID Measure Owner Target Date
─────────────────────────────────────────────────────────────────────────────
PM-01 Implement automatic connection pool IT Infra 28 Feb 2025
scaling with dynamic upper bound
PM-02 Revise monitoring thresholds: alert at SOC Manager 7 Feb 2025
75% pool utilisation (was 95%)
PM-03 Add connection leak detection tests to QA Lead 14 Feb 2025
all database-accessing modules
PM-04 Move capacity planning to monthly CIO 1 Mar 2025
review cycle (was quarterly)
PM-05 Deploy connection pool health dashboard IT Infra 28 Feb 2025
with trend analysis and forecasting
Implementation Status (as of report date):
PM-01: In progress (vendor engagement completed)
PM-02: Completed (7 February 2025)
PM-03: Completed (12 February 2025)
PM-04: Approved; first monthly review scheduled 1 March 2025
PM-05: In progress (dashboard design completed)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
LESSONS LEARNED
1. Connection pool monitoring thresholds must provide sufficient lead time for
proactive intervention — 95% is too late for a pool that can exhaust in
minutes under peak load.
2. Capacity planning must be responsive to business growth metrics. A quarterly
cycle is insufficient when digital adoption is growing at 15% per quarter.
3. Integration testing for database-accessing modules must include error-path
scenarios that verify connection cleanup.
4. The secondary processing node proved effective as a containment measure,
validating the investment in active-passive redundancy for the T24 platform.
5. Regulatory notification process worked as designed — all deadlines met with
margin. The pre-prepared notification templates reduced preparation time.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
REGULATORY COMPLIANCE ASSESSMENT
Reporting Obligation Deadline Actual Status
─────────────────────────────────────────────────────────────────────────────
Initial notification (4h) 28 Jan 13:17 CET 28 Jan 12:48 CET Met
Intermediate report (72h) 31 Jan 09:17 CET 30 Jan 13:15 CET Met
Final report (1 month) 28 Feb 2025 19 Feb 2025 Met
The Bank's incident management process operated in compliance with DORA
Articles 17–20 throughout this incident. All regulatory reporting deadlines
were met, and the competent authority was kept informed at each stage.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Prepared by: Marcus Weber, Incident Manager
Root Cause Analysis by: Marcus Weber, Incident Manager
Reviewed by: Katrin Halvorsen, CISO
Approved by: Erik Lindqvist, CRO
Board Risk Committee notified: 19 February 2025
Date: 19 February 2025
Example structured facts that Detrixa would extract from this evidence (synthetic data, deterministic seed).
incident_final_report_status — fs-incident-final-report
{
"factId": "b1c2d3e4-f5a6-7890-abcd-200000000006",
"evidenceId": "a0b1c2d3-e4f5-6789-abcd-200000000006",
"evidenceClassId": "incident-final-report",
"factType": "incident_final_report_status",
"data": {
"incident_id": "INC-2025-0042",
"submission_date": "2025-02-19",
"has_root_cause_analysis": true,
"has_preventive_measures": true,
"total_duration_hours": 5.22,
"total_clients_affected": 12340,
"total_financial_impact_eur": 138200.00,
"preventive_measures_count": 5,
"submitted_within_deadline": true
},
"provenance": "deterministic",
"extractorVersion": "dora-test-generator/0.1.0",
"extractedAt": "2025-02-28T09:00:00Z",
"supersededBy": null
}
incident_root_cause_analysis — fs-incident-root-cause-analysis
{
"factId": "b1c2d3e4-f5a6-7890-abcd-200000000007",
"evidenceId": "a0b1c2d3-e4f5-6789-abcd-200000000007",
"evidenceClassId": "incident-root-cause-analysis",
"factType": "incident_root_cause_analysis",
"data": {
"incident_id": "INC-2025-0042",
"analysis_date": "2025-02-10",
"methodology_used": "combined",
"root_cause_identified": true,
"root_cause_category": "technical",
"corrective_actions_count": 5,
"corrective_actions_completed": 4,
"systemic_risk_identified": true
},
"provenance": "deterministic",
"extractorVersion": "dora-test-generator/0.1.0",
"extractedAt": "2025-02-28T09:00:00Z",
"supersededBy": null
}
Detailed root cause analysis document for ICT-related incidents, using structured methodologies (5 Whys, fishbone diagram, fault tree analysis) to identify underlying causes and recommend corrective actions.
incident-root-cause-analysisGenerated example artifact using the default institution profile (COMMON availability, synthetic data only).
ICT INCIDENT ROOT CAUSE ANALYSIS REPORT
Nordvik Bank AG
Analysis Reference: RCA-INC-2025-0042
Incident ID: INC-2025-0042
Analysis Date: 10 February 2025
Classification: Confidential
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1. INCIDENT SUMMARY
Incident Title: Core Banking Platform — Transaction Processing Failure
Incident Date: 28 January 2025
Duration: 5 hours 13 minutes (09:17 – 14:30 CET)
Severity: High (reclassified to Major during incident)
Impact: 12,340 clients affected; EUR 138,200 total financial impact
Data Loss: None
Brief Description:
Intermittent transaction processing failures on the Temenos T24 core banking
platform caused by database connection pool exhaustion. Approximately 35% of
payment transactions through digital channels failed with timeout errors during
the incident window.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
2. ANALYSIS METHODOLOGY
Primary Method: 5 Whys Analysis
Supporting Method: Fault Tree Analysis
Data Sources:
— Application server logs (Temenos T24, 28 Jan 08:00–19:00 CET)
— Database connection pool metrics (Oracle, 28 Jan 00:00–23:59 CET)
— SIEM correlation events (Splunk, 28 Jan 09:00–15:00 CET)
— Application performance monitoring data (Dynatrace, 7-day window)
— Deployment records (December 2024 release notes)
— Capacity planning documentation (June 2024 review)
— Digital banking adoption metrics (Q3–Q4 2024)
Analysis Team:
— Marcus Weber, Incident Manager (Lead Analyst)
— Lars Eriksson, Head of IT Infrastructure
— Thomas Andersen, Head of Digital Banking
— DBA Lead, Database Administration
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
3. CONTRIBUTING FACTORS
Factor 1: Connection Leak in Batch Processing Module
─────────────────────────────────────────────────────────────────────────────
The batch payment reconciliation module, deployed in the December 2024 release
(v2024.12.1), contained a defect in its error handling path. When a batch
reconciliation query encountered a timeout or constraint violation, the module
failed to close the database connection before throwing the exception. Under
normal operation, this leak was masked because error-path execution was rare.
On the morning of 28 January, a batch reconciliation job processing 4,200
records encountered 23 constraint violations, leaking 23 connections.
Factor 2: Insufficient Connection Pool Capacity
─────────────────────────────────────────────────────────────────────────────
The T24 database connection pool was configured with a maximum of 500
connections, based on the capacity planning exercise conducted in June 2024.
That exercise used Q1 2024 peak transaction volumes (18,500 transactions/hour)
as the baseline. By January 2025, peak volumes had reached 22,400
transactions/hour — a 21% increase driven by the 15% growth in digital banking
adoption during Q4 2024.
Factor 3: Monitoring Threshold Too High
─────────────────────────────────────────────────────────────────────────────
The connection pool utilisation alert was configured to trigger at 95% (475 of
500 connections). At peak transaction rates, the pool could move from 95% to
100% in under 3 minutes, providing insufficient time for proactive intervention.
The alert triggered at 09:14 CET, only 3 minutes before the first transaction
failures.
Factor 4: Capacity Planning Cycle Gap
─────────────────────────────────────────────────────────────────────────────
The quarterly capacity planning cycle scheduled for Q4 2024 was deferred to
January 2025 due to the year-end change freeze. This meant the 15% growth in
digital banking adoption during Q4 2024 was not reflected in infrastructure
capacity adjustments before the incident.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
4. ROOT CAUSE IDENTIFICATION
Primary Root Cause:
Connection leak in the batch payment reconciliation module (v2024.12.1) that
failed to release database connections on the error handling path, combined
with a connection pool sized for Q1 2024 volumes that had been outgrown by
21% transaction growth.
Root Cause Category: Technical (software defect + infrastructure configuration)
Fault Tree Summary:
Transaction failures
├── Connection pool exhausted (500/500)
│ ├── Leaked connections from batch module (23 connections)
│ │ └── Missing connection.close() in catch block
│ ├── Peak load exceeding pool capacity
│ │ └── 21% volume growth since last capacity review
│ └── No automatic pool scaling configured
└── No failover to secondary node (manual process)
└── Automatic failover not implemented for T24
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
5. CORRECTIVE ACTIONS
ID Action Owner Deadline Status
─────────────────────────────────────────────────────────────────────────────
CA-01 Patch connection leak in batch module App Support 31 Jan Done
CA-02 Increase pool to 1500 (permanent) IT Infra 31 Jan Done
CA-03 Add error-path integration tests QA Lead 14 Feb Done
CA-04 Lower monitoring threshold to 75% SOC Manager 7 Feb Done
CA-05 Implement auto-scaling for pool IT Infra 28 Feb In progress
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
6. PREVENTIVE RECOMMENDATIONS
ID Recommendation Owner Deadline
─────────────────────────────────────────────────────────────────────────────
PR-01 Move capacity planning to monthly cycle CIO 1 Mar 2025
PR-02 Deploy connection pool health dashboard IT Infra 28 Feb 2025
PR-03 Mandate connection cleanup verification Dev Lead 31 Mar 2025
in all code reviews for DB-accessing code
PR-04 Implement automated failover for T24 IT Infra 30 Jun 2025
PR-05 Add connection leak detection to CI/CD QA Lead 31 Mar 2025
pipeline (static analysis rule)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
7. IMPLEMENTATION PLAN
Phase 1 — Immediate (completed):
CA-01 through CA-04 completed within 14 days of incident
Phase 2 — Short-term (February 2025):
CA-05, PR-01, PR-02 targeted for completion by end of February
Phase 3 — Medium-term (Q1–Q2 2025):
PR-03, PR-04, PR-05 targeted for completion by end of Q2 2025
Tracking: All actions tracked in the ICT risk register under risk ID
ICT-R-2025-42. Monthly progress review by ICT Risk Committee.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Prepared by: Marcus Weber, Incident Manager
Reviewed by: Katrin Halvorsen, CISO
Approved by: Erik Lindqvist, CRO
Date: 10 February 2025
Example structured facts that Detrixa would extract from this evidence (synthetic data, deterministic seed).
incident_final_report_status — fs-incident-final-report
{
"factId": "b1c2d3e4-f5a6-7890-abcd-200000000006",
"evidenceId": "a0b1c2d3-e4f5-6789-abcd-200000000006",
"evidenceClassId": "incident-final-report",
"factType": "incident_final_report_status",
"data": {
"incident_id": "INC-2025-0042",
"submission_date": "2025-02-19",
"has_root_cause_analysis": true,
"has_preventive_measures": true,
"total_duration_hours": 5.22,
"total_clients_affected": 12340,
"total_financial_impact_eur": 138200.00,
"preventive_measures_count": 5,
"submitted_within_deadline": true
},
"provenance": "deterministic",
"extractorVersion": "dora-test-generator/0.1.0",
"extractedAt": "2025-02-28T09:00:00Z",
"supersededBy": null
}
incident_root_cause_analysis — fs-incident-root-cause-analysis
{
"factId": "b1c2d3e4-f5a6-7890-abcd-200000000007",
"evidenceId": "a0b1c2d3-e4f5-6789-abcd-200000000007",
"evidenceClassId": "incident-root-cause-analysis",
"factType": "incident_root_cause_analysis",
"data": {
"incident_id": "INC-2025-0042",
"analysis_date": "2025-02-10",
"methodology_used": "combined",
"root_cause_identified": true,
"root_cause_category": "technical",
"corrective_actions_count": 5,
"corrective_actions_completed": 4,
"systemic_risk_identified": true
},
"provenance": "deterministic",
"extractorVersion": "dora-test-generator/0.1.0",
"extractedAt": "2025-02-28T09:00:00Z",
"supersededBy": null
}
fs-incident-final-reportDORA-Art20-P1submission_datetotal_duration_hours{
"properties": {
"has_preventive_measures": {
"type": "boolean"
},
"has_root_cause_analysis": {
"type": "boolean"
},
"incident_id": {
"minLength": 1,
"type": "string"
},
"preventive_measures_count": {
"minimum": 0,
"type": "integer"
},
"submission_date": {
"format": "date",
"type": "string"
},
"submitted_within_deadline": {
"type": "boolean"
},
"total_clients_affected": {
"minimum": 0,
"type": "integer"
},
"total_duration_hours": {
"minimum": 0,
"type": "number"
},
"total_financial_impact_eur": {
"minimum": 0,
"type": "number"
}
},
"required": [
"incident_id",
"submission_date",
"has_root_cause_analysis",
"has_preventive_measures",
"total_duration_hours"
],
"type": "object"
}
fs-incident-root-cause-analysisDORA-Art21-P1analysis_datecorrective_actions_completed{
"properties": {
"analysis_date": {
"format": "date",
"type": "string"
},
"corrective_actions_completed": {
"minimum": 0,
"type": "integer"
},
"corrective_actions_count": {
"minimum": 0,
"type": "integer"
},
"incident_id": {
"minLength": 1,
"type": "string"
},
"methodology_used": {
"enum": [
"five_whys",
"fishbone",
"fault_tree",
"combined",
"other"
],
"type": "string"
},
"root_cause_category": {
"enum": [
"technical",
"process",
"human",
"external",
"vendor"
],
"type": "string"
},
"root_cause_identified": {
"type": "boolean"
},
"systemic_risk_identified": {
"type": "boolean"
}
},
"required": [
"incident_id",
"analysis_date",
"methodology_used",
"root_cause_identified",
"corrective_actions_count"
],
"type": "object"
}