# Artifact Security — The Independent Security Test Lab > We measure whether security products actually stop attacks — under identical, published methodologies — and publish the results in the open. Independent test lab · London, UK · AMTSO members HTML version: https://artifactsecurity.co.uk/ · Site directory for agents: https://artifactsecurity.co.uk/llms.txt ## Running now - **AI Scam & Phishing Test** (world's largest) — live lure corpus, AMTSO standard, under Scam & Phishing 1.5 - **VPN Performance World Tour** (world first) — real hardware on real networks, city by city, under VPN Performance 1.0 ## The lab in numbers - 15+ years testing experience - 100+ products tested - 10 public methodologies - 2 live AMTSO evaluations ## How we score — every test runs the same open loop 1. **Methodology published** — public before any product is scored 2. **Open for challenge** — anyone can dispute any step 3. **Test runs** — identical conditions; weeks, not months 4. **Right of reply** — vendor responses printed beside verdicts 5. **Results published** — ranked in the open, with evidence ## Latest tests - **AI Scam & Phishing** (LIVE, Scam & Phishing 1.5) — the world's largest AI scam & phishing test - **VPN Performance World Tour** (LIVE, VPN 1.0) — world-first global tour on real networks - **AI-Powered SOC** (AI SOC 1.0) — noise, metrics, trust — measured - **Cloud & Identity** (Cloud & Identity 1.0) — posture, access, and identity defenses tested Full registry: [Methodologies](https://artifactsecurity.co.uk/test-methodologies/index.md) · Published results: [Reports](https://artifactsecurity.co.uk/reports/index.md) ## Research & insights - [Building resilience using a layered approach to combat ransomware](https://artifactsecurity.co.uk/post/ransomware-impact/index.md) (22 Mar 2026) - [Cyber 2025: the year we stopped pretending legacy defenses work](https://artifactsecurity.co.uk/post/cyber-2025-the-year-we-stopped-pretending-legacy-defenses-work/index.md) (5 Jan 2026) - [AI-powered SOCs — noise, metrics, trust](https://artifactsecurity.co.uk/post/ai-powered-socs-noise-metrics-trust/index.md) (5 Jan 2026) ## Get your product tested Prove your capabilities, differentiate your product. - Email the lab: hello@artifactsecurity.co.uk - Programmes: [Tests & Certification](https://artifactsecurity.co.uk/services/index.md) · [Penetration Testing](https://artifactsecurity.co.uk/penetration-testing/index.md) --- © 2026 Artifact Security Ltd · London · Values: Transparency, Innovation, Partnership · AMTSO member since 2025 · Right of reply, always --- # Tests & Certification — Artifact Security > Six evaluations and two certification tracks — for security vendors, enterprises, SMBs and consumers. Real-world tradecraft, in weeks not months. Every methodology is public. HTML version: https://artifactsecurity.co.uk/services · Contact: hello@artifactsecurity.co.uk ## Four ways in 1. **Evaluations** — beyond checklists: real-world tradecraft with personalised next steps 2. **Certification** — efficient product validation, currently valid for VirusTotal participation 3. **Bespoke testing** — prove a new product's value under real-world methodologies 4. **Consultation** — a plan tailored to your needs; the right test, not the biggest one ## Evaluations ### 1. Scam & Phishing (LIVE · AMTSO · Scam & Phishing 1.5) AI has lowered the bar for launching convincing scams. Platform-agnostic evaluation — real-world scenarios, tested the way attackers actually work. Consumer focus, AI lures, live corpus. Running now (H1 2026) as the world's largest AI scam & phishing test. - Verdicts: blocked / missed, with evidence — every lure stamped, screenshotted, reproducible - Comparable: every product faces an identical set of live scams ### 2. AI SOC (AI SOC 1.0) Do AI assistants actually pay off in the SOC? We measure return on investment in real workflows — key metrics, not vibes. Enterprise / autonomous SOC. Metrics measured: - Detection accuracy & false positives - Alert efficiency (volume change vs baselined period) - Time to initial understanding (TIU) - Dwell time (earliest possible detection → intrusion caught) - Time to contain & remediate (TTC / TTR) - Time to advise & implement (TTA / TTI) ### 3. Secure Browser (public commentary phase) Quantifies the defensive ROI of secure browsers against AI, identity and exfiltration risks — consumer secure browsers, remote browser isolation, enterprise browsers. Methodology being shaped in the open; challenge it before anything is scored. Modules: identity security, GenAI guardrail bypass, data loss prevention. ### 4. Identity & Cloud Security (Cloud & Identity 1.0) Credential abuse is the most common way attackers get in. The first ITDR and cloud-focused tradecraft — for MSSP, MDR, identity, cloud and email solutions. - Identity scenarios: ongoing attack detection, post-compromise detection, alert efficiency - Cloud scenarios: intrusion, infiltration, propagation ### 5. Ransomware Impact (Ransomware Impact 1.0) Benchmarks a product's ability to minimise downtime during a live ransomware surge — full chain, pre-encryption to recovery. Cross-industry coverage, 15+ ransomware groups (the most comprehensive group coverage in a public test), realistic environments. ### 6. Advanced Persistent Testing (APT) The most comprehensive detection-and-response testing for EDR, NDR and XDR: efficacy, soft features and false positives across the full killchain (evaluation scope: APT29). - Efficacy ratings: detection (stage by stage), protection, false positives - Capabilities assessment: response customisability, options in the fight, response detail ## Certification ### Real World Protection Consumer and business tracks, live threats, no synthetics. To certify: 97% protection attack rating. Verdicts published as certified / partial / failed with evidence; right of reply always printed beside the verdict. ### VirusTotal Certification Malware scanning to AMTSO's Fundamental Principles using Real-Time Threat List samples. To certify: 100% detection on the malicious corpus, 0% false positives. Certification currently valid for VirusTotal participation. ## FAQ **How are your tests different from a typical pentest or vulnerability scan?** Tests are designed around each client's real systems, threats and goals — revealing practical attack paths and concrete fixes. Reports are designed for all stakeholders, not only engineers. **What organisations do you work with?** Enterprises with complex or regulated environments; security vendors needing independent product testing; high-growth startups selling into enterprise or regulated sectors. **How often should we repeat testing?** After major architecture/product changes, or annually if stable. High-change organisations often move toward continuous assessment. **Confidentiality?** NDAs standard; minimum-necessary data access, logged; artifacts stored under strict controls; report details can be further restricted on request. **Can I give feedback on methodologies or reports?** Yes — every methodology is open for challenge before a product is scored, and the changelog is public. --- Not sure which test fits? Email hello@artifactsecurity.co.uk — we'll point you at the right programme, or say honestly if none fit yet. --- # Penetration Testing — Artifact Security > Penetration testing, priced in the open. Three tiers, public pricing, manual testing by certified engineers — reports designed for every stakeholder, not just the engineers. 15+ years of independent security testing. HTML version: https://artifactsecurity.co.uk/penetration-testing · Contact: hello@artifactsecurity.co.uk ## Tiers ### Tier 1 — Essentials · $5k Your first line of defence. Results within 5 business days. - Vulnerability scanning; external attack surface assessment - OWASP Top 10 coverage; CVE identification and severity rating - Detailed findings report + remediation recommendations - Single scope — one app or domain ### Tier 2 — Startup · $10k Everything in Essentials, plus: - Manual penetration testing by certified engineers - Authentication and authorisation testing; session management analysis - Business logic flaw detection; API security testing - Exploit chaining and risk validation - Executive summary + technical report; 30-day remediation support window ### Tier 3 — Enterprise · $30k+ The full battery: - APT-style simulation from bespoke lab threat intelligence - Internal network penetration testing; social engineering and phishing simulation - Cloud infrastructure review (AWS / Azure / GCP); Active Directory and identity security testing - Third-party integrations assessment; retesting after remediation - Dedicated security engineer throughout; board-ready executive report; 90-day support window - Add-on: physical security assessment ## How it runs 1. Scope agreed — fixed before we start 2. Test runs — manual work by certified engineers 3. Findings reported — executive summary + technical detail 4. Remediation support — we stay on hand while you fix 5. Retest — fixes verified, not assumed --- Not sure which tier? Tell us what you're protecting and what your buyers or regulators need to see — we'll recommend the smallest engagement that answers the question. hello@artifactsecurity.co.uk --- # Methodology Registry — Artifact Security > Every methodology public, versioned, and open for challenge before a single product is scored. If a methodology can't survive scrutiny, neither can its verdicts. HTML version: https://artifactsecurity.co.uk/test-methodologies · Challenge a methodology: hello@artifactsecurity.co.uk (subject: "Methodology challenge") ## At a glance - 10 public methodologies · 2 live AMTSO evaluations · right of reply via AMTSO Standard ## Running now - **VPN Performance World Tour** (live, world first, H1 2026) — real hardware on real networks, city by city, under VPN Performance 1.0. Fastest verified so far: 908 Mbps. [AMTSO listing](https://www.amtso.org/tests/artifact-security-vpn-performance-evaluation-h1-2026/) - **AI Scam & Phishing Test** (live, world's largest, H1 2026) — live lure corpus under Scam & Phishing 1.5. [AMTSO listing](https://www.amtso.org/tests/artifact-security-scam-and-phishing-evaluation-h1-2026/) ## The registry Ten methodologies, all public as versioned PDFs, across: Protection (5), Phishing & scam (2), AI & cloud (2), Performance (1). Each entry links the published PDF, its AMTSO listing where applicable, and a challenge route. See the HTML registry for the full interactive list. ## How a methodology lives 1. **Drafted in the open** — shaped with AMTSO working groups where possible 2. **Published** — full PDF, versioned 3. **Open for challenge** — security vendors, researchers, practitioners feedback 4. **Revised** — 1.0 → 1.1 → 1.5, changelog public 5. **Test runs** — identical conditions, weeks not months 6. **Right of reply** — vendor responses beside verdicts ## Disagree with a step? Good. Transparency means the methodology is challengeable before the test, not after the verdict. Tell us what we got wrong — we publish the changelog. --- # Test Reports — Artifact Security > Every published test: headline verdicts up front, full evidence behind a download. Vendors get right of reply, readers get the raw story — new rounds land every month. HTML version: https://artifactsecurity.co.uk/reports · Raw data: [Transparency Portal](https://portal.artifactsecurity.co.uk/) ## At a glance - 6 reports published · 2 tests running now · right of reply on every report ## Latest report — VPN Performance World Tour, Round 1 (June 2026) Consumer · multi-platform · global (5 origin cities) · AMTSO aligned · report ID VPNPerf2026v1 The race result: one Platinum, three Golds, a Silver. Real hardware on real networks across 55 highways from 5 origin cities. NordVPN tops the regional medal table — the only overall Platinum, winning 3 of 5 regions. Every dimension scored on its merits, Olympics-style. Headline numbers: - Fastest download: 908 Mbps (NordVPN) - Fastest upload: 920 Mbps (NordVPN) - Lowest latency: 183 ms (Vendor C) - Best time-to-connect: 1.8 s at 98% success - Leak testing SC-LR-01–05: all clear - CAPTCHA friction walk: 98% clean-site rate Coverage: speed & consistency (55 highways), packet-level time-to-connect, leak testing, kill switch & feature verification (SHA-256-hashed captures), CAPTCHA friction. Full report PDF and [AMTSO listing](https://www.amtso.org/tests/artifact-security-vpn-performance-evaluation-h1-2026/) on the HTML page. ## Inside every report - Executive verdicts a board can read in ninety seconds - Plain-English explainers written for buyers, not just engineers - Scenario-ID rigour — every case numbered, reproducible, browsable - Full result matrices — per-region, per-platform, per-stage, filterable - Hashed evidence — screenshots, probe JSON, SHA-256 packet captures - Right of reply printed alongside every verdict ## Transparency Portal (live) Every result open to inspect at https://portal.artifactsecurity.co.uk — searchable per vendor, per region, per stage. The evidence doesn't stop at the PDF. ## All reports The published record spans VPN, scam & phishing, and ransomware tests, for consumers and enterprise. Vendor missing? Ask them to take part: hello@artifactsecurity.co.uk --- # Our Team — Artifact Security > We innovate, challenge the status quo, and never compromise on our values — to keep the industry to the highest standard of quality and transparency. HTML version: https://artifactsecurity.co.uk/our-team · London, UK ## Standards & industry - **AMTSO** — Artifact Security is a member of AMTSO, the cybersecurity industry's testing-standards community. Co-founder **Stefan Dumitrascu is AMTSO President**, helping steer how the whole industry tests. - **Sigma Squared** — Co-founder **Ana M. Pricop** serves as UK executive for Sigma Squared, connecting the lab to a global network of founders and leaders. ## How we operate — three values, enforced 1. **Transparency** — methodologies published before we test; changelogs public; right of reply printed beside every verdict 2. **Innovation** — first ITDR & cloud-focused tradecraft, world-first VPN tour, world's largest AI scam test — we build tests the industry doesn't have yet 3. **Partnership** — vendors challenge our methodologies before we score; clients get next steps, not just verdicts; feedback changes the methodology ## Work with the lab Vendor, enterprise or journalist — if you want your product measured or a question answered: hello@artifactsecurity.co.uk --- # Media & Press — Artifact Security > Numbers you can print. Detailed test results, advance copies and content tailored to your editorial needs — from a completely independent, unsponsored testing team. HTML version: https://artifactsecurity.co.uk/media · Press enquiries: hello@artifactsecurity.co.uk (subject: "Press enquiry") ## What working with the lab looks like 1. **Data** — detailed test results, advance copies under embargo, extended report copies and datasets (including how AI tackles scams, phishing and other cyber threats) 2. **Editorial** — story angles, data cuts and technical explanations worked through together, aligned with what your readers care about 3. **Access** — interview requests, expert quotes, or ongoing collaboration ## Ask us for - **Advance copies** — reports before they publish, under embargo - **Data cuts** — the numbers behind a story, cut for your angle - **Expert quotes** — from the people who actually ran the test - **Explainers** — how the technology works, in plain English ## Start here — the published record, free to cite - [Test reports](https://artifactsecurity.co.uk/reports/index.md) - [Methodologies](https://artifactsecurity.co.uk/test-methodologies/index.md) - [Blog](https://artifactsecurity.co.uk/blog/index.md) On deadline? Tell us your outlet, angle and timing — we'll come back with data, quotes or a better story than the one the press release gave you. Media kit coming soon. --- # Blog — Artifact Security > Research & insights, written from the bench — by the people who run the tests, not a content team. HTML version: https://artifactsecurity.co.uk/blog · Newsletter: https://substack.com/@artifactsecurity ## Posts - [Building resilience using a layered approach to combat ransomware](https://artifactsecurity.co.uk/post/ransomware-impact/index.md) — 22 March 2026 · Ransomware · Why single-layer prevention scores mislead, and what full-chain testing shows instead - [Cyber 2025: the year we stopped pretending legacy defenses work](https://artifactsecurity.co.uk/post/cyber-2025-the-year-we-stopped-pretending-legacy-defenses-work/index.md) — 5 January 2026 · Industry · A year of evidence against checkbox security, from the lab floor - [AI-powered SOCs — noise, metrics, trust](https://artifactsecurity.co.uk/post/ai-powered-socs-noise-metrics-trust/index.md) — 5 January 2026 · AI SOC · What autonomous triage gets right, where it hallucinates, and how we score it ## Where we publish 1. **Substack** — our cyber newsletter: summaries that cut through the noise, plus every report the day it lands 2. **LinkedIn** — test announcements, results and lab news: https://www.linkedin.com/company/artifact-security/ 3. **X** — live updates while tests are running: https://x.com/artifactseq --- # Security & Privacy Policy — Artifact Security > How we protect the data entrusted to us, in accordance with UK GDPR and the Data Protection Act 2018. Last updated: January 2026. HTML version: https://artifactsecurity.co.uk/policies/privacy-policy · DPO: privacy@artifactsecurity.co.uk Artifact Security Ltd is registered in England and Wales, company number 16044234. Registered office: International House, 109-111 Fulham Palace Road, London, W6 8JA. ICO registration: ZC044539. ## 1. Who we are UK-based provider of security services supporting security vendors with specialized expertise. We act as Data Controller for personal information we collect about you, and Data Processor when handling data on behalf of clients. ## 2. Commitment to security We maintain an ISMS aligned with ISO 27001:2022. Core technical controls: - **Encryption** — AES-256 at rest, TLS 1.2+ in transit - **IAM** — least privilege; MFA mandatory on all internal and client-facing systems - **Endpoints** — full disk encryption, centralized MDM - **Zero-root policy** — no direct system-level account logins; named accounts, fully audited - **Offboarding** — all project-specific access revoked within 24 hours of engagement completion ## 3. How we collect information Information you give us (forms, email: name, business email), information collected automatically (IP address, browser, OS), and information from third parties (professional networks, industry partners). ## 4. How your information is used Contract obligations, seeking feedback, service change notifications, requested communications, and network/service security and integrity. ## 5. Data governance - **UK-first data residency**; international transfers protected via UK Adequacy Regulations or IDTA - **Retention** — engagement data deleted within 30 days of contract end; legal/tax records kept 6 years (UK statutory); cryptographic erasure and certified cloud deletion ## 6. Your rights Under UK GDPR: access, rectification, erasure, and objection to processing. Cookies can be disabled via browser preferences. ## 7. Incident management Confirmed breaches: ICO notified within 72 hours where required; affected clients informed without undue delay. ## 8. Other websites This policy applies only to our website; read the privacy statements of linked sites. ## 9. Contact DPO: privacy@artifactsecurity.co.uk --- # Building resilience using a layered approach to combat ransomware > Rather than asking whether ransomware can be stopped entirely, the more meaningful questions are whether your business has enough protections in place — and how well it sustains itself when ransomware passes through. By Artifact Security · 22 March 2026 · Ransomware · 5 min read HTML version: https://artifactsecurity.co.uk/post/ransomware-impact What's the best way to protect against ransomware? If we knew the answer we'd be billionaires living on a sunny beach with a coconut in hand. So let's talk about resilience instead — resilience to attacks, resilience to ransomware. As with everything in security, you need a multi-layered approach that fits the needs of your business. Let's ask the right questions first: - Are you focusing on detecting? What are you hunting for in your IOCs? - From a technological and human perspective, can you deal with the threat after you detect it? - If you focus on preventing it, are you blindly trusting that everything is up to date with the latest evasion techniques? - Backups seem great — but how about the reputational damage that may occur? ## The impact on SMBs Some figures: 1 in 5 businesses that suffer a major cyberattack are forced to cease operations (source: Mastercard's 2025 Global SMB Study). It's not just operations that are affected, but reputation too. The supply chain is under more scrutiny than ever — if you want to be in the chain of an enterprise, you need to show your resilience. Attacks will happen; showing you've done your due diligence means you've thought about the security of your business and customers. According to Verizon's DBIR, ransomware is present in 88% of breaches in SMBs. Ransomware-as-a-service is ever increasing, and SMBs usually lack the multi-layered approach that large enterprises can afford. Since the cost of launching attacks keeps falling, attackers can afford a wide variety of targets. We were excited to work with QuellSecure from early stages to help validate and improve their approach. A low-cost, hands-off approach is exactly what SMBs should aim for — they usually can't afford huge teams to deal with attacks. As QuellSecure's solution is an addition rather than a replacement, it tackles the resilience aspect SMBs typically lack. It focuses on what happens when the usual layers fail: users will inevitably click on something they shouldn't; credentials will leak. It targets ransomware once it actually starts encrypting your files. Our testing focused on measuring the impact of ransomware on target organisations, and how a product helps the organisation stay resilient. The approach is simple at heart but complex in practice: get real ransomware against real environments and measure how many files are lost. Our targets held thousands of files — a mix of typical office, editing and development formats. Rather than asking whether ransomware can be stopped entirely, the more meaningful questions are whether your business has enough protections in place, and how well it sustains itself when ransomware passes through. Read more about ransomware, our testing and the full results in the QuellSecure report on [reports](https://artifactsecurity.co.uk/reports/index.md). --- # Cyber 2025: the year we stopped pretending legacy defenses work > Fighting 2025 threats with 2015 tools is a losing battle. Last year showed us what fails, what survives, and what must change now. By Artifact Security · 5 January 2026 · Industry · 9 min read HTML version: https://artifactsecurity.co.uk/post/cyber-2025-the-year-we-stopped-pretending-legacy-defenses-work/ Dear readers — security theater does not equal actual security. 2025 was yet another wake-up call: a direct shot in the face of every organization that thought their firewall from 2015 would save them. AI went from a boardroom buzzword to an easy, accessible weapon for both attackers and defenders. Nations got more serious about technology, healthcare systems got hammered, and somewhere a CISO is still explaining why "we've always done it this way" isn't a security strategy. Here's the uncomfortable truth: most organizations are fighting 2025 threats with 2015 tools. And it's costing them everything. ## The attacks that defined 2025 - **Bybit.** In February, the Lazarus Group pulled off the largest cryptocurrency theft in history, stealing $1.447 billion worth of Ethereum. Over 596 suspicious domains were detected targeting Bybit customers. - **Microsoft SharePoint zero-days.** State-linked hackers exploited two critical vulnerabilities (CVE-2025-53770 and CVE-2025-53771) in what became known as "ToolShell." By the time Microsoft patched in July, 396 SharePoint systems had already been compromised. - **Jaguar Land Rover.** Described as the most economically damaging cyberattack to hit the UK in history. A September hack delayed car production for months, hitting suppliers so hard that some went out of business — the UK government stepped in with a £1.5 billion bailout. Disruption is often more valuable to attackers than stolen data. - **Coupang.** Data was stolen for five months before anyone noticed. By discovery, 33 million customers' information had been compromised; the breach led to the CEO's resignation. - **China's surveillance leak** became the largest known data breach in the country's history — over 4 billion user records. - **Iran's Bank Sepah** lost 42 million customer records (~12 TB) to a collective demanding $42 million in Bitcoin. The pattern? Attackers aren't just breaking in anymore. They're staying for months, taking everything, causing maximum disruption — and organizations only discover the breach when it's far too late. ## The numbers don't lie Organizations faced an average of 1,876 cyberattacks in Q3 2024 — a 75% year-over-year increase that accelerated into 2025. Phishing attacks have risen 4,151% since ChatGPT's public release, and AI-generated phishing emails now achieve a 54% click-through rate versus 12% for traditional phishing. More than half of people can't tell the difference. 93% of US healthcare organizations experienced an average of 43 cyberattacks over the past year. The average data breach now costs $4.9 million; healthcare breaches cost $9.77 million. By 2027, cybercrime is projected to cost the global economy $24 trillion. ## What actually improved Over 86% of organizations have begun moving to Zero Trust; the market reached $38.37 billion in 2025. Companies that complete all Zero Trust pillars are twice as likely to avoid a reported incident. Consistent use of AI and automation saves an average of $2.2 million per breach, and MFA — when actually implemented — blocks over 99% of identity-based attacks. ## The uncomfortable truths nobody wants to say Security is still treated as a cost rather than a business enabler — only 28% of companies embed security controls in transformation initiatives from the start. The talent shortage is worsening, with an estimated 4.8 million unfilled positions worldwide. And speed is still prioritized over security: spending on generative-AI initiatives outpaced security budgets by 2.6× in 2025. Organizations are building AI systems on insecure foundations. ## What actually works - **Treat cybersecurity as a business imperative**, not an IT problem — organizations that do are 69% less likely to experience advanced attacks. - **Implement Zero Trust like you mean it.** Full implementation drops incident rates from 66% to 33%. - **Embrace AI for defense** — thoughtfully, understanding both benefits and risks. - **Fix the basics.** MFA, encryption, patch management, access controls. - **Build security into procurement.** Your security is only as strong as your weakest vendor. - **Invest in people**, not just tools. Security is at a turning point. The game changes when we stop treating it as a checkbox and start treating it as what it actually is — the foundation everything else is built on. Stay secure, A ## Sources & references (selected) Accenture — State of Cybersecurity Resilience 2025 · World Economic Forum — Global Cybersecurity Outlook 2025 · Verizon DBIR 2025 · IBM Cost of a Data Breach 2024 · ISC2 Cybersecurity Workforce Study · Checkpoint Research Q3 2024. Full reference list: https://substack.com/@artifactsecurity --- # AI-powered SOCs — noise, metrics, trust > AI is reshaping the SOC — but hype creates noise. Here is how to evaluate AI tools using the right metrics to cut alert fatigue and build real trust. By Artifact Security · 5 January 2026 · AI SOC · 6 min read HTML version: https://artifactsecurity.co.uk/post/ai-powered-socs-noise-metrics-trust/ AI has been on everyone's lips for the last few years, and RSAC pushed the "AI hype" even further. Whether you're a full believer or a major skeptic, it's here to stay. The real use cases vary wildly — from lowering the barrier of entry for attackers, to empowering L1 analysts to do more, to more efficient SOAR workflows. Sometimes it feels like we're bombarded with potential. ## Noise, metrics & trust When it comes to noise, we're talking about both marketing claims and alert fatigue. New tools are added to empower your SOC team across various workflows — but how do you know what actually benefits your business? You need to identify the key metrics to track, set appropriate prioritization, and then make the right decisions when investing. No one can hand you the single "most important" metric — yet different vendors each claim theirs is the most crucial. In our evaluation we combined our own research with public information on SOC workflows to establish key metrics. Forrester's approach categorizes them as strategic, operational and tactical. Classic metrics like detection accuracy and false positives matter, but businesses need to establish what's most important for *them*. We track 9 metrics across those categories by splitting the evaluation into different operations, each with its own goal: - Detection accuracy & false positive rate - Dwell time / mean time to detect - Alert efficiency - Time to contain & remediate - Time to initial understanding - Time to advise & implement To trust these metrics they must be looked at in the context of each other, never on their own. That gives customers a real picture of how a tested solution benefits their own needs. ## How is this evaluated? This is the only evaluation that looks at these metrics together, and it's a complex procedure with unprecedented cooperation between teams. We designed a complete target organization — target employees, relationships with their own external vendors, normal staff workflows during the evaluation, and threat prioritization. Read more in our [methodology registry](https://artifactsecurity.co.uk/test-methodologies/index.md). AI can be costly in both dollars and person-hours. The promise of more efficient workflows — better detection, lower false positives — is appetizing for decision makers. Before you commit to a new AI companion, tool or solution, consider the priorities you've set in your long-term strategy. Our evaluation is here to help — get in touch (hello@artifactsecurity.co.uk) or encourage your vendor to take part. ---