Practice simplicity

Be proactive, rather than reactive

Think creatively, but adhere to standards

Employ best practices

Measurability is survivability

numbat - AI agent observability

toolsmith #154: endpoint visibility into AI agent activity - local detection, optional pre-action blocking, forensic reconstruction

/post/numbat/numbat_logo_small-thumb.png

Enterprises face an unmanaged crisis of AI agent and MCP server sprawl, characterized by rapid, decentralized proliferation of autonomous agents, protocol connections operating with excessive privilege, opaque execution paths, and identity blind spots.
Absent agent-aware governance, modern enterprises struggle to prevent, detect, or contain multi-hop autonomous exploits, leaving environments vulnerable to lateral movement, shadow collaboration, and unauthorized data exfiltration.
More succinctly, in light of the recent OpenAI/Hugging Face incident, monitoring clearly lags behind agent capability. The tooling to observe what agents are actually doing, in real time, is not yet standard practice, even at the labs building the agents.
To that end, Perplexity AI’s open source numbat offers excellent observability and visibility to supported desktop, CLI, IDE, and gateway agents through local hooks and plugins, OTLP/HTTP logs, and on-disk session artifacts.[1]

[Read More]
Categories: toolsmith  Tags: AI observability agent toolsmith LLM MCP 

AI-Powered Knowledge Graph Generator & APTs

toolsmith #153: unstructured text to interactive knowledge graph via LLM & SPO triplet extraction

/post/aikg/AIKGlogo-thumb.jpg

Courtesy of TLDR InfoSec Launches & Tools again, another fine discovery in Robert McDermott’s AI Powered Knowledge Graph Generator. Robert’s system takes unstructured text, uses your preferred LLM and extracts knowledge in the form of Subject-Predicate-Object (SPO) triplets, then visualizes the relationships as an interactive knowledge graph.[1]

[Read More]

OctoSQL & Vulnerability Data

toolsmith #152: CLI SQL to query and join disparate databases and file formats

/post/octosql/logo-thumb.png

As an avid daily reader of TLDR Information Security I benefit twofold. First, I gain interesting insights and recommendations regarding launches and tools, where I first learned about OctoSQL. Second, concerning vulnerability details inevitably land in my inbox on a near daily basis. Aside from my recommendation to join the TLDR InfoSec mailing list, toolsmith readers also benefit twofold as, herein, I share the use of OctoSQL as a fast CLI interface to vulnerability data aggregated via CVE-Vulnerability-Information-Downloader. If ever you’ve wanted to join vulnerability data (CVE, CVSS, EPSS, etc) from disparate data sources and file types, this is the toolsmith for you.

[Read More]
Categories: toolsmith  Tags: SQL OctoSQL CVE EPSS toolsmith CVSS KEV vulnerabilities 

toolsmith snapshot: Protect AI ModelScan

Protection Against Model Serialization Attacks

/post/modelscan/modelscan-thumb.png

Protect AI’s OSS portfolio includes tools aimed at improving security of AI/ML software. These tools are meant for a wide range of engineering, security and ML practitioners including developers, security engineers/researchers, ML engineers, LLM engineers and prompt engineers, and data scientists.
Of particular interest in light of model serialization attacks is ModelScan.

[Read More]

Sandfly Security

toolsmith #151: Agentless Linux security with unmatched speed and reliability

Sandfly Security, headquartered in New Zealand (where they know sandflies all to well), refers to itself as such because they’re like sandflies: they relentlessly bug and discourage intruders, deploying like a swarm onto endpoints, then disappear only to return again and again. Theses swarms of checks make life miserable for hackers on Linux hosts while minimizing system impact. I’ve been following Sandfly’s Craig Rowland on Twitter for awhile with the intent of giving Sandlfy a look for toolsmith, and in the time I’ve kept watch, the offering has grown into a comprehensive and robust platform for Linux security.

[Read More]

EDA with CISSM

toolsmith #150: Exploratory Data Analysis with University of Maryland's Center for International and Security Studies Cyber Attacks Database

Introduction

Exploratory data analysis (EDA) is a mission critical task underpinning the predominance of detection development and preparation for cybersecurity-centric machine learning. There are a number of actions that analysts can take to better understand a particular data set and ready it for more robust utilization. In the spirit of toolsmith, and celebration of this being the 150th issue since toolsmith’s inception in late 2006, consider what follows a collection of tools for your security data analytics tool kit.

[Read More]

Prowler v3 - AWS & Azure security assessments

/post/prowler/ProwlerLogo-thumb.png

As a current Google Cloud Platform defender, and former Microsoft Azure defender, I appreciate any tool or capability intended to provide multi-cloud protection. As noted via LinkedIn, Toni announced the release of Prowler v3 just before Christmas. Prowler v3 is a complete overhaul of Prowler, fully rewritten it in Python. Prowler is an open source security tool to perform AWS and Azure security best practices assessments, audits, incident response, continuous monitoring, hardening and forensics readiness. Prowler v3 is now multi-cloud with Azure added as the second supported cloud provider. Prowler is significantly faster as well, now able to scan an entire AWS account across all regions 37 times faster than before.

[Read More]
Categories: toolsmith  Tags: cloud Azure AWS GCP OCI 

Chainsaw

toolsmith #148: Hunt, search, and extract event log records

/post/chainsaw/logo-thumb.PNG
I first spotted Chainsaw courtesy of Florian Roth’s Twitter feed given that Chainsaw favors using Sigma as one of its rule engines. Chainsaw is a standalone tools that provides a simple and fast method to triage Windows event logs and identify interesting elements within the logs while applying detection logic (Sigma and Chainsaw) to detect malicious activity. Chainsaw’s powerful ‘first-response’ capability offers a generic and fast method of searching through event logs for keywords (Kornitzer & D, 2022).

[Read More]
Categories: toolsmith  Tags: SOC blue team DFIR DART TI Chainsaw 

EPSScall - An Exploit Prediction Scoring System App

toolsmith #147: EPSScall - Shiny app for the EPSS API

If you follow Cyentia Institute’s Jay Jacobs via social media you may FIRST ;-) have learned about the Exploit Prediction Scoring System (EPSS) from him, as I did. I quickly learned that FIRST offers an API for the EPSS Model, which immediately piqued my interest. Per FIRST, EPSS provides a fundamentally new capability for efficient, data-driven vulnerability management. While EPSS predicts the probability (threat) of a specific vulnerability being exploited, it can scale to estimate the threat for multiple vulnerabilities on a server, a subnet, mobile device, or at an enterprise level (Jacobs, 2022).
“The (EPSS) is a community-driven effort to combine descriptive information about vulnerabilities (CVEs) with evidence of actual exploitation in-the-wild. By collecting and analyzing these data, EPSS seeks to improve vulnerability prioritization by estimating the likelihood that a vulnerability will be exploited. The EPSS model produces a probability score between 0 and 1 (0% and 100%). The higher the score, the greater the probability that a vulnerability will be exploited (in the next 30 days)” (Jacobs, 2022).
As of February 2022, EPSS version 2 is available; give Jay’s write-up a good read before proceeding. EPSS v2 is preceded by EPSS v1 and CVSS v3. Note the significant increase in model coverage and efficiency per Figure 1.

/post/147/Figure1-thumb.JPG
Figure 1: EPSS Comparison by Effort

[Read More]

LotL Classifier tests for shells, exfil, and miners

toolsmith #146: A supervised learning approach to Living off the Land attack classification from Adobe SI

Happy Holidays, readers!
First, a relevant quote from a preeminent author in the realm of intelligence analysis, Richards J. Heuer, Jr.:
“When inferring the causes of behavior, too much weight is accorded to personal qualities and dispositions of the actor and not enough to situational determinants of the actor’s behavior.”
Please consider Mr. Heuer’s Psychology of Intelligence Analysis required reading.
The security intelligence team from Adobe’s Security Coordination Center (SCC) have sought to apply deeper analysis of situational determinants per adversary behaviors as they pertain to living-off-the-land (LotL) techniques. As the authors indicate, “bad actors have been using legitimate software and functions to target systems and carry out malicious attacks for many years…LotL is still one of the preferred approaches even for highly skilled attackers.” While we, as security analysts, are party to adversary and actor group qualities and dispositions, the use of LotL techniques (situational determinants) proffer challenges for us. Given that classic LotL detection is rife with false positives, Adobe’s SI team used open source and representative incident data to develop a dynamic and high-confidence LotL Classifier, and open-sourced it. Please treat their Medium post, Living off the Land (LotL) Classifier Open-Source Project and related GitHub repo as mandatory reading before proceeding here. I’ll not repeat what they’ve quite capably already documented.

[Read More]