SophosAI team presents three papers on AI applied to cybersecurity at CAMLIS

Sophos defends more than 600,000 organizations and more than 100 million users globally from active adversaries, ransomware, phishing, malware, and more.

For more info on how you can get the best protection from Sophos visit the link below
SOPHOS

On October 24 and 25, SophosAI presents ideas on how to use models large and small—and defend against malignant ones.

Written by Sean Gallagher

October 23, 2024

AI Research AI Trojans anti-phishing CAMLIS featured Google LLM small model machine learning

Applied Machine Learning in Information Security (CAMLIS), held this week in Arlington, Virginia—one in a featured talk, and the others in a more informal “poster session” during the event. The topics covered cut straight to the heart of what the SophosAI team’s research focuses on—finding more effective ways to use machine learning and artificial intelligence technologies to protect against information security risks and guarding against the risks inherent with AI models themselves.

On October 24, SophosAI’s Ben Gelman, Sean Bergeron and Younghoo Lee will present during a poster session. Gelman and Bergeron will deliver a talk entitled ” The Revitalization of Small Cybersecurity Models in the New Era of AI.”

Smaller machine learning models have gotten short shrift in much of the research focused on Large Language Models (LLMs) such as OpenAI’s GPT-4, Google’s Gemini and Meta’s LLaMA. But they remain critical to information security at network edges and endpoints, where the computational and network costs of LLMs make them impractical.

In their presentation, Gelman and Bergeron will talk about how to use LLM technology to supercharge the training process for smaller models, discussing techniques SophosAI has used to make small, cost-effective models perform at much higher levels in a variety of cybersecurity tasks.

In a related talk, Lee will present “A fusion of LLMs and lightweight ML for effective phishing email detection.” With adversaries now turning to LLMs to generate more convincing, targeted phishing emails with unique text patterns in addition to leveraging previously unseen domain names to evade traditional spam and phishing defenses, Lee investigated how LLMs can be used to counter them—and how they can be combined with traditional smaller machine learning models to be even more effective.

In the approach Lee presents in his paper, LLMs can be harnessed to detect suspicious intentions and signals, such as sender impersonation and deceptive domains. And by fusing LLMs with more lightweight machine learning models, it is possible to both enhance phishing detection accuracy and get past the limitations of both types of models when used on their own.

On the second day of CAMLIS, SophosAI’s Tamás Vörös will present a talk on his research into defanging malicious LLMs—models that carry embedded backdoors or malware intended to be activated by specific inputs. His presentation—entitled “LLM Backdoor Activations Stick Together”— demonstrates both the risks of using “black box” LLMs (by showing how the SophosAI team injected their own controlled Trojans into models) and “noising” methods that can be used to disable pre-existing Trojan activation commands.

• Share on Facebook
• Share on X
• Share on LinkedIn

About the Author

Sean Gallagher

Sean Gallagher is Principal Threat Researcher, Sophos X-Ops. Prior to joining Sophos, he was an information security and technology journalist for over 30 years, including 10 as information security and national security editor for Ars Technica.

For more info on how you can get the best protection from Sophos visit the link below
SOPHOS