Cybersecurity AI - AI News https://www.artificialintelligence-news.com/categories/ai-in-action/cybersecurity-ai/ Artificial Intelligence News Thu, 12 Feb 2026 14:02:29 +0000 en-GB hourly 1 https://wordpress.org/?v=6.9.1 https://www.artificialintelligence-news.com/wp-content/uploads/2020/09/cropped-ai-icon-32x32.png Cybersecurity AI - AI News https://www.artificialintelligence-news.com/categories/ai-in-action/cybersecurity-ai/ 32 32 Google identifies state-sponsored hackers using AI in attacks https://www.artificialintelligence-news.com/news/state-sponsored-hackers-ai-cyberattacks-google/ Thu, 12 Feb 2026 09:00:00 +0000 https://www.artificialintelligence-news.com/?p=112167 State-sponsored hackers are exploiting highly-advanced tooling to accelerate their particular flavours of cyberattacks, with threat actors from Iran, North Korea, China, and Russia using models like Google’s Gemini to further their campaigns. They are able to craft sophisticated phishing campaigns and develop malware, according to a new report from Google’s Threat Intelligence Group (GTIG). The […]

The post Google identifies state-sponsored hackers using AI in attacks appeared first on AI News.

]]>
State-sponsored hackers are exploiting highly-advanced tooling to accelerate their particular flavours of cyberattacks, with threat actors from Iran, North Korea, China, and Russia using models like Google’s Gemini to further their campaigns. They are able to craft sophisticated phishing campaigns and develop malware, according to a new report from Google’s Threat Intelligence Group (GTIG).

The quarterly AI Threat Tracker report, released today, reveals how government-backed attackers have begun to use artificial intelligence in the attack lifecycle – reconnaissance, social engineering, and eventually, malware development. This activity has become apparent thanks to the GTIG’s work during the final quarter of 2025.

“For government-backed threat actors, large language models have become essential tools for technical research, targeting, and the rapid generation of nuanced phishing lures,” GTIG researchers stated in their report.

Reconnaissance by state-sponsored hackers targets the defence sector

Iranian threat actor APT42 is reported as having used Gemini to augment its reconnaissance and targeted social engineering operations. The group used an AI to create official-seeming email addresses for specific entities and then conducted research to establish credible pretexts for approaching targets.

APT42 crafted personas and scenarios designed to better elicit engagement by their targets, translating between languages and deploying natural, native phrases that helped it get round traditional phishing red flags, such as poor grammar or awkward syntax.

North Korean government-backed actor UNC2970, which focuses on defence targeting and impersonating corporate recruiters, used Gemini to help it profile high-value targets. The group’s reconnaissance included searching for information on major cybersecurity and defence companies, mapping specific technical job roles, and gathering salary information.

“This activity blurs the distinction between routine professional research and malicious reconnaissance, as the actor gathers the necessary components to create tailored, high-fidelity phishing personas,” GTIG noted.

Model extraction attacks surge

Beyond operational misuse, Google DeepMind and GTIG identified a increase in model extraction attempts – also known as “distillation attacks” – aimed at stealing intellectual property from AI models.

One campaign targeting Gemini’s reasoning abilities involved the collation and use of over 100,000 prompts designed to coerce the model into outputting reasoning processes. The breadth of questions suggested an attempt to replicate Gemini’s reasoning ability in non-English target languages in various tasks.

How model extraction attacks work to steal AI intellectual property. (Image: Google GTIG)

While GTIG observed no direct attacks on frontier models from advanced persistent threat actors, the team identified and disrupted frequent model extraction attacks from private sector entities globally and researchers seeking to clone proprietary logic.

Google’s systems recognised these attacks in real-time and deployed defences to protect internal reasoning traces.

AI-integrated malware emerges

GTIG observed malware samples, tracked as HONESTCUE, that use Gemini’s API to outsource functionality generation. The malware is designed to undermine traditional network-based detection and static analysis through a multi-layered obfuscation approach.

HONESTCUE functions as a downloader and launcher framework that sends prompts via Gemini’s API and receives C# source code as responses. The fileless secondary stage compiles and executes payloads directly in memory, leaving no artefacts on disk.

HONESTCUE malware’s two-stage attack process using Gemini’s API. (Image: Google GTIG)

Separately, GTIG identified COINBAIT, a phishing kit whose construction was likely accelerated by AI code generation tools. The kit, which masquerades as a major cryptocurrency exchange for credential harvesting, was built using the AI-powered platform Lovable AI.

ClickFix campaigns abuse AI chat platforms

In a novel social engineering campaign first observed in December 2025, Google saw threat actors abuse the public sharing features of generative AI services – including Gemini, ChatGPT, Copilot, DeepSeek, and Grok – to host deceptive content distributing ATOMIC malware targeting macOS systems.

Attackers manipulated AI models to create realistic-looking instructions for common computer tasks, embedding malicious command-line scripts as the “solution.” By creating shareable links to these AI chat transcripts, threat actors used trusted domains to host their initial attack stage.

The three-stage ClickFix attack chain exploiting AI chat platforms. (Image: Google GTIG)

Underground marketplace thrives on stolen API keys

GTIG’s observations of English and Russian-language underground forums indicate a persistent demand for AI-enabled tools and services. However, state-sponsored hackers and cybercriminals struggle to develop custom AI models, instead relying on mature commercial products accessed through stolen credentials.

One toolkit, “Xanthorox,” advertised itself as a custom AI for autonomous malware generation and phishing campaign development. GTIG’s investigation revealed Xanthorox was not a bespoke model but actually powered by several commercial AI products, including Gemini, accessed through stolen API keys.

Google’s response and mitigations

Google has taken action against identified threat actors by disabling accounts and assets associated with malicious activity. The company has also applied intelligence to strengthen both classifiers and models, letting them refuse assistance with similar attacks moving forward.\

“We are committed to developing AI boldly and responsibly, which means taking proactive steps to disrupt malicious activity by disabling the projects and accounts associated with bad actors, while continuously improving our models to make them less susceptible to misuse,” the report stated.

GTIG emphasised that despite these developments, no APT or information operations actors have achieved breakthrough abilities that fundamentally alter the threat landscape.

The findings underscore the evolving role of AI in cybersecurity, as both defenders and attackers race to use the technology’s abilities.

For enterprise security teams, particularly in the Asia-Pacific region where Chinese and North Korean state-sponsored hackers remain active, the report serves as an important reminder to enhance defences against AI-augmented social engineering and reconnaissance operations.

(Photo by SCARECROW artworks)

See also: Anthropic just revealed how AI-orchestrated cyberattacks actually work – Here’s what enterprises need to know

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Google identifies state-sponsored hackers using AI in attacks appeared first on AI News.

]]>
Google, Sony Innovation Fund, and Okta back Resemble AI’s push into deepfake detection https://www.artificialintelligence-news.com/news/google-sony-and-okta-back-resemble-ai-push-into-deepfake-detection/ Mon, 08 Dec 2025 14:00:00 +0000 https://www.artificialintelligence-news.com/?p=111181 Resemble AI has raised US$13 million in a new strategic investment round for AI deepfake detection. The funding brings its total venture investment to US$25 million, with participation from Berkeley CalFund, Berkeley Frontier Fund, Comcast Ventures, Craft Ventures, Gentree, Google’s AI Futures Fund, IAG Capital Partners, and others. The funding comes as organisations are under […]

The post Google, Sony Innovation Fund, and Okta back Resemble AI’s push into deepfake detection appeared first on AI News.

]]>
Resemble AI has raised US$13 million in a new strategic investment round for AI deepfake detection. The funding brings its total venture investment to US$25 million, with participation from Berkeley CalFund, Berkeley Frontier Fund, Comcast Ventures, Craft Ventures, Gentree, Google’s AI Futures Fund, IAG Capital Partners, and others.

The funding comes as organisations are under pressure to verify the authenticity of digital content. Generative AI has made it easier for criminals to produce convincing deepfakes, contributing to more than US$1.56 billion in fraud losses in 2025. Analysts estimate that generative AI could enable US$40 billion in fraud losses in the US by 2027.

Recent incidents highlight how quickly threats evolve. In Singapore, 13 individuals collectively lost more than SGD 360,000 after scammers impersonated a telecommunications provider and the Monetary Authority of Singapore. The attackers used caller ID spoofing, voice deepfakes, and social engineering techniques that created urgency and used the public’s trust in government and telecom brands.

Deepfake detection tools and new AI capabilities

Resemble AI develops real-time verification tools that help enterprises detect AI-generated audio, video, images, and text. The company plans to use its new funding to expand global access to its AI deepfake detection platform, which includes two recent releases:

  • DETECT-3B Omni, a deepfake detection model designed for enterprise environments. The company reports 98% detection accuracy in more than 38 languages.
  • Resemble Intelligence, a platform that provides explainability for multimodal and AI-generated content, using Google’s Gemini 3 models.

Resemble AI positions these tools as part of a broader effort to support real-time verification for human users and AI agents interacting with digital content.

According to the company, DETECT-3B Omni is already used in sectors like entertainment, telecommunications, and government. Public benchmark results on Hugging Face show the model ranking among the strongest performers on image and speech deepfake detection, with a lower average error rate than competing models.

Industry stakeholders say the rapid improvement of generative AI is reshaping how enterprises think about content trust and identity systems. Representatives from Google’s AI Futures Fund, Sony Ventures, and Okta noted organisations are moving toward verification layers that can help maintain trust in authentication processes.

Alongside the investment announcement, Resemble AI released its outlook on how deepfake-related risks may evolve in 2026. The company expects several shifts that could shape enterprise planning:

Deepfake verification could become standard for official communications

Following incidents involving government officials, it anticipates real-time deepfake detection may eventually be required for official video conferencing. Such a move would likely create new procurement activity and increase adoption in the public sector.

Organisational readiness may determine competitive positioning

As more jurisdictions introduce AI regulations, enterprises that integrate training, governance, and compliance processes early may find themselves better prepared for operational and regulatory demands.

Identity emerges as a central focus in AI security

With many AI-related attacks relying on impersonation, organisations may place greater emphasis on identity-centric security models, including zero-trust approaches for human and machine identities.

Cyber insurance costs may rise

The growing number of corporate deepfake incidents could lead insurers to reassess their policies on offer. Companies without detection tools could face higher premiums or limited coverage.

The investment underscores the growing need for enterprises to understand how generative AI changes their risk exposure. Organisations in all sectors are evaluating how verification, identity safeguards, and incident readiness can fit into their broader security and compliance strategies.

(Photo by Pau Casals)

See also: AWS re:Invent 2025: Frontier AI agents replace chatbots

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Google, Sony Innovation Fund, and Okta back Resemble AI’s push into deepfake detection appeared first on AI News.

]]>
HTB AI Range offers experiments in cyber-resilience training https://www.artificialintelligence-news.com/news/htb-ai-range-testing-ai-security-in-sandbox-agentic-ai-experiments/ Wed, 03 Dec 2025 14:46:14 +0000 https://www.artificialintelligence-news.com/?p=111135 The cybersecurity training provider Hack The Box (HTB) has launched the HTB AI Range, designed to let organisations test autonomous AI security agents under realistic conditions, albeit with oversight from human cybersecurity professionals. Its goal is to help users assess how well AI, and mixed human–AI teams might defend infrastructure. Vulnerabilities in AI models add […]

The post HTB AI Range offers experiments in cyber-resilience training appeared first on AI News.

]]>
The cybersecurity training provider Hack The Box (HTB) has launched the HTB AI Range, designed to let organisations test autonomous AI security agents under realistic conditions, albeit with oversight from human cybersecurity professionals. Its goal is to help users assess how well AI, and mixed human–AI teams might defend infrastructure.

Vulnerabilities in AI models add to those already present in traditional IT, so before agentic or AI-based cybersecurity tools can be deployed in anger, HTB is proposing a testing environment where AI agents and human defenders can work together under realistic pressure to measure their cybersecurity prowess.

How HTB AI Range works

HTB describes the AI Range as a simulation of enterprise complexity with thousands of offensive and defensive targets that are continuously updated. The platform supports mapping to established cyber frameworks, including MITRE ATT&CK, the NIST/NICE guidelines, and the Open Worldwide Application Security Project (OWASP) Top 10.

HTB says in a recent AI vs. human capture the flag (CTF) exercise, autonomous AI agents solved 19 out of 20 basic challenges. But in multi-step challenges in more complex environments, human teams outperformed the AI agents.

The company suggests AI struggles with complexity and multi-stage operations, and this points to the continuing value of human expertise, especially in high-stakes or complex work.

Testing, and closing the skills gap

Enterprises can use the AI Range to validate whether existing security measures work under AI-powered attacks, give their cybersecurity teams experience of AI-powered threats, and develop more resilient cybersecurity tools based on agentic AI. Such exercises could be used to justify cybersecurity investment to financial decision-makers, Hack The Box suggests.

HTB’s AI Range can be used for continuous testing and validation of cybersecurity defences, which the company states is more effective in the long-term than static audits or pen-testing exercises, and thus is closer to a CTEM model (continuous threat exposure management).

HTB is launching a AI Red Teamer Certification early next year in an attempt quantify the skills necessary to harden AI defences.

At present it seems wise to regard AI cyber-ranges as part of a layered security and resilience offering. As AI matures and frameworks like MITRE ATLAS gain traction, tools like HTB’s AI Range may become standard components in enterprise security programmes.

“Hack The Box is where AI agents and humans learn to operate under real pressure together,” said Gerasimos Marketos, chief product officer at Hack The Box. “We’re addressing the urgent need to continuously validate AI systems in realistic operational contexts where stakes are high and human oversight remains vital. HTB AI Range makes that possible.”

Haris Pylarinos, CEO and founder of Hack The Box said, “For over two years, we’ve been advancing AI-driven learning paths, labs, and research where machines and humans compete, collaborate, and co-evolve. With HTB AI Range, we’re not reacting to AI’s rise in cyber; we’re defining how defence evolves alongside it. This is how cybersecurity advances: not through fear, but through mastery.”

(Image source: “The main cast” by Tim Dorr is licensed under CC BY-SA 2.0.)

See also: New Nvidia Blackwell chip for China may outpace H20 model

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post HTB AI Range offers experiments in cyber-resilience training appeared first on AI News.

]]>
Anthropic just revealed how AI-orchestrated cyberattacks actually work—Here’s what enterprises need to know https://www.artificialintelligence-news.com/news/ai-orchestrated-cyberattacks-anthropic-discovery/ Wed, 03 Dec 2025 10:00:00 +0000 https://www.artificialintelligence-news.com/?p=111093 For years, cybersecurity experts debated when – not if – artificial intelligence would cross the threshold from advisor to autonomous attacker. That theoretical milestone has arrived. Anthropic’s recent investigation into a Chinese state-sponsored operation has documented [PDF] the first case of AI-orchestrated cyber attacks executing at scale with minimal human oversight, altering what enterprises must […]

The post Anthropic just revealed how AI-orchestrated cyberattacks actually work—Here’s what enterprises need to know appeared first on AI News.

]]>
For years, cybersecurity experts debated when – not if – artificial intelligence would cross the threshold from advisor to autonomous attacker. That theoretical milestone has arrived.

Anthropic’s recent investigation into a Chinese state-sponsored operation has documented [PDF] the first case of AI-orchestrated cyber attacks executing at scale with minimal human oversight, altering what enterprises must prepare for in the threat landscape ahead.

The campaign, attributed to a group Anthropic designates as GTG-1002, represents what security researchers have long warned about but never actually witnessed in the wild: an AI system autonomously conducting nearly every phase of cyber intrusion – from initial reconnaissance to data exfiltration – while human operators merely supervised strategic checkpoints.

This isn’t incremental evolution but a shift in offensive capabilities that compresses what would take skilled hacking teams weeks into operations measured in hours, executed at machine speed on dozens of targets simultaneously.

The numbers tell the story. Anthropic’s forensic analysis revealed that 80 to 90% of GTG-1002’s tactical operations ran autonomously, with humans intervening at just four to six critical decision points per campaign.

The operation targeted approximately 30 entities – major technology corporations, financial institutions, chemical manufacturers, and government agencies – achieving confirmed breaches of several high-value targets. At peak activity, the AI system generated thousands of requests at rates of multiple operations per second, a tempo physically impossible for human teams to sustain.

Anatomy of an autonomous breach

The technical architecture behind these AI-orchestrated cyber attacks reveals a sophisticated understanding of both AI capabilities and safety bypass techniques.

GTG-1002 built an autonomous attack framework around Claude Code, Anthropic’s coding assistance tool, integrated with Model Context Protocol (MCP) servers that provided interfaces to standard penetration testing utilities – network scanners, database exploitation frameworks, password crackers, and binary analysis suites.

The breakthrough wasn’t in novel malware development but in orchestration. The attackers manipulated Claude through carefully constructed social engineering, convincing the AI it was conducting legitimate defensive security testing for a cybersecurity firm.

They decomposed complex multi-stage attacks into discrete, seemingly innocuous tasks – vulnerability scanning, credential validation, data extraction – each appearing legitimate when evaluated in isolation, preventing Claude from recognising the broader malicious context.

Once operational, the framework demonstrated remarkable autonomy.

In one documented compromise, Claude independently discovered internal services in a target network, mapped complete network topology in multiple IP ranges, identified high-value systems including databases and workflow orchestration platforms, researched and wrote custom exploit code, validated vulnerabilities through callback communication systems, harvested credentials, tested them systematically in discovered infrastructure, and analysed/stolen data to categorise findings by intelligence value – all without step-by-step human direction.

The AI maintained a persistent operational context in sessions spanning days, letting campaigns resume seamlessly after interruptions.

It made autonomous targeting decisions based on discovered infrastructure, adapted exploitation techniques when initial approaches failed, and generated comprehensive documentation throughout all phases – structured markdown files tracking discovered services, harvested credentials, extracted data, and complete attack progression.

What this means for enterprise security

The GTG-1002 campaign dismantles several foundational assumptions that have shaped enterprise security strategies. Traditional defences calibrated around human attacker limitations – rate limiting, behavioural anomaly detection, operational tempo baselines – face an adversary operating at machine speed with machine endurance.

The economics of cyber attacks have shifted dramatically, as 80-90% of tactical work can be automated, potentially bringing nation-state-level capabilities in reach of less sophisticated threat actors.

Yet AI-orchestrated cyber attacks face inherent limitations that enterprise defenders should understand. Anthropic’s investigation documented frequent AI hallucinations during operations – Claude claiming to have obtained credentials that didn’t function, identifying “critical discoveries” that proved to be publicly available information, and overstating findings that required human validation.

The reliability issues remain a significant friction point for fully autonomous operations, though assuming they’ll persist indefinitely would be dangerously naive as AI capabilities continue advancing.

The defensive imperative

The dual-use reality of advanced AI presents both challenge and opportunity. The same capabilities enabling GTG-1002’s operation proved essential for defence – Anthropic’s Threat Intelligence team relied heavily on Claude to analyse the massive data volumes generated during their investigation.

Building organisational experience with what works in specific environments – understanding AI’s strengths and limitations in defensive contexts – becomes important before the next wave of more sophisticated autonomous attacks arrives.

Anthropic’s disclosure signals an inflexion point. As AI models advance and threat actors refine autonomous attack frameworks, the question isn’t whether AI-orchestrated cyber attacks will proliferate in the threat landscape – it’s whether enterprise defences can evolve rapidly enough to counter them.

The window for preparation, while still open, is narrowing faster than many security leaders may realise.

See also: New Nvidia Blackwell chip for China may outpace H20 model

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Anthropic just revealed how AI-orchestrated cyberattacks actually work—Here’s what enterprises need to know appeared first on AI News.

]]>
Adversarial learning breakthrough enables real-time AI security https://www.artificialintelligence-news.com/news/adversarial-learning-breakthrough-real-time-ai-security/ Tue, 25 Nov 2025 14:12:05 +0000 https://www.artificialintelligence-news.com/?p=110918 The ability to execute adversarial learning for real-time AI security offers a decisive advantage over static defence mechanisms. The emergence of AI-driven attacks – utilising reinforcement learning (RL) and Large Language Model (LLM) capabilities – has created a class of “vibe hacking” and adaptive threats that mutate faster than human teams can respond. This represents […]

The post Adversarial learning breakthrough enables real-time AI security appeared first on AI News.

]]>
The ability to execute adversarial learning for real-time AI security offers a decisive advantage over static defence mechanisms.

The emergence of AI-driven attacks – utilising reinforcement learning (RL) and Large Language Model (LLM) capabilities – has created a class of “vibe hacking” and adaptive threats that mutate faster than human teams can respond. This represents a governance and operational risk for enterprise leaders that policy alone cannot mitigate.

Attackers now employ multi-step reasoning and automated code generation to bypass established defences. Consequently, the industry is observing a necessary migration toward “autonomic defence” (i.e. systems capable of learning, anticipating, and responding intelligently without human intervention.)

Transitioning to these sophisticated defence models, though, has historically hit a hard operational ceiling: latency.

Applying adversarial learning, where threat and defence models are trained continuously against one another, offers a method for countering malicious AI security threats. Yet, deploying the necessary transformer-based architectures into a live production environment creates a bottleneck.

Abe Starosta, Principal Applied Research Manager at Microsoft NEXT.ai, said: “Adversarial learning only works in production when latency, throughput, and accuracy move together. 

Computational costs associated with running these dense models previously forced leaders to choose between high-accuracy detection (which is slow) and high-throughput heuristics (which are less accurate).

Engineering collaboration between Microsoft and NVIDIA shows how hardware acceleration and kernel-level optimisation remove this barrier, making real-time adversarial defence viable at enterprise scale.

Operationalising transformer models for live traffic required the engineering teams to target the inherent limitations of CPU-based inference. Standard processing units struggle to handle the volume and velocity of production workloads when burdened with complex neural networks.

In baseline tests conducted by the research teams, a CPU-based setup yielded an end-to-end latency of 1239.67ms with a throughput of just 0.81req/s. For a financial institution or global e-commerce platform, a one-second delay on every request is operationally untenable.

By transitioning to a GPU-accelerated architecture (specifically utilising NVIDIA H100 units), the baseline latency dropped to 17.8ms. Hardware upgrades alone, though, proved insufficient to meet the strict requirements of real-time AI security.

Through further optimisation of the inference engine and tokenisation processes, the teams achieved a final end-to-end latency of 7.67ms—a 160x performance speedup compared to the CPU baseline. Such a reduction brings the system well within the acceptable thresholds for inline traffic analysis, enabling the deployment of detection models with greater than 95 percent accuracy on adversarial learning benchmarks.

One operational hurdle identified during this project offers valuable insight for CTOs overseeing AI integration. While the classifier model itself is computationally heavy, the data pre-processing pipeline – specifically tokenisation – emerged as a secondary bottleneck.

Standard tokenisation techniques, often relying on whitespace segmentation, are designed for natural language processing (e.g. articles and documentation). They prove inadequate for cybersecurity data, which consists of densely packed request strings and machine-generated payloads that lack natural breaks.

To address this, the engineering teams developed a domain-specific tokeniser. By integrating security-specific segmentation points tailored to the structural nuances of machine data, they enabled finer-grained parallelism. This bespoke approach for security delivered a 3.5x reduction in tokenisation latency, highlighting that off-the-shelf AI components often require domain-specific re-engineering to function effectively in niche environments.

Achieving these results required a cohesive inference stack rather than isolated upgrades. The architecture utilised NVIDIA Dynamo and Triton Inference Server for serving, coupled with a TensorRT implementation of Microsoft’s threat classifier.

The optimisation process involved fusing key operations – such as normalisation, embedding, and activation functions – into single custom CUDA kernels. This fusion minimises memory traffic and launch overhead, which are frequent silent killers of performance in high-frequency trading or security applications. TensorRT automatically fused normalisation operations into preceding kernels, while developers built custom kernels for sliding window attention.

The result of these specific inference optimisations was a reduction in forward-pass latency from 9.45ms to 3.39ms, a 2.8x speedup that contributed the majority of the latency reduction seen in the final metrics.

Rachel Allen, Cybersecurity Manager at NVIDIA, explained: “Securing enterprises means matching the volume and velocity of cybersecurity data and adapting to the innovation speed of adversaries.

“Defensive models need the ultra-low latency to run at line-rate and the adaptability to protect against the latest threats. The combination of adversarial learning with NVIDIA TensorRT accelerated transformer-based detection models does just that.”

Success here points to a broader requirement for enterprise infrastructure. As threat actors leverage AI to mutate attacks in real-time, security mechanisms must possess the computational headroom to run complex inference models without introducing latency.

Reliance on CPU compute for advanced threat detection is becoming a liability. Just as graphics rendering moved to GPUs, real-time security inference requires specialised hardware to maintain throughput >130 req/s while ensuring robust coverage.

Furthermore, generic AI models and tokenisers often fail on specialised data. The “vibe hacking” and complex payloads of modern threats require models trained specifically on malicious patterns and input segmentations that reflect the reality of machine data.

Looking ahead, the roadmap for future security involves training models and architectures specifically for adversarial robustness, potentially using techniques like quantisation to further enhance speed.

By continuously training threat and defence models in tandem, organisations can build a foundation for real-time AI protection that scales with the complexity of evolving security threats. The adversarial learning breakthrough demonstrates the technology to achieve this – balancing latency, throughput, and accuracy – is now capable of being deployed today.

See also: ZAYA1: AI model using AMD GPUs for training hits milestone

Banner for AI & Big Data Expo by TechEx events.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security Expo. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Adversarial learning breakthrough enables real-time AI security appeared first on AI News.

]]>
Anthropic details cyber espionage campaign orchestrated by AI https://www.artificialintelligence-news.com/news/anthropic-details-cyber-espionage-campaign-orchestrated-by-ai/ Fri, 14 Nov 2025 11:34:00 +0000 https://www.artificialintelligence-news.com/?p=110566 Security leaders face a new class of autonomous threat as Anthropic details the first cyber espionage campaign orchestrated by AI. In a report released this week, the company’s Threat Intelligence team outlined its disruption of a sophisticated operation by a Chinese state-sponsored group – an assessment made with high confidence – dubbed GTG-1002 and detected […]

The post Anthropic details cyber espionage campaign orchestrated by AI appeared first on AI News.

]]>
Security leaders face a new class of autonomous threat as Anthropic details the first cyber espionage campaign orchestrated by AI.

In a report released this week, the company’s Threat Intelligence team outlined its disruption of a sophisticated operation by a Chinese state-sponsored group – an assessment made with high confidence – dubbed GTG-1002 and detected in mid-September 2025.

The operation targeted approximately 30 entities, including large tech companies, financial institutions, chemical manufacturing companies, and government agencies.

Rather than AI assisting human operators, the attackers successfully manipulated Anthropic’s Claude Code model to function as an autonomous agent to execute the vast majority of tactical operations independently.

This marks a worrying development for CISOs, moving cyber attacks from human-directed efforts to a model where AI agents perform 80-90 percent of the offensive work with humans acting only as high-level supervisors. Anthropic believes this is the first documented case of a large-scale cyberattack executed without substantial human intervention.

AI agents: A new operational model for cyberattacks

The group used an orchestration system that tasked instances of Claude Code to function as autonomous penetration testing agents. These AI agents were directed as part of the espionage campaign to perform reconnaissance, discover vulnerabilities, develop exploits, harvest credentials, move laterally across networks, and exfiltrate data. This enabled the AI to perform reconnaissance in a fraction of the time it would have taken a team of human hackers.

Human involvement was limited to 10-20 percent of the total effort, primarily focused on campaign initiation and providing authorisation at a few key escalation points. For example, human operators would approve the transition from reconnaissance to active exploitation or authorise the final scope of data exfiltration.

The attackers bypassed the AI model’s built-in safeguards, which are trained to avoid harmful behaviours. They did this by jailbreaking the model, tricking it by breaking down attacks into seemingly innocent tasks and by adopting a “role-play” persona. Operators told Claude that it was an employee of a legitimate cybersecurity firm and was being used in defensive testing. This allowed the operation to proceed long enough to gain access to a handful of validated targets.

The technical sophistication of the attack lay not in novel malware, but in orchestration. The report notes the framework relied “overwhelmingly on open-source penetration testing tools”. The attackers used Model Context Protocol (MCP) servers as an interface between the AI and these commodity tools, enabling the AI to execute commands, analyse results, and maintain operational state across multiple targets and sessions. The AI was even directed to research and write its own exploit code for the espionage campaign.

AI hallucinations become a good thing

While the campaign successfully breached high-value targets, Anthropic’s investigation uncovered a noteworthy limitation: the AI hallucinated during offensive operations.

The report states that Claude “frequently overstated findings and occasionally fabricated data”. This manifested as the AI claiming to have obtained credentials that did not work or identifying discoveries that “proved to be publicly available information.”

This tendency required the human operators to carefully validate all results, presenting challenges for the attackers’ operational effectiveness. According to Anthropic, this “remains an obstacle to fully autonomous cyberattacks”. For security leaders, this highlights a potential weakness in AI-driven attacks: they may generate a high volume of noise and false positives that can be identified with robust monitoring.

A defensive AI arms race against new cyber espionage threats

The primary implication for business and technology leaders is that the barriers to performing sophisticated cyberattacks have dropped considerably. Groups with fewer resources may now be able to execute campaigns that previously required entire teams of experienced hackers.

This attack demonstrates a capability beyond “vibe hacking,” where humans remained firmly in control of operations. The GTG-1002 campaign proves that AI can be used to autonomously discover and exploit vulnerabilities in live operations.

Anthropic, which banned the accounts and notified authorities over a ten-day investigation, argues that this development shows the urgent need for AI-powered defence. The company states that “the very abilities that allow Claude to be used in these attacks also make it essential for cyber defense”. The company’s own Threat Intelligence team “used Claude extensively to analyse “the enormous amounts of data generated” during this investigation.

Security teams should operate under the assumption that a major change has occurred in cybersecurity. The report urges defenders to “experiment with applying AI for defense in areas like SOC automation, threat detection, vulnerability assessment, and incident response.”

The contest between AI-driven attacks and AI-powered defence has begun, and proactive adaptation to counter new espionage threats is the only viable path forward.

See also: Wiz: Security lapses emerge amid the global AI race

Banner for AI & Big Data Expo by TechEx events.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security Expo. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Anthropic details cyber espionage campaign orchestrated by AI appeared first on AI News.

]]>
Google reveals its own version of Apple’s AI cloud https://www.artificialintelligence-news.com/news/google-reveals-its-own-version-of-apple-ai-cloud/ Wed, 12 Nov 2025 09:00:00 +0000 https://www.artificialintelligence-news.com/?p=110521 Google has rolled out Private AI Compute, a new cloud-based processing system designed to bring the privacy of on-device AI to the cloud. The platform aims to give users faster, more capable AI experiences without compromising data security. It combines Google’s most advanced Gemini models with strict privacy safeguards, reflecting the company’s ongoing effort to […]

The post Google reveals its own version of Apple’s AI cloud appeared first on AI News.

]]>
Google has rolled out Private AI Compute, a new cloud-based processing system designed to bring the privacy of on-device AI to the cloud. The platform aims to give users faster, more capable AI experiences without compromising data security. It combines Google’s most advanced Gemini models with strict privacy safeguards, reflecting the company’s ongoing effort to make AI both powerful and responsible.

The feature closely resembles Apple’s Private Cloud Compute, signalling how major tech firms are rethinking privacy in the age of large-scale AI. Both companies are trying to balance two competing needs — the huge computing power required to run advanced AI models and users’ expectations for data privacy.

Why Google built Private AI Compute

As AI systems get smarter, they’re also becoming more personal. What started as tools that completed simple tasks or answered direct questions are now systems that can anticipate user needs, suggest actions, and handle complex processes in real time. That kind of intelligence demands a level of reasoning and computation that often exceeds what’s possible on a single device.

Private AI Compute bridges that gap. It lets Gemini models in the cloud process data faster and more efficiently while ensuring that sensitive information remains private and inaccessible to anyone else — not even Google engineers. Google describes it as combining the power of cloud AI with the security users expect from local processing.

In practical terms, this means you could get quicker responses, smarter suggestions, and more personalised results without your personal data ever leaving your control.

How Private AI Compute keeps data secure

Google claims the new platform is based on the same principles that underpin its broader AI and privacy strategy: giving users control, maintaining security, and earning trust. The system acts as a protected computing environment, isolating data so it can be processed safely and privately.

It uses a multi-layered design centred on three key components:

  • Unified Google tech stack: Private AI Compute runs entirely on Google’s own infrastructure, powered by custom Tensor Processing Units (TPUs). It’s secured through Titanium Intelligence Enclaves (TIE), which create an additional layer of protection for data processed in the cloud.
  • Encrypted connections: Before data is sent for processing, remote attestation and encryption verify that it’s connecting to a trusted, hardware-secured environment. Once inside this sealed cloud space, information stays private to the user.
  • Zero access assurance: Google says the system is designed so that no one — not even the company itself — can access the data processed within Private AI Compute.

This design builds on Google’s Secure AI Framework (SAIF), AI Principles, and Privacy Principles, which outline how the company develops and deploys AI responsibly.

What users can expect

Private AI Compute also improves the performance of AI features that are already running on devices. Magic Cue on the Pixel 10 can now offer more relevant and timely suggestions by leveraging cloud-level processing power. Similarly, the Recorder app can use the system to summarise transcriptions across a wider range of languages — something that would be difficult to do entirely on-device.

These examples hint at what’s ahead. With Private AI Compute, Google can deliver AI experiences that combine the privacy of local models with the intelligence of cloud-based ones. It’s an approach that could eventually apply to everything from personal assistants and photo organisation to productivity and accessibility tools.

Google calls this launch “just the beginning.” The company says Private AI Compute opens the door to a new generation of AI tools that are both more capable and more private. As AI becomes increasingly woven into everyday tasks, users are demanding greater transparency and control over how their data is used — and Google appears to be positioning this technology as part of that answer.

For those interested in the technical details, Google has published a technical brief explaining how Private AI Compute works and how it fits into the company’s larger vision for responsible AI development.

(Photo by Solen Feyissa)

See also: Apple plans big Siri update with help from Google AI

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Google reveals its own version of Apple’s AI cloud appeared first on AI News.

]]>
Google’s new AI agent rewrites code to automate vulnerability fixes https://www.artificialintelligence-news.com/news/google-new-ai-agent-rewrites-code-automate-vulnerability-fixes/ Mon, 06 Oct 2025 13:56:40 +0000 https://www.artificialintelligence-news.com/?p=109743 Google DeepMind has deployed a new AI agent designed to autonomously find and fix critical security vulnerabilities in software code. The system, aptly-named CodeMender, has already contributed 72 security fixes to established open-source projects in the last six months. Identifying and patching vulnerabilities is a notoriously difficult and time-consuming process, even with the aid of […]

The post Google’s new AI agent rewrites code to automate vulnerability fixes appeared first on AI News.

]]>
Google DeepMind has deployed a new AI agent designed to autonomously find and fix critical security vulnerabilities in software code. The system, aptly-named CodeMender, has already contributed 72 security fixes to established open-source projects in the last six months.

Identifying and patching vulnerabilities is a notoriously difficult and time-consuming process, even with the aid of traditional automated methods like fuzzing. Google DeepMind’s own research, including AI-based projects such as Big Sleep and OSS-Fuzz, has proven effective at discovering new zero-day vulnerabilities in well-audited code. This success, however, creates a new bottleneck: as AI accelerates the discovery of flaws, the burden on human developers to fix them intensifies.

CodeMender is engineered to address this imbalance. It functions as an autonomous AI agent that takes a comprehensive approach to fix code security. Its capabilities are both reactive, allowing it to patch newly discovered vulnerabilities instantly, and proactive, enabling it to rewrite existing code to eliminate entire classes of security flaws before they can be exploited. This allows human developers and project maintainers to dedicate more of their time to building features and improving software functionality.

The system operates by leveraging the advanced reasoning capabilities of Google’s recent Gemini Deep Think models. This foundation allows the agent to debug and resolve complex security issues with a high degree of autonomy. To achieve this, the system is equipped with a set of tools that permit it to analyse and reason about code before implementing any changes. CodeMender also includes a validation process to ensure any modifications are correct and do not introduce new problems, known as regressions.

While large language models are advancing rapidly, a mistake when it comes to code security can have costly consequences. CodeMender’s automatic validation framework is therefore essential. It systematically checks that any proposed changes fix the root cause of an issue, are functionally correct, do not break existing tests, and adhere to the project’s coding style guidelines. Only high-quality patches that satisfy these stringent criteria are surfaced for human review.

To enhance its code fixing effectiveness, the DeepMind team developed new techniques for the AI agent. CodeMender employs advanced program analysis, utilising a suite of tools including static and dynamic analysis, differential testing, fuzzing, and SMT solvers. These instruments allow it to systematically scrutinise code patterns, control flow, and data flow to identify the fundamental causes of security flaws and architectural weaknesses.

The system also uses a multi-agent architecture, where specialised agents are deployed to tackle specific aspects of a problem. For example, a dedicated large language model-based critique tool reveals the differences between original and modified code. This allows the primary agent to verify that its proposed changes do not introduce unintended side effects and to self-correct its approach when necessary.

In one practical example, CodeMender addressed a vulnerability where a crash report indicated a heap buffer overflow. Although the final patch only required changing a few lines of code, the root cause was not immediately obvious. By using a debugger and code search tools, the agent determined the true problem was an incorrect stack management issue with Extensible Markup Language (XML) elements during parsing, located elsewhere in the codebase. In another case, the agent devised a non-trivial patch for a complex object lifetime issue, modifying a custom system for generating C code within the target project.

Beyond simply reacting to existing bugs, CodeMender is designed to proactively harden software against future threats. The team deployed the agent to apply -fbounds-safety annotations to parts of libwebp, a widely used image compression library. These annotations instruct the compiler to add bounds checks to the code, which can prevent an attacker from exploiting a buffer overflow to execute arbitrary code.

This work is particularly relevant given that a heap buffer overflow vulnerability in libwebp, tracked as CVE-2023-4863, was used by a threat actor in a zero-click iOS exploit several years ago. DeepMind notes that with these annotations in place, that specific vulnerability, along with most other buffer overflows in the annotated sections, would have been rendered unexploitable.

The AI agent’s proactive code fixing involves a sophisticated decision-making process. When applying annotations, it can automatically correct new compilation errors and test failures that arise from its own changes. If its validation tools detect that a modification has broken functionality, the agent self-corrects based on the feedback and attempts a different solution.

Despite these promising early results, Google DeepMind is taking a cautious and deliberate approach to deployment, with a strong focus on reliability. At present, every patch generated by CodeMender is reviewed by human researchers before being submitted to an open-source project. The team is gradually increasing its submissions to ensure high quality and to systematically incorporate feedback from the open-source community.

Looking ahead, the researchers plan to reach out to maintainers of critical open-source projects with CodeMender-generated patches. By iterating on community feedback, they hope to eventually release CodeMender as a publicly available tool for all software developers.

The DeepMind team also intends to publish technical papers and reports in the coming months to share their techniques and results. This work represents the first steps in exploring the potential of AI agents to proactively fix code and fundamentally enhance software security for everyone.

See also: CAMIA privacy attack reveals what AI models memorise

Banner for AI & Big Data Expo by TechEx events.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events including the Cyber Security Expo, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Google’s new AI agent rewrites code to automate vulnerability fixes appeared first on AI News.

]]>
Ethical cybersecurity practice reshapes enterprise security in 2025 https://www.artificialintelligence-news.com/news/manageengine-ethical-cybersecurity-2025/ Fri, 26 Sep 2025 08:20:45 +0000 https://www.artificialintelligence-news.com/?p=109598 When ransomware attacks like Akira and Ryuk began crippling organisations worldwide, the cybersecurity industry’s first instinct was predictable: build bigger walls, deploy more aggressive automated responses, and lock down everything. But there was a different problem emerging, according to Romanus Prabhu Raymond, Director of Technology at ManageEngine. The company’s customers were demanding aggressive containment features, […]

The post Ethical cybersecurity practice reshapes enterprise security in 2025 appeared first on AI News.

]]>
When ransomware attacks like Akira and Ryuk began crippling organisations worldwide, the cybersecurity industry’s first instinct was predictable: build bigger walls, deploy more aggressive automated responses, and lock down everything. But there was a different problem emerging, according to Romanus Prabhu Raymond, Director of Technology at ManageEngine.

The company’s customers were demanding aggressive containment features, yet automatically quarantining a suspicious hospital computer or bank teller system might prove more devastating than the original threat. The dilemma – balancing rapid threat response with real-world consequences – exemplifies why ethical cybersecurity practices have become one of the defining challenges of 2025.

In our exclusive interview shortly before his presentation at the Cyber Security Expo in Amsterdam, Raymond revealed how leading organisations are breaking free from the traditional security-versus-privacy trade-off and why the companies embracing this “trust revolution” can reshape enterprise security.

For starters, the cybersecurity industry stands at a important juncture. High-profile breaches, evolving regulatory frameworks, and the rapid integration of AI into security systems have created new challenges that extend far beyond technical protection. Organisations now face important questions about how to balance innovation with responsibility, privacy with security, and automation with human oversight.

Defining ethical cybersecurity in the modern era

According to Raymond, ethical cybersecurity transcends traditional notions of defence. “Ethical cybersecurity goes beyond defending systems and data – it’s about applying security practices responsibly to protect organisations, individuals, and society at large,” he explained during our interview ahead of his presentation.

In 2025’s cloud-first environment, security isn’t a competitive differentiator, but a baseline expectation. What distinguishes organisations today is how ethically they handle data and implement security measures.

Raymond uses the analogy of installing security cameras in a neighbourhood to protect public spaces without intruding on private areas; the avoidance of peering into residents’ windows. Cybersecurity must operate under the same principle.

ManageEngine has operationalised this philosophy through what Raymond calls an “ethical by design” approach, embedding fairness, transparency, and accountability into every product from conception. The company’s stance on customer data exemplifies this commitment: it neither monetises nor monitors customer data, maintaining that it belongs solely to the customer.

The innovation-risk paradox

The tension between innovation and risk management represents an important challenge for modern organisations. Push too hard for innovation without adequate safeguards and companies risk data breaches and compliance violations. Focus too heavily on risk mitigation, and organisations may find themselves unable to compete in evolving markets.

The “trust by design” philosophy embeds responsibility and accountability into every development stage, which allows rapid innovation and maintains compliance and ethical standards. When deploying important components like endpoint agents, the company ensures new functionality inherently complies with industry standards and security requirements.

The method extends to the company’s global operations. ManageEngine maintains datacentres worldwide which align with local privacy and regulatory demands, and trains every employee – from developers to support engineers – to treat customer data with integrity. The company’s “trans-localisation strategy” ensures local teams serve local customers, creating operational efficiency and cultural trust.

AI integration and human oversight

As artificial intelligence becomes increasingly central to cybersecurity operations, the ethical implications of AI-driven security solutions have become more complex. Raymond acknowledges that AI is evolving from purely assistive roles to more decisive functions, raising questions about accountability, transparency, and fairness.

Raymond expounds ManageEngine’s “SHE AI principles”: Secure AI, Human AI, and Ethical AI. Secure AI involves building robust protections against manipulation and adversarial attacks. Human AI ensures human oversight remains integral to important security actions—for instance, if AI detects a suspicious endpoint, it escalates for human validation rather than automatically removing the device from the network.

This is particularly important in sensitive environments like hospitals or banks, where automatically blocking systems could have severe consequences.

The ethical AI component emphasises explainability. Rather than generating “black box” alerts, ManageEngine’s systems explain their reasoning. An alert might read: “The endpoint cannot log in at this time and is trying to connect to too many network devices.” This transparency is essential for compliance and building trust in AI-driven security systems.

Navigating privacy-security trade-offs

The balance between necessary security monitoring and privacy invasion represents one of the most delicate aspects of ethical cybersecurity practices. Raymond acknowledges that while proactive monitoring is essential for detecting threats early, over-monitoring risks creating a surveillance environment that treats employees as suspects rather than trusted partners.

ManageEngine uses principles that emphasise data minimisation, purpose-driven monitoring, anonymisation, and clear governance structures. The company collects only information necessary for security purposes, ensures every piece of data has a defined security use case, uses anonymised data for pattern analysis, and defines data access privileges and retention periods.

The framework demonstrates that security and privacy need not be mutually exclusive when guided by ethics, transparency, and accountability.

Industry leadership and future challenges

Raymond argues that technology vendors must act as custodians of digital ethics, earning trust rather than expecting it to be given blindly. ManageEngine says it contributes to industry standards by thought leadership, advocacy, and by embedding compliance standards like ISO 27000 and GDPR into products from the start.

Raymond identifies AI-driven autonomous security and quantum computing as the biggest ethical challenges facing the industry. As security operations centres move toward full autonomy, questions of explainability and accountability become critical. Quantum computing’s ability to break traditional encryption threatens secure communication foundations, while technologies like biometrics raise privacy concerns if not managed carefully.

Practical implementation

For organisations seeking to integrate ethical considerations into their cybersecurity strategies, Raymond recommends three concrete steps: adopting a cybersecurity ethics charter at the board level, embedding privacy and ethics in technology decisions when selecting vendors, and operationalising ethics through comprehensive training and controls that explain not just what to do, but why it matters.

As the cybersecurity landscape evolves, companies that will thrive are those that recognise ethical cybersecurity practices as the foundation for sustainable, trusted technological advancement, not as constraints on innovation. In the future organisations have to innovate responsibly and maintain human oversight and the ethical principles that digital trust requires.

See also: CERTAIN drives ethical AI compliance in Europe

Banner for the AI & Big Data Expo event series.

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Ethical cybersecurity practice reshapes enterprise security in 2025 appeared first on AI News.

]]>
Governing the age of agentic AI: Balancing autonomy and accountability   https://www.artificialintelligence-news.com/news/governing-the-age-of-agentic-ai-balancing-autonomy-and-accountability/ Wed, 24 Sep 2025 08:19:42 +0000 https://www.artificialintelligence-news.com/?p=109538 Author: Rodrigo Coutinho, Co-Founder and AI Product Manager at OutSystems AI has moved beyond pilot projects and future promises. Today, it’s embedded in industries, with more than three-quarters of organisations (78%) now using AI in at least one business function. The next leap, however, is agentic AI: systems that don’t just provide insights or automate […]

The post Governing the age of agentic AI: Balancing autonomy and accountability   appeared first on AI News.

]]>
Author: Rodrigo Coutinho, Co-Founder and AI Product Manager at OutSystems

AI has moved beyond pilot projects and future promises. Today, it’s embedded in industries, with more than three-quarters of organisations (78%) now using AI in at least one business function. The next leap, however, is agentic AI: systems that don’t just provide insights or automate narrow tasks but operate as autonomous agents, capable of adapting to changing inputs, connecting with other systems, and influencing business-critical decisions. Although these agents will deliver greater value, agentic AI also poses challenges.

Imagine agents that proactively resolve customer issues in real-time or adapt applications dynamically to meet shifting business priorities. The greater autonomy inevitably brings new risks. Without the right safeguards, AI agents may drift from their intended purpose or make choices that clash with business rules, regulations, or ethical standards. Navigating this new era requires stronger oversight, where human judgement, governance frameworks, and transparency are built-in from the start. The potential of agentic AI is vast but so are the obligations that come with deployment. Low-code platforms offer one path forward, serving as a control layer between autonomous agents and enterprise systems. By embedding governance and compliance into development, they give organisations the confidence that AI-driven processes will advance strategic goals without adding unnecessary risk.

Designing safeguards instead of code for agentic AI

Agentic AI marks a steep change in how people interact with software. It’s indicative of a fundamental shift in the relationship between people and software. Traditionally, developers have focused on building applications with clear requirements and predictable outputs. Now, instead of fragmented applications, teams will orchestrate entire ecosystems of agents that interact with people, systems and data. 

As these systems mature, developers shift from writing code line by line to defining the safeguards that steer them. Because these agents adapt and may respond differently to the same input, transparency and accountability must be built in from the start. By embedding oversight and compliance into design, developers ensure AI-driven decisions stay reliable, explainable and aligned with business goals. The change demands that developers and IT leaders embrace a broader supervisor role, guiding both technological and organisational change over time. 

Why transparency and control matter in agentic AI

Greater autonomy exposes organisations to additional vulnerabilities. According to a recent OutSystems study, 64% of technology leaders cite governance, trust and safety as top concerns when deploying AI agents at scale. Without strong safeguards, these risks extend beyond compliance gaps to include security breaches and reputational damage. Opacity in agentic systems makes it difficult for leaders to understand or validate decisions, eroding confidence internally and with customers, leading to concrete risks.

Left unchecked, autonomous agents can blur accountability, widen the attack surface and create inconsistency at scale. Without visibility into why an AI system acts, organisations risk losing accountability in critical workflows. At the same time, agents that interact in sensitive data and systems expand the attack surface for cyber threats, while un-monitored “agent sprawl” can create redundancy, fragmentation and inconsistent decisions. Together, these challenges underscore the need for strong governance frameworks that maintain trust and control as autonomy scales. 

Scaling AI safely with low-code foundations

Crucially, adopting agentic AI need not involve rebuilding governance from the ground up. Organisations have multiple approaches available to them, including low-code platforms, which offer a reliable, scalable framework where security, compliance and governance are already part of the development fabric.

Across enterprises, IT teams are being asked to embed agents into operations without disrupting what already works. With the right frameworks, IT teams can deploy AI agents directly into enterprise-wide operations without disrupting current workflows or re-architecting core systems. Organisations have full control over how AI agents operate at every step, ultimately building trust to scale confidently in the enterprise.

Low-code places governance, security and scalability at the heart of AI adoption. By unifying app and agent development in a single environment, it is easier to embed compliance and oversight from the start. The ability to integrate seamlessly in enterprise systems, combined with built-in DevSecOps practices, ensures that vulnerabilities are addressed before deployment. And with out-of-the-box infrastructure, organisations can scale confidently without having to reinvent foundational elements of governance or security.

The approach lets organisations pilot and scale agentic AI while keeping compliance and security intact. Low-code makes it easier to deliver with speed and security, giving developers and IT leaders confidence to progress.

Smarter oversight for smarter systems

Ultimately, low-code provides a dependable route to scaling autonomous AI while preserving trust. By unifying app and agent development in one environment, low-code embeds compliance and oversight from the start. Seamless integration in systems and built-in DevSecOps practices help address vulnerabilities before deployment, while ready-made infrastructure enables scale without reinventing governance from scratch. For developers and IT leaders, this shift means moving beyond writing code to guiding the rules and safeguards that shape autonomous systems. In a fast-changing landscape, low-code provides the flexibility and resilience needed to experiment confidently, embrace innovation early, and maintain trust as AI grows more autonomous.

Author: Rodrigo Coutinho, Co-Founder and AI Product Manager at OutSystems

(Image by Alexandra_Koch)

See also: Agentic AI: Promise, scepticism, and its meaning for Southeast Asia

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Governing the age of agentic AI: Balancing autonomy and accountability   appeared first on AI News.

]]>
AI-enabled threats and stricter regulation in France https://www.artificialintelligence-news.com/news/ai-enabled-threats-and-stricter-regulation-in-france/ Wed, 17 Sep 2025 11:58:52 +0000 https://www.artificialintelligence-news.com/?p=109441 A new research report from technology advisory firm Information Services Group (ISG) has revealed AI threats and more stringent regulations are shifting the French cybersecurity landscape, resulting in businesses reassessing their security strategies. Increasing security budgets mean many French enterprises require fresh guidance and expertise to establish effective priorities and combat their security challenges. According […]

The post AI-enabled threats and stricter regulation in France appeared first on AI News.

]]>
A new research report from technology advisory firm Information Services Group (ISG) has revealed AI threats and more stringent regulations are shifting the French cybersecurity landscape, resulting in businesses reassessing their security strategies.

Increasing security budgets mean many French enterprises require fresh guidance and expertise to establish effective priorities and combat their security challenges.

According to the 2025 ISG Provider Lens Cybersecurity – Services and Solutions report, businesses in France are adapting to a more complex, layered security landscape, with many adopting AI-powered defence. This is to meet the demands introduced by new regulations, cloud adoption, financial constraints, and a shortage of skilled workers, the AI security specialist claims.

“The way companies in France choose security services is changing,” said Julien Escribe, partner and managing director, ISG. “With increasing security budgets, enterprises need guidance and insight to set the right priorities and tackle security problems.”

Companies are starting to rely on all-in-one solutions for security, rather than depending on a range of separate tools, according to the report. Organisations seek service providers that can supplement their security teams. Companies migrating to multicloud and cloud topologies face integration, visibility, and management challenges, the company says. Businesses find it hard to maintain oversight of applications, so use solutions like secure access service edge (SASE), which merges network security and connectivity in a unified service.

The report found that French organisations are seeking integrated security platforms for a single view of possible threats and central oversight of their defences. Because of financial stress and a continuing deficit of cybersecurity talent, many businesses still use technical security service (TSS) providers for support, centralised platforms, and automation.

ISG says that French enterprises are have to integrate governance, risk and commence (GRC) policies into their security strategies, because EU regulations like the NIS2 directive and AI Act are becoming French law. More than 15,000 French businesses are now subject to extra compliance, the report states.

The paper shows malicious actors use AI in cyberattacks, which is posing fresh challenges for detection and response. In reaction, companies turn to security service providers that themselves use genAI and ML. Their clients are investing more in AI-driven detection, employee training, and automated responses.

Benoît Scheuber, a principal consultant and security analyst at ISG, said that AI is transforming the cybersecurity landscape, prompting companies to seek providers that can integrate tooling that increases efficiency, saying, “[Clients] seek providers that can integrate the best products into a unified platform for operational efficiency.”

(Image source: “French Army Soldier at the Eiffel Tower” by derekskey is licensed under CC BY 2.0.)

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and co-located with other leading technology events. Click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post AI-enabled threats and stricter regulation in France appeared first on AI News.

]]>
Thinking Machines becomes OpenAI’s first services partner in APAC https://www.artificialintelligence-news.com/news/thinking-machines-becomes-openai-first-services-partner-in-apac/ Tue, 09 Sep 2025 10:16:22 +0000 https://www.artificialintelligence-news.com/?p=109276 Thinking Machines Data Science is joining forces with OpenAI to help more businesses across Asia Pacific turn artificial intelligence into measurable results. The collaboration makes Thinking Machines the first official Services Partner for OpenAI in the region. The partnership comes as AI adoption in APAC continues to rise. An IBM study found that 61% of […]

The post Thinking Machines becomes OpenAI’s first services partner in APAC appeared first on AI News.

]]>
Thinking Machines Data Science is joining forces with OpenAI to help more businesses across Asia Pacific turn artificial intelligence into measurable results. The collaboration makes Thinking Machines the first official Services Partner for OpenAI in the region.

The partnership comes as AI adoption in APAC continues to rise. An IBM study found that 61% of enterprises already use AI, yet many struggle to move beyond pilot projects and deliver real business impact. Thinking Machines and OpenAI aim to change that by offering executive training on ChatGPT Enterprise, support for building custom AI applications, and guidance on embedding AI into everyday operations.

Stephanie Sy, Founder and CEO of Thinking Machines, framed the partnership around capability building: “We’re not just bringing in new technology but we’re helping organisations build the skills, strategies, and support systems they need to take advantage of AI. For us, it’s about reinventing the future of work through human-AI collaboration and making AI truly work for people across the Asia Pacific region.”

Turning AI pilots into results with Thinking Machines

In an interview with AI News, Sy explained that one of the biggest hurdles for enterprises is how they frame AI adoption. Too often, organisations see it as a technology acquisition rather than a business transformation. That approach leads to pilots that stall or fail to scale.

Stephanie Sy, Founder and CEO of Thinking Machines.
Stephanie Sy, Founder and CEO of Thinking Machines.

“The main challenge is that many organisations approach AI as a technology acquisition rather than a business transformation,” she said. “This leads to pilots that never scale because three fundamentals are missing: clear leadership alignment on the value to create, redesign of workflows to embed AI into how work gets done, and investment in workforce skills to ensure adoption. Get those three right—vision, process, people—and pilots scale into impact.”

Leadership at the centre

Many executives still treat AI as a technical project rather than a strategic priority. Sy believes that boards and C-suites need to set the tone. Their role is to decide whether AI is a growth driver or just a managed risk.

“Boards and C-suites set the tone: Is AI a strategic growth driver or a managed risk? Their role is to name a few priority outcomes, define risk appetite, and assign clear ownership,” she said. Thinking Machines often begins with executive sessions where leaders can explore where tools like ChatGPT add value, how to govern them, and when to scale. “That top-down clarity is what turns AI from an experiment into an enterprise capability.”

Human-AI collaboration in practice

Sy often talks about “reinventing the future of work through human-AI collaboration.” She explained what this looks like in practice: a “human-in-command” approach where people focus on judgment, decision-making, and exceptions, while AI handles routine steps like retrieval, drafting, or summarising.

“Human-in-command means redesigning work so people focus on judgment and exceptions, while AI takes on retrieval, drafting, and routine steps, with transparency through audit trails and source links,” she said. The results are measured in time saved and quality improvements.

In workshops run by Thinking Machines, professionals using ChatGPT often free up one to two hours per day. Research supports these outcomes—Sy pointed to an MIT study showing a 14% productivity boost for contact centre agents, with the biggest gains seen among less-experienced staff. “That’s clear evidence AI can elevate human talent rather than displace it,” she added.

Agentic AI with Thinking Machines’ guardrails

Another area of focus for Thinking Machines is agentic AI, which goes beyond single queries to handle multi-step processes. Instead of just answering a question, agentic systems can manage research, fill forms, and make API calls, coordinating entire workflows with a human still in charge.

“Agentic systems can take work from ‘ask-and-answer’ to multi-step execution: coordinating research, browsing, form-filling, and API calls so teams ship faster with a human in command,” Sy said. The promise is faster execution and productivity, but the risks are real. “The principles of human-in-command and auditability remain critical; to avoid the lack of proper guardrails. Our approach is to pair enterprise controls and auditability with agent capabilities to ensure actions are traceable, reversible, and policy-aligned before we scale.”

Governance that builds trust

While adoption is accelerating, governance often lags behind. Sy cautioned that governance fails when it’s treated as paperwork instead of part of daily work.

“We keep humans in command and make governance visible in daily work: use approved data sources, enforce role-based access, maintain audit trails, and require human decision points for sensitive actions,” she explained. Thinking Machines also applies what it calls “control + reliability”: restricting retrieval to trusted content and returning answers with citations. Workflows are then adapted to local rules in sectors such as finance, government, and healthcare.

For Sy, success isn’t measured in the volume of policies but in auditability and exception rates. “Good governance accelerates adoption because teams trust what they ship,” she said.

Local context, regional scale

Asia Pacific’s cultural and linguistic diversity poses unique challenges for scaling AI. A one-size-fits-all model doesn’t work. Sy emphasised that the right playbook is to build locally first and then scale deliberately.

“Global templates fail when they ignore how local teams work. The playbook is build locally, scale deliberately: fit the AI to local language, forms, policies, and escalation paths; then standardise the parts that travel such as your governance pattern, data connectors, and impact metrics,” she said.

That’s the approach Thinking Machines has taken in Singapore, the Philippines, and Thailand—prove value with local teams first, then roll out region by region. The aim is not a uniform chatbot but a reliable pattern that respects local context while maintaining scalability.

Skills over tools

When asked what skills will matter most in an AI-enabled workplace, Sy pointed out that scale comes from skills, not just tools. She broke this down into three categories:

  • Executive literacy: the ability for leaders to set outcomes and guardrails, and know when and where to scale AI.
  • Workflow design: the redesign of human-AI handoffs, clarifying who drafts, who approves, and how exceptions escalate.
  • Hands-on skills: prompting, evaluation, and retrieval from trusted sources so answers are verifiable, not just plausible.

“When leaders and teams share that foundation, adoption moves from experimenting to repeatable, production-level results,” she said. In Thinking Machines’ programs, many professionals report saving one to two hours per day after just a one-day workshop. To date, more than 10,000 people across roles have been trained, and Sy noted the pattern is consistent: “skills + governance unlock scale.”

Industry transformation ahead

Looking to the next five years, Sy sees AI shifting from drafting to full execution in critical business functions. She expects major gains in software development, marketing, service operations, and supply chain management.

“For the next wave, we see three concrete patterns: policy-aware assistants in finance, supply chain copilots in manufacturing, and personalised yet compliant CX in retail—each built with human checkpoints and verifiable sources so leaders can scale with confidence,” she said.

A practical example is a system Thinking Machines built with the Bank of the Philippine Islands. Called BEAi, it’s a retrieval-augmented generation (RAG) system that supports English, Filipino, and Taglish. It returns answers linked to sources with page numbers and understands policy supersession, turning complex policy documents into everyday guidance for staff. “That’s what ‘AI-native’ looks like in practice,” Sy said.

Thinking Machines expands AI across APAC

The partnership with OpenAI will start with programs in Singapore, the Philippines, and Thailand through Thinking Machines’ regional offices before expanding further across APAC. Future plans include tailoring services to sectors such as finance, retail, and manufacturing, where AI can address specific challenges and open new opportunities.

For Sy, the goal is clear: “AI adoption isn’t just about experimenting with new tools. It’s about building the vision, processes, and skills that let organisations move from pilots to impact. When leaders, teams, and technology come together, that’s when AI delivers lasting value.”

See also: X and xAI sue Apple and OpenAI over AI monopoly claims

Want to learn more about AI and big data from industry leaders? Check out AI & Big Data Expo taking place in Amsterdam, California, and London. The comprehensive event is part of TechEx and is co-located with other leading technology events, click here for more information.

AI News is powered by TechForge Media. Explore other upcoming enterprise technology events and webinars here.

The post Thinking Machines becomes OpenAI’s first services partner in APAC appeared first on AI News.

]]>