🎉 We’re excited to share that our recent session at AIA Meetup 2025 was a tremendous success! HydroX AI COO, Victor Bian, delivered a compelling and insightful talk on AI Red-Teaming — Stress Testing AI Systems for Safety & Reliability.

Ensuring LLMs generate safe, helpful, and value-aligned responses is challenging. RLHF improves alignment but is costly and unstable, while DPO simplifies training but struggles with conflicting objectives like safety and helpfulness. At HydroX AI, we introduce Group Relative Policy Optimization (GRPO) with Multi-Label Reward Regression — a more efficient, scalable, and interpretable approach to AI alignment.

🎉 We’re excited to announce that HydroX AI has been selected as one of 15 companies for the 2025 Google for Startups Cloud AI Accelerator — a prestigious program supporting the top AI-first startups across the U.S. and Canada.

In the 4th Episode of Season 1 of the Attention Needed podcast, we explore the future of AI innovation with Dean Wampler, IBM’s Chief Technical Representative to the AI Alliance — a collaboration led by IBM, Meta, and other key organizations to promote open, safe, and responsible AI development. This conversation dives into the opportunities and challenges of building AI in the open.

We’re beyond excited to announce that HydroX AI is proudly sponsoring and collaborating with ROOST (Robust Open Online Safety Tools) – a groundbreaking initiative incubated with the support of major funders like Discord, OpenAI, Google, and more. ROOST recently completed its spinout, and it’s on a mission to revolutionize online safety in the digital age.

At HydroX AI, we are constantly striving to advance the field of AI safety. Today, we’re excited to share the release of our latest paper, which explores an important, yet often overlooked aspect of large language models (LLMs): the impact of output length on model safety and reasoning capabilities.

In the 3rd Episode of Season 1 of the Attention Needed podcast, we sit down with Roland Cloutier, a trailblazer in cybersecurity and digital risk management. As the former Global Chief Security Officer for TikTok and a leader at major organizations like ADP and Dell/EMC, Roland shares his unparalleled insights on securing AI-driven platforms in an ever-evolving digital world.

We’re excited to present the 2nd Episode of Season 1 of the Attention Needed podcast, where we explore the intersection of responsible AI and governance with Dr. Rumman Chowdhury, CEO of Humane Intelligence and former U.S. Science Envoy for AI.

We’re excited to introduce The AI Alliance – What it is, who it’s for, and why it was created., a new podcast that dives deep into the world of artificial intelligence, exploring its advancements, challenges, and the critical role of safety and security in its evolution. Hosted by Victor Bian, our Chief Operating Officer, this podcast will feature conversations with leading business and technical experts who are shaping the future of AI.

DeepSeek, a Chinese AI company, has recently been turning heads in the artificial intelligence community. Known for its innovative approaches, the company has developed models that challenge leading AI systems like OpenAI’s offerings—delivering competitive performance at a fraction of the cost and computational resources.

With a group of major announcements made from close-source frontier models at the end of 2024, there were also notable releases in the open-source community. In this short blog post, we delve into two of the latest open-source models, Llama-3.3 and Tulu-3, to evaluate their performance from the perspectives of AI Safety and Security.

Artificial Intelligence (AI) agents, powered by cutting-edge large language models (LLMs), are transforming the way we interact with machines. From answering complex queries to assisting in critical decision-making processes, LLMs like GPT-4, Llama, and others have enabled seamless, natural, and context-aware communication. However, with great power comes great responsibility and risk. In this blog, we explore how a deceptively simple adversarial strategy can expose critical vulnerabilities in these advanced systems, potentially leading to unintended and even dangerous consequences.

We’re thrilled to introduce the Attack Prompt Tool, a new Google Chrome Extension designed to support AI safety research by making adversarial prompt testing easier. This tool is built for AI researchers, security professionals, and anyone interested in understanding the resilience of large language models (LLMs) against adversarial techniques, particularly jailbreak prompts.

In the fast development of artificial intelligence, an enormous amount of unstructured data like online crawled information or private information is used to fuel the model advancement and AI applications. Protecting Personally Identifiable Information (PII) is a paramount requirement of responsible AI. As organizations increasingly leverage unstructured data for AI applications such as Retrieval-Augmented Generation, ensuring the safety of PII during model training and information retrieval is essential. To address this critical need, Zilliz, the creator of the world’s most popular open-source vector database Milvus—has partnered with HydroX AI to introduce PII Masker, an advanced tool designed to enhance data privacy in AI applications.

Anthropic’s release of Claude 3.5 marks a significant leap forward in the evolution of large language models (LLMs) and their ability to interact with computers. At HydroX AI, we’re excited by the potential this unlocks for AI-powered computer operations, but we also recognize the importance of keeping safety front and center as AI takes on more sophisticated roles in digital environments.

Today we are thrilled to announce our partnership with Anthropic, a leader in AI research and development, to further enhance the safety and security of large language models (LLMs). Anthropic has been at the forefront of creating advanced AI systems with a strong focus on safety, and we couldn’t be more excited to collaborate with one of the best teams in the world!

In our previous Llama-generation report, we analyzed the safety and advancements of Llama-2, Llama-3, and Llama-3.1. One key finding that emerged was the surprising result that the larger Llama-3.1-70B model exhibited lower safety compared to its smaller counterpart, Llama-3.1-8B. In this article, we will take a deep dive into the safety of Llama-3.1, exploring the relationship between model size and safety. Our goal is to shed light on why larger, smarter models are not always the safest, and what this means for the future of AI safety.

On September 12, 2024, OpenAI unveiled its latest model, OpenAI o1, which boasts powerful reasoning capabilities along with enhanced safety measures against jailbreak attempts. This release has sparked significant interest in the AI community, as it sets a new benchmark for secure AI interactions. At the same time, the GPT-4o mini model, which utilizes instruction hierarchy, has been recognized for its robust safety features.

The Llama series, an open-source large language model (LLM) developed by Meta, has gained recognition for its high performance and the emphasis placed on safety and security during its development. Over the years, several significant versions of the Llama series have been released, each aimed at improving upon its predecessor.

In today's rapidly evolving digital landscape, ensuring the safety and security of AI systems within the video business sector is paramount. Our comprehensive training course, "AI Safety & Security for Video Business Compliance," is designed to equip industry professionals with the knowledge and skills necessary to navigate the complex regulatory environment and implement robust security measures.

In this article, we will explore a novel type of attack: code injection via images on the Gemini Advanced platform. We will provide a detailed explanation of the attack's principles, implementation process, and how to defend against such attacks.

Today we are announcing some major developments for HydroX AI that mark an exciting time ahead as we continue expanding our work in enabling AI safety and building safe AI. It's hard to believe that as a company, we have yet to celebrate our 1-year anniversary since establishment. We are pleased to receive many positive endorsements on our work from across the industry and we remain grateful (and awed) at the opportunity of collaborating with some of the world's most innovative and prestigious partners.

HydroX AI, the AI security company enabling safe and responsible use of Artificial Intelligence (AI), today announced that David Danks, PhD, of University of California, San Diego (UCSD), has joined its advisory board.

In an era dominated by the rapid evolution of AI technologies, ensuring their safety and security has become paramount. The potential benefits of AI are vast, ranging from revolutionizing healthcare and transportation to optimizing resource allocation and enhancing productivity.However, with these advancements come significant ethical, social, and existential concerns.