Healthcare AI Agent Guardrails
Runtime authorization for healthcare AI agents. Protect PHI, enforce HIPAA compliance, and maintain control over clinical decision support without modifying your agent's code.
What are healthcare AI agent guardrails?
Healthcare AI agent guardrails are runtime controls that intercept, evaluate, and enforce authorization policies on actions taken by AI agents operating in healthcare environments. They protect Protected Health Information (PHI), enforce HIPAA compliance requirements, and ensure clinical decision support operates within authorized boundaries.
Healthcare AI faces unique compliance challenges
Healthcare organizations deploying AI agents must navigate HIPAA regulations, state privacy laws, and institutional policies. In 2024, OCR enforcement saw 725 breaches affecting 275 million records, with civil penalties reaching $2.19 million per violation category annually. AI systems processing PHI become Business Associates under HIPAA, triggering mandatory BAA requirements and direct liability for compliance violations.
AI agents with unfettered database access can expose thousands of patient records in seconds. The minimum necessary standard applies acutely to AI: systems must access only PHI strictly required.
Unauthorized disclosures can trigger fines up to $1.5 million per violation category per year. The 2024 Security Rule NPRM proposes mandatory cybersecurity controls including MFA.
Agents making unauthorized clinical recommendations could impact patient care. Guardrails against automation bias and human oversight mechanisms are essential.
HIPAA Security Rule technical safeguards for AI
HIPAA compliance for AI agents centers on the Security Rule's five technical safeguards: Access Control, Audit Controls, Integrity, Person or Entity Authentication, and Transmission Security. The 2024 proposed Security Rule update mandates multi-factor authentication for all ePHI access.
Access Control (45 CFR 164.312(a)(1))
Implement Role-Based Access Control (RBAC) or Attribute-Based Access Control (ABAC) for AI agent permissions. AI agents must have unique identifiers and access rights scoped to minimum necessary data. The 2024 proposed rule makes MFA mandatory.
Audit Controls (45 CFR 164.312(b))
All AI interactions with PHI must be logged immutably. Required elements: user/agent identity, timestamp, action type, data accessed, IP address, session ID, and outcome. Logs retained 6 years minimum, protected against tampering.
Integrity Controls (45 CFR 164.312(c)(1))
Prevent unauthorized PHI modification. For AI: input validation, output verification, and safeguards against model manipulation or data poisoning. Cryptographic hashes verify data integrity.
Encryption Requirements
AES-256 encryption for PHI at rest; TLS 1.2+ for PHI in transit. The proposed 2025 rule makes encryption mandatory (currently addressable). Hardware Security Modules (HSM) recommended for key management.
Attribute-Based Access Control for PHI
ABAC provides fine-grained, context-aware access control ideal for healthcare AI agents. Unlike static RBAC, ABAC evaluates attributes like agent role, purpose, department, and data sensitivity to enforce the minimum necessary standard in real-time.
import { AuditLogger } from './audit-logger';
import { PHIService } from './phi-service';
interface AgentContext {
agentId: string;
role: 'clinical_assistant' | 'admin_bot' | 'scheduling_agent';
department: string;
purpose: 'treatment' | 'payment' | 'operations' | 'research';
sessionId: string;
}
interface AccessDecision {
allowed: boolean;
scope: string[];
reason: string;
conditions?: string[];
}
class HIPAAAccessController {
private auditLogger: AuditLogger;
private phiService: PHIService;
// Minimum necessary scope definitions by role and purpose
private readonly minimumNecessaryScopes: Record<string, string[]> = {
'clinical_assistant:treatment': ['Patient.read', 'Observation.read', 'Condition.read'],
'scheduling_agent:operations': ['Patient.read.name', 'Patient.read.contact', 'Appointment.*'],
'admin_bot:payment': ['Patient.read.demographics', 'Claim.*', 'Coverage.read'],
};
// Sensitive data categories requiring additional consent
private readonly sensitiveCategories = new Set([
'psychotherapy_notes',
'hiv_status',
'substance_abuse',
'reproductive_health',
'genetic_information'
]);
async evaluateAccess(
context: AgentContext,
requestedResource: string,
requestedAction: 'read' | 'write' | 'delete'
): Promise<AccessDecision> {
const decision: AccessDecision = {
allowed: false,
scope: [],
reason: '',
conditions: []
};
// Build access key and retrieve minimum necessary scope
const accessKey = `${context.role}:${context.purpose}`;
const permittedScopes = this.minimumNecessaryScopes[accessKey] || [];
// Check if requested resource falls within minimum necessary
const isWithinScope = permittedScopes.some(scope =>
this.matchesScope(requestedResource, requestedAction, scope)
);
if (!isWithinScope) {
decision.reason = `Resource ${requestedResource} exceeds minimum necessary`;
await this.auditLogger.logDeniedAccess({
agentId: context.agentId,
resource: requestedResource,
action: requestedAction,
reason: decision.reason,
sessionId: context.sessionId,
timestamp: new Date().toISOString()
});
return decision;
}
// Check for sensitive data segmentation
if (this.isSensitiveCategory(requestedResource)) {
const hasConsent = await this.checkSegmentationConsent(requestedResource, context);
if (!hasConsent) {
decision.reason = 'Sensitive data category requires additional patient consent';
decision.conditions = ['patient_consent_required'];
await this.auditLogger.logDeniedAccess({
agentId: context.agentId,
resource: requestedResource,
action: requestedAction,
reason: decision.reason,
sessionId: context.sessionId,
timestamp: new Date().toISOString()
});
return decision;
}
}
decision.allowed = true;
decision.scope = permittedScopes;
decision.reason = 'Access granted within minimum necessary standard';
// Log approved access
await this.auditLogger.logAccess({
agentId: context.agentId,
resource: requestedResource,
action: requestedAction,
scope: permittedScopes,
sessionId: context.sessionId,
timestamp: new Date().toISOString(),
purpose: context.purpose
});
return decision;
}
private matchesScope(resource: string, action: string, scope: string): boolean {
const [scopeResource, scopeAction] = scope.split('.');
const actionMatch = scopeAction === '*' || scopeAction === action;
const resourceMatch = resource.startsWith(scopeResource) || scopeResource === '*';
return actionMatch && resourceMatch;
}
private isSensitiveCategory(resource: string): boolean {
return Array.from(this.sensitiveCategories).some(cat =>
resource.toLowerCase().includes(cat)
);
}
private async checkSegmentationConsent(resource: string, context: AgentContext): Promise<boolean> {
return this.phiService.verifyConsent(resource, context.purpose);
}
}This implementation demonstrates ABAC with enforcement of the HIPAA minimum necessary standard. Key features: role and purpose-based scope definitions that restrict AI agents to only PHI required for their specific function, sensitive data category detection triggering additional consent verification, and comprehensive audit logging.
HIPAA-compliant audit logging for AI agents
HIPAA Security Rule requires immutable audit trails capturing all PHI interactions. For AI systems, this includes prompts, responses, file uploads, model inference decisions, and data transformations with timestamped records.
import crypto from 'crypto';
interface AuditEntry {
id: string;
timestamp: string;
agentId: string;
sessionId: string;
action: 'PHI_ACCESS' | 'PHI_READ' | 'PHI_WRITE' | 'AI_INFERENCE' | 'DENIED_ACCESS';
resource: string;
dataType: string;
patientId?: string;
outcome: 'SUCCESS' | 'FAILURE' | 'ERROR';
details: Record<string, unknown>;
ipAddress: string;
userAgent: string;
dataHash?: string;
previousHash: string; // Blockchain-style chaining for tamper detection
}
class HIPAAAuditLogger {
private readonly logStore: AuditLogStore;
private lastHash: string = '0'.repeat(64); // Genesis hash
async logAccess(entry: Omit<AuditEntry, 'id' | 'previousHash'>): Promise<void> {
const auditEntry: AuditEntry = {
...entry,
id: crypto.randomUUID(),
previousHash: this.lastHash
};
// Create hash chain for integrity
const entryHash = this.createEntryHash(auditEntry);
this.lastHash = entryHash;
// Sign entry for authenticity
const signature = await this.signEntry(auditEntry);
await this.logStore.append({
...auditEntry,
signature
});
}
async logAIInference(params: {
agentId: string;
sessionId: string;
modelVersion: string;
inputTokens: number;
outputTokens: number;
promptHash: string;
responseHash: string;
phiAccessed: string[];
decision: string;
confidence?: number;
humanReviewRequired: boolean;
}): Promise<void> {
await this.logAccess({
timestamp: new Date().toISOString(),
agentId: params.agentId,
sessionId: params.sessionId,
action: 'AI_INFERENCE',
resource: `model:${params.modelVersion}`,
dataType: 'AI_MODEL_OUTPUT',
outcome: 'SUCCESS',
details: {
inputTokens: params.inputTokens,
outputTokens: params.outputTokens,
promptHash: params.promptHash,
responseHash: params.responseHash,
phiAccessed: params.phiAccessed,
decision: params.decision,
confidence: params.confidence,
humanReviewRequired: params.humanReviewRequired
},
ipAddress: 'system',
userAgent: `AI-Agent/${params.agentId}`
});
}
private createEntryHash(entry: AuditEntry): string {
const entryString = JSON.stringify({
timestamp: entry.timestamp,
agentId: entry.agentId,
action: entry.action,
resource: entry.resource,
outcome: entry.outcome,
previousHash: entry.previousHash
});
return crypto.createHash('sha256').update(entryString).digest('hex');
}
// Verify log integrity - detect tampering
async verifyIntegrity(): Promise<{ valid: boolean; errors: string[] }> {
const entries = await this.logStore.getAll();
const errors: string[] = [];
for (let i = 0; i < entries.length; i++) {
const expectedPrevHash = i === 0 ? '0'.repeat(64) : entries[i - 1].hash;
if (entries[i].previousHash !== expectedPrevHash) {
errors.push(`Chain break at entry ${entries[i].id}`);
}
const isValid = await this.verifySignature(entries[i]);
if (!isValid) {
errors.push(`Invalid signature at entry ${entries[i].id}`);
}
}
return { valid: errors.length === 0, errors };
}
}PHI sanitization for AI prompts and responses
AI agents must sanitize PHI before sending data to models or external services. The minimum necessary standard requires filtering data fields and reducing date precision to the minimum required for each task.
import re
from typing import List, Dict, Any
from dataclasses import dataclass
from enum import Enum
class PHIType(Enum):
NAME = "name"
SSN = "ssn"
MRN = "mrn"
DATE = "date"
PHONE = "phone"
EMAIL = "email"
ADDRESS = "address"
@dataclass
class PHIDetectionResult:
contains_phi: bool
phi_types: List[PHIType]
redacted_text: str
phi_count: int
class PHISanitizer:
"""HIPAA-compliant PHI sanitization for AI agent inputs/outputs."""
# HIPAA Safe Harbor identifiers regex patterns
PHI_PATTERNS = {
PHIType.SSN: re.compile(r'\b\d{3}-\d{2}-\d{4}\b|\b\d{9}\b'),
PHIType.MRN: re.compile(r'\bMRN[:\s]*(\d{6,12})\b', re.IGNORECASE),
PHIType.PHONE: re.compile(
r'\b(?:\+?1[-.]?)?\(?\d{3}\)?[-.]?\d{3}[-.]?\d{4}\b'
),
PHIType.EMAIL: re.compile(
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b'
),
PHIType.DATE: re.compile(
r'\b\d{1,2}[/-]\d{1,2}[/-]\d{2,4}\b|'
r'\b(?:Jan|Feb|Mar|Apr|May|Jun|Jul|Aug|Sep|Oct|Nov|Dec)[a-z]* '
r'\d{1,2},? \d{4}\b',
re.IGNORECASE
),
}
def __init__(self, minimum_necessary_fields: Dict[str, List[str]]):
self.minimum_necessary_fields = minimum_necessary_fields
def detect_and_redact(
self,
text: str,
allowed_phi_types: List[PHIType]
) -> PHIDetectionResult:
"""Detect PHI in text and redact types not in allowed list."""
phi_types_found = []
redacted_text = text
phi_count = 0
for phi_type, pattern in self.PHI_PATTERNS.items():
matches = pattern.findall(text)
if matches:
phi_types_found.append(phi_type)
phi_count += len(matches)
if phi_type not in allowed_phi_types:
# Redact PHI not allowed by minimum necessary
redacted_text = pattern.sub(
f'[{phi_type.value.upper()}_REDACTED]',
redacted_text
)
return PHIDetectionResult(
contains_phi=len(phi_types_found) > 0,
phi_types=phi_types_found,
redacted_text=redacted_text,
phi_count=phi_count
)
class MinimumNecessaryEnforcer:
"""Enforces HIPAA minimum necessary standard for AI data access."""
def __init__(self):
self.scope_definitions = {
'clinical_assistant:treatment': {
'patient_fields': ['name', 'dob', 'diagnosis', 'medications', 'allergies'],
'exclude_fields': ['ssn', 'financial_info', 'insurance_account'],
'date_precision': 'month_only' # Reduce date specificity
},
'scheduling_agent:operations': {
'patient_fields': ['name', 'phone', 'preferred_times'],
'exclude_fields': ['diagnosis', 'medications', 'ssn', 'mrn'],
'date_precision': 'full'
},
'billing_agent:payment': {
'patient_fields': ['name', 'dob', 'insurance_info', 'account_number'],
'exclude_fields': ['diagnosis', 'medications', 'clinical_notes'],
'date_precision': 'full'
}
}
def filter_data(
self,
data: Dict[str, Any],
role: str,
purpose: str
) -> Dict[str, Any]:
"""Filter data to minimum necessary for role/purpose."""
scope_key = f"{role}:{purpose}"
scope = self.scope_definitions.get(scope_key)
if not scope:
# Deny all if scope not defined - fail secure
return {'error': 'No access scope defined', 'data': None}
filtered = {}
allowed_fields = scope['patient_fields']
excluded = scope['exclude_fields']
date_precision = scope['date_precision']
for key, value in data.items():
if key in excluded:
continue
if key in allowed_fields:
# Apply date precision if applicable
if 'date' in key.lower() or 'dob' in key.lower():
value = self._apply_date_precision(value, date_precision)
filtered[key] = value
return filtered
def _apply_date_precision(self, date_value: str, precision: str) -> str:
"""Reduce date precision to minimum necessary."""
if precision == 'month_only':
return re.sub(r'-(\d{2})$', '-XX', date_value)
elif precision == 'year_only':
return date_value[:4] + '-XX-XX'
return date_value
# Usage example
sanitizer = PHISanitizer(
minimum_necessary_fields={
'scheduling_agent:operations': ['name', 'phone'],
'clinical_assistant:treatment': ['name', 'mrn', 'date']
}
)
# Sanitize a scheduling prompt
prompt = """Schedule appointment for John Doe, SSN 123-45-6789,
phone 555-123-4567, DOB 01/15/1980"""
allowed = sanitizer.get_allowed_phi_types('scheduling_agent', 'operations')
result = sanitizer.detect_and_redact(prompt, allowed)
# Output: Schedule appointment for John Doe, [SSN_REDACTED],
# phone 555-123-4567, [DATE_REDACTED]Veto policy configuration for healthcare
Define declarative policies that enforce HIPAA-compliant data access patterns. Policies are version-controlled, auditable, and enforce minimum necessary access.
name: healthcare-phi-protection
description: PHI access control for clinical AI agents
rules:
# EHR read access - minimum necessary
- name: ehr-read-minimum-necessary
tools: ["ehr_read", "patient_lookup"]
action: allow
constraints:
fields: ["demographics", "allergies", "current_medications"]
requires_patient_id: true
audit:
log_arguments: true
retention_days: 2555 # 7 years for HIPAA
# Block clinical notes access for non-clinical agents
- name: block-clinical-notes
tools: ["ehr_read"]
arguments:
fields: ["clinical_notes", "psychiatric_notes", "hiv_status"]
action: deny
response:
error: "Access to clinical notes requires clinical staff authorization"
# Sensitive data segmentation
- name: sensitive-category-consent
tools: ["ehr_read", "patient_lookup"]
arguments:
fields: ["psychotherapy_notes", "substance_abuse", "genetic_information"]
action: require_approval
constraints:
patient_consent_verified: true
response:
message: "Sensitive data category requires verified patient consent"
# Prescription limits for controlled substances
- name: controlled-substance-check
tools: ["prescribe_medication"]
action: require_approval
constraints:
schedule: ["II", "III", "IV"]
audit:
log_arguments: true
alert_on_deny: true
response:
approver_role: "pharmacist"
message: "Controlled substance prescription requires pharmacist review"
# Patient communication guardrails
- name: patient-communication-phi-check
tools: ["send_patient_message"]
action: allow
constraints:
no_phi_in_message: true
verified_patient_identity: true
response:
require_confirmation: "Confirm message contains no PHI and patient identity verified"
# Research data - de-identification enforcement
- name: research-data-deidentified
tools: ["query_research_dataset"]
action: allow
constraints:
dataset_type: "de_identified"
approved_irb_protocol: true
audit:
log_query: true
researcher_id: requiredHIPAA compliance mapping for AI agents
Veto guardrails provide technical controls necessary to meet HIPAA Security Rule requirements.
| HIPAA Requirement | Implementation |
|---|---|
| Access Control (164.312(a)(1)) | RBAC/ABAC for agent authorization, unique agent identifiers, automatic logoff, MFA for admin access |
| Audit Controls (164.312(b)) | Immutable audit trails with hash chaining, 6+ year retention, tamper detection |
| Integrity (164.312(c)(1)) | Input validation, prompt injection guardrails, cryptographic integrity verification |
| Transmission Security (164.312(e)(1)) | TLS 1.2+ for all communications, certificate pinning, in-process evaluation |
| Minimum Necessary (164.502(b)) | Purpose-specific data access scopes, field-level restrictions, date precision controls |
| Accounting of Disclosures (164.528) | Exportable audit logs, patient disclosure request support, structured compliance reports |
Sensitive data segmentation controls
HIPAA requires specialized access controls for sensitive PHI categories. AI agents must check for segmentation flags and verify consent before accessing these categories.
Psychotherapy Notes
Requires explicit patient consent. Enhanced access logging. Separate authorization from general mental health records under 42 U.S.C. 290dd-2.
Substance Abuse Records
42 CFR Part 2 compliance required. Specific consent form requirements. Stricter restrictions than standard HIPAA.
HIV/AIDS Information
State-specific consent requirements. Many states require separate written authorization. Restrictions on disclosure without explicit consent.
Reproductive Health
Post-Dobbs state-specific restrictions. Enhanced privacy protections in some jurisdictions. Geographic context-aware policy enforcement.
Build vs buy for healthcare AI guardrails
| Capability | DIY | Veto |
|---|---|---|
| HIPAA-compliant logging with hash chaining | ||
| ABAC with minimum necessary enforcement | ||
| Sensitive data segmentation controls | ||
| PHI sanitization for prompts/responses | ||
| 7-year audit retention | ||
| BAA-ready documentation | Create yourself | Included |
| Time to HIPAA compliance | Months | Hours |
Related use cases
Document access control, client confidentiality, and ethical compliance
Insurance AI AgentsClaims processing guardrails, fraud detection limits, and regulatory compliance
Enterprise AI AgentsSSO, audit trails, role-based access, and multi-tenant isolation
Claude IntegrationAnthropic SDK security patterns and tool authorization
Frequently asked questions
How do healthcare AI guardrails help with HIPAA compliance?
What is the minimum necessary standard for AI agents?
How do audit logs prevent tampering?
What retention periods are supported for healthcare audit logs?
How does sensitive data segmentation work?
HIPAA-compliant guardrails for healthcare AI agents.