Artificial intelligence is moving fast in healthcare, laboratories, and regulated service environments. Most conversations focus on model quality and features. Far fewer focus on compliance reality.
HIPAA does not prohibit AI. It prohibits careless handling of protected health information. The difference matters.
To help cut through vendor marketing and vague claims, we published a provider neutral reference titled HIPAA-Eligible AI Capabilities by Major Cloud Provider. This document maps what is actually possible today across Microsoft Azure, Amazon Web Services, and Google Cloud when operating under a Business Associate Agreement. See whitepaper.
This post summarizes the key takeaways and explains how to use the document to make defensible architecture decisions.
The Core Question Is Not “Which Model”
The real question is where inference happens and who controls the data.
HIPAA compliance hinges on a few non negotiables.
- A signed BAA with the cloud provider
- No training on your prompts or outputs
- Network isolation through private endpoints or VPCs
- Encryption at rest and in transit
- Clear logging and retention policies
All three major providers can meet these requirements. They do so in different ways.
Managed AI vs Private Inference
The document breaks AI inference into two categories.
Managed Inference
Examples include Azure OpenAI, AWS Bedrock, and Google Vertex AI.
Benefits:
- Lower operational burden
- Faster time to value
- No GPU management
Tradeoffs:
- Token based pricing can grow quickly
- Long prompts and verbose outputs drive cost
- You inherit vendor limits and quotas
Managed inference works well for steady workloads, controlled prompts, and teams without GPU operations experience.
Private Inference on GPU VMs
This means running open models yourself on cloud GPUs using a runtime like vLLM.
Benefits:
- No third party model access
- Full control over logging and retention
- Predictable cost at scale
- OpenAI compatible API surface
Tradeoffs:
- You own patching, scaling, and uptime
- Requires ML and infrastructure discipline
For high volume tenants, strict data locality requirements, or sensitive PHI workflows, private inference is often the cleanest compliance story.
The document outlines exactly what “self hosting” actually means in operational terms, including model pinning, networking boundaries, and incident response expectations. hipaa_ai_capabilities_by_cloud_…
Voice to Text Is Often the Hidden Risk
Speech transcription is frequently overlooked and is a common PHI exposure vector.
All three providers offer HIPAA eligible speech to text services, but features vary.
- Azure AI Speech supports real time and batch transcription under BAA
- AWS Transcribe offers inline PII redaction during streaming
- Google Cloud Speech to Text supports medical vocabularies under BAA
Inline redaction can materially reduce downstream risk by preventing PHI from ever reaching storage or analytics systems.
The PDF includes a side by side comparison and pricing context for speech services, which is critical for call heavy or intake driven workflows. hipaa_ai_capabilities_by_cloud_…
Cost Is a Function of Architecture, Not Just Pricing Tables
Token pricing alone is misleading.
The document highlights practical cost drivers:
- Prompt length
- Context window size
- Output verbosity
- Caching strategy
- GPU utilization vs idle time
In many cases, a mid tier GPU running private inference becomes cheaper than managed APIs once usage stabilizes.
Conversely, bursty or experimental workloads are usually better served by managed services.
The right answer depends on workload shape, not brand preference.
How This Document Is Intended to Be Used
This is not a recommendation list. It is an architectural reference.
Use it to:
- Frame conversations with compliance and legal teams
- Explain tradeoffs to clients or auditors
- Decide when managed AI is sufficient
- Know when private inference is justified
- Avoid accidental PHI leakage through misconfigured services
Every service, model, and region still requires validation. HIPAA eligibility is configuration dependent. The document explicitly calls that out.
Final Thought
HIPAA compliant AI is no longer theoretical. It is operational today.
But compliance is an architectural property, not a checkbox.
If your AI strategy does not clearly answer who can see the data, where inference runs, and how PHI is contained, you do not yet have a compliant system.
The full reference document is available here and is updated as provider capabilities evolve.
