On this page
- Quick Reference (60 Seconds)
- What the Control Asks For
- Why Environment Separation Matters
- Scope and Applicability
- Key Definitions and Terminology
- Relationship to Other Controls
- Implementation Roadmap (Week-by-Week)
- Detailed Implementation Guidance
- Tools, Technologies, and Solutions
- Policy and Procedure Templates
- Risk Assessment and Treatment
- Audit and Compliance Checklist
- Metrics and KPIs
- Common Pitfalls and How to Avoid Them
- Illustrative Scenarios
- Multi-Framework Mapping
- Regulatory and Industry Context
- Roles and Responsibilities (RACI)
- Documentation and Evidence Requirements
- Continuous Improvement
- FAQ
- References and Further Reading
Quick Reference (60 Seconds)
Control: A.8.31, Separation of Development, Test and Production Environments
Purpose: Prevent unauthorized changes, data leakage, and security incidents by maintaining physical or logical separation between development, testing, and production environments.
ISO 27002 attributes: Control type: Preventive · Properties: Confidentiality, Integrity, Availability · Concepts: Protect · Capabilities: Application security, System and network security · Domains: Protection
Who it applies to: All organizations that develop, test, or operate software applications or information systems.
Minimum viable actions:
- Maintain separate networks/VPCs for dev, test, and production
- Implement different access controls for each environment (least privilege)
- Never use production data in development or test environments
- Use separate credentials and authentication for each environment
- Implement change control that prevents direct development-to-production deployments
Key deliverables: Environment Separation Policy, Network Architecture Diagram, Access Control Matrix, Environment Configuration Standards, Change Management Procedure.
Audit questions you should be able to answer:
- How are your development, test, and production environments separated?
- Who has access to production? How is that access different from development access?
- How do you prevent production data from being used in development or testing?
- Can developers deploy code directly to production?
What the Control Asks For
Do you need this control?
A.8.31 is not mandatory in itself: under clause 6.1.3 you include it if your risk assessment calls for it, and record the decision in your Statement of Applicability. Include it if you develop or test changes to systems. A small team may use separate cloud accounts or projects rather than physically separate environments.
Annex A 8.31 asks organizations to separate and secure development, test, and production environments.
What ISO 27002:2022 Adds
ISO/IEC 27002:2022 (8.31), paraphrased: decide how much separation you need to prevent production problems, and implement it. Consider:
- (a) separating development and production and running them in different domains (separate virtual or physical environments);
- (b) rules and authorisation for deploying software from development to production;
- (c) testing changes in a test or staging environment before production (8.29);
- (d) no testing in production except in defined, approved circumstances;
- (e) compilers, editors and other development tools not accessible from production when not needed;
- (f) environment labels in menus (and screens) to reduce mistakes;
- (g) no sensitive information copied into development and test unless equivalent controls apply there.
In all cases, protect development and test environments too: patch and update the toolchain (builders, compilers, configuration systems, libraries); configure securely; control access; monitor changes to the environment and the code in it; monitor securely; and back them up.
No single person should be able to change both development and production without prior review and approval, through segregated access rights or monitored rules. Where an exception is unavoidable, add detailed logging and real-time monitoring.
27002 also notes that developers and testers with production access can cause unwanted changes, failures, untested code in production, data disclosure and integrity problems, and that a known, stable environment is needed for meaningful testing.
What the Standard Does NOT Require
- The standard does not mandate physical separation (logical separation is acceptable)
- It does not require that no one can access both dev and production (separation of duties is the goal, not absolute isolation)
- It does not prevent developers from accessing production for troubleshooting (but such access must be controlled, monitored, and time-limited)
- It does not require separate hardware for each environment (virtualization and cloud isolation are acceptable)
Why Environment Separation Matters
Real-World Incidents from Environment Mixing
Capital One (2019): An attacker exploited a misconfigured web application firewall through server-side request forgery, obtained cloud credentials for a role with far broader storage access than it needed, and copied data on about 106 million people in the US and Canada. The A.8.31 lesson: a component should reach only the data its own environment and function need.
GitLab (2017): While troubleshooting database replication late at night, an engineer ran a delete command on the primary production database server, believing it was the secondary. About six hours of data were lost and the service took around 18 hours to restore, partly because several backup methods had not been working. The lessons: clear environment and host labels (27002 8.31(f)), restricted destructive commands in production, and tested backups.
A common pattern (hypothetical): a developer keeps production credentials in a test configuration file; a test script runs against production and moves real money or changes real records.
The Risks of Poor Environment Separation
| Risk | Impact | Example |
|---|---|---|
| Unauthorized production changes | Service disruption, data corruption | Developer pushes untested code to production |
| Production data exposure | Data breach, regulatory penalties | Production database copied to dev with no access controls |
| Credential leakage | Account takeover, lateral movement | Production credentials stored in dev environment code |
| Malware introduction | Production compromise | Test environment malware spreads to production via shared network |
| Debugging information exposure | Information disclosure | Stack traces, debug mode enabled in production |
| Unauthorized data extraction | Data exfiltration | Developer extracts production data for "testing" |
| Regulatory non-compliance | Fines, license revocation | Using production PHI in test environments violates HIPAA |
| Repudiation issues | Accountability gaps | Changes made in production cannot be traced to individual |
The Indian Context
Indian organizations face unique environment separation challenges:
- Cost constraints: Small organizations may not afford separate infrastructure for each environment
- Cloud adoption: Rapid cloud adoption creates environment sprawl without proper governance
- Shared hosting: Many Indian SMEs use shared hosting where separation is limited
- Regulatory expectations: RBI (Master Direction on IT Governance, Risk, Controls and Assurance Practices, 2023), SEBI (CSCRF, 2024) and IRDAI (2023 guidelines) expect controlled change and separated environments for regulated systems
- Outsourced development: Offshore vendors may not maintain proper separation between client environments
- DPDP Act 2023: copying production personal data into test environments without equivalent safeguards is hard to justify under the consent and purpose limits (s.6(1)) and the safeguards duty (s.8(5)); use masked or synthetic data (A.8.33)
- Startup culture: "Move fast and break things" culture often skips environment separation
Scope and Applicability
In Scope
This control applies to:
- Software development environments: Where code is written, compiled, and unit tested
- Testing environments: Where integration, system, UAT, and security testing occur
- Staging/Pre-production environments: Production-like environments for final validation
- Production environments: Live systems serving real users and processing real data
- Infrastructure environments: Where infrastructure code (Terraform, CloudFormation) is developed and tested
- Database environments: Separate database instances for dev, test, and production
- Network environments: Separate network segments for each environment
- Cloud environments: Separate cloud accounts, VPCs, or subscriptions for each environment
- Container environments: Separate Kubernetes clusters or namespaces for each environment
- AI/ML environments: Separate training, validation, and inference environments
Out of Scope (with caveats)
- Local development machines: Individual developer laptops (though they should not have production access)
- Documentation environments: Wikis, Confluence, SharePoint (unless they contain production credentials)
- Personal devices: Mobile devices, tablets (covered by A.8.1)
- Sandbox environments: Temporary, isolated playgrounds (should still be governed)
Caveat: Developer laptops with production VPN access or production credentials stored locally ARE in scope and represent a significant risk.
Applicability by Organization Type
| Organization Type | Applicability | Typical Environment Separation Focus |
|---|---|---|
| Software product companies | Critical | Dev → Test → Staging → Prod for SaaS platforms |
| Financial services | Critical | Strict separation for core banking, payment systems |
| Healthcare | Critical | PHI separation, HIPAA compliance |
| E-commerce | Critical | Payment processing, customer data separation |
| Government | Critical | Citizen data, classified system separation |
| Manufacturing | High | OT/IT separation, production system isolation |
| Startups | High | Cloud-native separation, rapid deployment |
| NGOs | Moderate | Donor data, grant management system separation |
Key Definitions and Terminology
| Term | Definition |
|---|---|
| Development Environment | The environment where software is written, compiled, and unit tested by developers |
| Testing Environment | The environment where software is tested for functionality, integration, performance, and security |
| Staging Environment | A production-like environment where final validation occurs before deployment to production |
| Production Environment | The live environment where software serves real users and processes real data |
| Environment Separation | The practice of isolating environments to prevent unauthorized access, data leakage, or changes between them |
| Physical Separation | Environments running on completely separate hardware with no shared infrastructure |
| Logical Separation | Environments running on shared hardware but isolated through virtualization, networking, or access controls |
| Network Segmentation | Dividing a network into subnetworks (segments) to isolate environments |
| VPC (Virtual Private Cloud) | A logically isolated section of a cloud provider's network |
| Jump Server / Bastion Host | A hardened server that provides access to internal resources from external networks |
| Just-in-Time (JIT) Access | Temporary, time-limited access to production granted on-demand with approval |
| Break-Glass Access | Emergency access to production for critical incidents with full audit logging |
| Data Masking / Anonymization | Techniques to protect sensitive data by transforming it into non-sensitive equivalents |
| Environment Drift | Differences between environments that cause inconsistencies in application behavior |
| Infrastructure as Code (IaC) | Managing infrastructure through code files rather than manual configuration |
| Configuration Management | Process of maintaining consistency and traceability of system configurations |
| Environment Promotion | The process of moving code, configuration, or data through environments (dev → test → staging → prod) |
| Rollback | The process of reverting to a previous version after a failed deployment |
Relationship to Other Controls
Directly Related Controls
| Control | Relationship |
|---|---|
| A.5.3, Segregation of duties | No single person changes both development and production without review |
| A.8.2, Privileged access rights | Restricted, just-in-time production access |
| A.8.4, Access to source code | Protect code in development environments |
| A.8.9, Configuration management | Environment-specific secure configuration |
| A.8.11, Data masking | Masked data for lower environments |
| A.8.13, Information backup | Back up development and test environments too |
| A.8.15 / A.8.16, Logging; monitoring | Monitor changes and access in each environment |
| A.8.19, Installation of software on operational systems | No development tools in production |
| A.8.22, Segregation of networks | Network separation between environments |
| A.8.25, Secure development life cycle | Environments are part of the SDLC |
| A.8.29, Security testing | Test in a production-like environment |
| A.8.30, Outsourced development | Vendor environment requirements |
| A.8.32, Change management | Approval to promote to production |
| A.8.33, Test information | Protect test data |
Framework Mapping
| Framework | Relevant Control / Reference |
|---|---|
| NIST CSF 2.0 | PR.IR-01 (networks and environments protected), PR.PS-01 (configuration management practices), PR.AA-05 (access permissions and least privilege) |
| NIST SP 800-53 Rev 5 | CM-4(1) (separate test environments), CM-5 (access restrictions for change), CM-7 (least functionality), SC-32 (system partitioning), AC-5 (separation of duties) |
| PCI DSS v4.0.1 | 6.5.3 (pre-production separated from production), 6.5.4 (roles separated), 6.5.5 (no live PANs in pre-production), 6.5.6 (test data and accounts removed before go-live) |
| COBIT 2019 | BAI06.01 (evaluate, prioritise and authorise changes), BAI07.04 (establish a test environment), BAI10.02 (configuration baseline) |
| CIS Controls v8 | 16.8 (separate production and non-production systems), 4.x (secure configuration), 6.x (access control) |
| GDPR | Art 32 (security of processing) |
| DPDP Act 2023 | Section 6(1), Section 8(5) |
Implementation Roadmap (Week-by-Week)
Figure · Tiers
Maturity levels for separation of development, test and production environments

Phase 1: Foundation (Weeks 1–3)
Week 1: Environment Inventory and Assessment
- Inventory all existing environments (dev, test, staging, prod, and any others)
- Map network architecture and identify environment boundaries
- Identify who has access to each environment
- Identify data flows between environments
- Document current environment separation gaps
Week 2: Policy and Standard Definition
- Draft the Environment Separation Policy
- Define environment standards (network, access, data, configuration)
- Define environment promotion process (dev → test → staging → prod)
- Define emergency access (break-glass) procedures
Week 3: Architecture Design
- Design network architecture for environment separation
- Design access control model (different roles per environment)
- Design data handling model (no production data in dev/test without masking)
- Design monitoring and alerting for cross-environment activity
Deliverables: Environment Inventory, Policy Draft, Architecture Design, Gap Analysis
Phase 2: Pilot (Weeks 4–6)
Week 4-5: Pilot Implementation
- Select one critical application for pilot separation
- Implement network separation (VPCs, subnets, firewalls)
- Implement access controls (different roles, MFA, least privilege)
- Implement data handling (synthetic data for dev/test)
- Implement monitoring (alerts for cross-environment access)
Week 6: Validation and Refinement
- Validate that developers cannot access production without approval
- Validate that production data cannot flow to dev/test without masking
- Validate that deployment pipeline enforces environment promotion
- Refine policy and procedures based on pilot feedback
Deliverables: Pilot Environment, Validation Report, Updated Policy, Refined Procedures
Phase 3: Rollout (Weeks 7–12)
Week 7-9: Organization-Wide Implementation
- Apply environment separation to all applications and systems
- Implement network segmentation for all environments
- Implement access controls for all environments
- Implement data handling standards for all environments
- Deploy monitoring and alerting across all environments
Week 10-12: Process Integration
- Integrate environment separation into CI/CD pipeline
- Integrate environment separation into change management
- Train all developers, testers, and operations staff
- Establish environment governance committee
Deliverables: All environments separated, CI/CD integrated, Staff trained, Governance established
Phase 4: Optimization (Weeks 13–16)
Week 13-14: Metrics and Monitoring
- Define and collect KPIs (see Section 13)
- Conduct first internal audit of environment separation
- Identify gaps and improvement opportunities
Week 15-16: Continuous Improvement
- Implement automated environment compliance checks
- Implement Infrastructure as Code for consistent environment provisioning
- Implement automated drift detection
- Update training and documentation
Deliverables: KPI dashboard, Internal audit report, Automated compliance, IaC deployment
Maturity Model
| Level | Description | Typical Timeline |
|---|---|---|
| 1, Ad-hoc | No environment separation, dev/test/prod on same systems, shared credentials | Pre-implementation |
| 2, Managed | Basic separation for some systems, manual controls, informal process | Weeks 1–3 |
| 3, Defined | Standardized separation for all systems, automated access controls, formal promotion process, monitoring | Weeks 4–8 |
| 4, Quantitatively Managed | Metrics tracked, automated compliance checks, IaC, drift detection, just-in-time access | Weeks 9–12 |
| 5, Optimizing | Continuous improvement, dynamic environment provisioning, automated remediation, zero-trust environment access | Ongoing |
Detailed Implementation Guidance
Figure · Matrix
Comparison: Development to Production
Environment Separation Architecture
Network-Level Separation
Traditional Data Center:
- Separate physical networks or VLANs for each environment
- Firewall rules prevent traffic between environments
- Dedicated network hardware for production where possible
- DMZ for public-facing production systems
Cloud (AWS Example):
Production Account/VPC
├── Private subnets (application servers, databases)
├── Public subnets (load balancers, CDN)
├── Transit Gateway (controlled connectivity)
└── Dedicated IAM roles and policies
Staging Account/VPC
├── Mirrors production architecture
├── No production data (masked/synthetic)
├── Separate IAM roles
└── Network peering with production (if needed) with strict controls
Development Account/VPC
├── Lightweight architecture
├── No production data
├── Broader developer access
└── Separate IAM roles
Testing Account/VPC
├── Test-specific configurations
├── Synthetic data
├── Automated testing tools
└── Separate IAM roles
Cloud (Azure Example):
- Separate subscriptions or resource groups per environment
- Virtual network peering with NSG rules restricting traffic
- Azure Policy enforcing environment-specific configurations
- Dedicated service principals per environment
Cloud (GCP Example):
- Separate projects per environment
- VPC Service Controls for data exfiltration prevention
- Dedicated IAM policies per project
- Shared VPC with host project controlling connectivity
Container/Kubernetes:
- Separate clusters for production and non-production
- Or separate namespaces within a cluster with network policies
- Resource quotas preventing dev/test from consuming production resources
- Pod Security Standards (admission control) enforcing different security levels per environment
Environment Labels, Tooling and the Two-Person Rule
- Labels (27002 8.31(f)): show a red "PRODUCTION" banner in admin consoles and internal tools, use coloured shell prompts (for example red for prod), name hosts and accounts with the environment, and set different colours for cloud console roles.
- No build tools in production (8.31(e)): production images and servers contain no compilers, package managers beyond what runtime needs, debuggers or editors; use minimal or distroless images.
- Two-person rule for small teams: where one engineer does everything, require pull-request approval by a second person and give deploy rights only to the CI pipeline. Direct production changes use break-glass access with session logging and an after-the-fact review.
Development and Test Environment Baseline
27002 says development and test environments must be protected in all cases:
| Item | Minimum |
|---|---|
| Toolchain | Patched compilers, build tools, libraries and CI runners |
| Configuration | Hardened images; no default credentials |
| Access | SSO and MFA; least privilege; leavers removed |
| Exposure | No public exposure of staging or dev without authentication |
| Secrets | Separate from production; stored in a secrets manager |
| Monitoring | Log changes to environments and code; alert on unusual access |
| Backups | Repositories, pipelines and environment definitions backed up |
Access Control Separation
Environment-Specific Roles:
| Role | Development | Testing | Staging | Production |
|---|---|---|---|---|
| Developer | Full access | Read-only | Read-only | No access (except JIT) |
| Tester | No access | Full access | Read-only | No access |
| DevOps Engineer | Full access | Full access | Full access | Limited (deploy only) |
| Security Engineer | Read-only | Read-only | Full access | Read-only (audit) |
| Database Administrator | Full access | Full access | Full access | JIT only (with approval) |
| System Administrator | Full access | Full access | Full access | JIT only (with approval) |
| Production Support | No access | No access | Read-only | JIT only (with approval) |
| Executive | No access | No access | No access | Dashboard only (read-only) |
Access Control Mechanisms:
- Different credentials: Separate accounts per environment (not shared passwords)
- MFA: Required for all environment access, especially production
- RBAC: Role-based access control with environment-specific roles
- PAM: Privileged Access Management for production access (CyberArk, BeyondTrust, Delinea)
- JIT Access: Just-in-time access for production (Azure PIM, AWS IAM Identity Center, Google Cloud IAM)
- Break-Glass: Emergency access with full audit logging and immediate notification
Example: Just-in-Time Production Access
Developer needs production access for incident response:
1. Developer requests access via PAM system (reason: incident #1234)
2. Manager approves request (time-limited: 2 hours)
3. PAM system grants temporary production access
4. All activity is logged and recorded (session recording)
5. Access automatically revoked after 2 hours
6. Session recording reviewed by security team within 24 hours
Data Separation
The Golden Rule: No Production Data in Dev/Test Without Masking
Data Handling by Environment:
| Environment | Data Source | Handling |
|---|---|---|
| Development | Synthetic data | Completely synthetic, no real customer data |
| Testing | Masked production data | Production data with PII/PHI masked, subset |
| Staging | Masked production data | Production data with PII/PHI masked, full volume |
| Production | Real data | Production data with full protection |
Data Masking Techniques:
- Substitution: Replace real names with fake names from a dictionary
- Shuffling: Shuffle values within a column (maintains distribution)
- Encryption: Encrypt sensitive fields with keys that are not available in dev/test
- Nulling: Replace sensitive fields with NULL or default values
- Tokenization: Replace sensitive data with non-sensitive tokens
- Redaction: Remove sensitive data entirely
- Date shifting: Shift dates by random amounts (preserves intervals)
- Number variance: Modify numeric values by a random percentage
Data Masking Example:
Original Production Record:
Name: Rajesh Kumar
Email: rajesh.kumar@example.com
Phone: +91-98765-43210
PAN: ABCDE1234F
Account: 123456789012
Balance: Masked for Test Environment:
Name: Anil Sharma
Email: anil.sharma@example.com
Phone: +91-98765-00000
PAN: XXXXX1234X
Account: 987654321098
Balance: (varied by 5%)
Data Subsetting:
- Use a representative subset of production data (e.g., 10% of records)
- Include edge cases and boundary conditions
- Maintain referential integrity across tables
- Use synthetic data for new features not yet in production
Data Refresh Process:
- Extract production data to secure staging area
- Apply masking/subsetting/anonymization
- Verify that no real PII/PHI remains
- Transfer to test environment
- Log the transfer for audit
- Delete staging data after verification
Configuration Separation
Environment-Specific Configuration:
| Configuration Item | Development | Testing | Staging | Production |
|---|---|---|---|---|
| Debug mode | Enabled | Enabled | Disabled | Disabled |
| Error messages | Verbose | Verbose | Generic | Generic |
| Logging level | Debug | Debug | Info | Warning |
| Database connection | Dev DB | Test DB | Staging DB | Prod DB |
| API endpoints | Mock/Local | Test APIs | Staging APIs | Production APIs |
| External services | Sandbox | Sandbox | Staging | Production |
| Encryption keys | Dev keys | Test keys | Staging keys | Production keys (HSM) |
| Feature flags | All features | All features | Pre-release | Production features |
| Rate limits | Disabled | Disabled | Production-like | Production |
| WAF rules | Permissive | Permissive | Production-like | Strict |
| CSP headers | Report-only | Report-only | Enforced | Enforced |
| Session timeout | 8 hours | 8 hours | 1 hour | 30 minutes |
| Password policy | Weak (for testing) | Weak (for testing) | Production | Production |
Configuration Management:
- Use environment variables for environment-specific settings (never hardcode)
- Use configuration management tools (Ansible, Chef, Puppet, SaltStack)
- Use Infrastructure as Code (Terraform, CloudFormation, Pulumi) for consistent environment provisioning
- Use secrets management (HashiCorp Vault, AWS Secrets Manager, Azure Key Vault) for credentials
- Implement configuration drift detection
Deployment Pipeline Separation
Environment Promotion Pipeline:
Developer Commit → CI Build (Dev) → Unit Tests → Deploy to Dev →
Integration Tests → Deploy to Test → System Tests → Security Tests →
Deploy to Staging → UAT → Performance Tests → Security Sign-off →
Deploy to Production → Smoke Tests → Monitoring
Deployment Controls:
- No direct deployment from dev to prod: Must flow through all intermediate environments
- Automated deployment to dev/test: Developers can deploy to dev/test via CI/CD
- Manual approval for staging: Staging deployment requires QA/security approval
- Manual approval for production: Production deployment requires change advisory board (CAB) or security sign-off
- Blue/green deployment: Production deployment uses blue/green or canary deployment to minimize risk
- Rollback capability: Every production deployment must be rollback-capable within minutes
- Deployment freeze windows: No production deployments during critical periods (financial quarter-end, Black Friday, etc.)
Monitoring and Alerting for Environment Separation
What to Monitor:
- Cross-environment network traffic (dev → prod, test → prod)
- Data transfers between environments (production data to dev/test)
- Access requests to production from non-production accounts
- Credential usage across environments (production credentials used in dev)
- Configuration changes that affect environment boundaries
- Unauthorized environment creation (shadow IT environments)
Alerting Rules:
- Alert when production data is detected in dev/test (DLP rules)
- Alert when production credentials are used in non-production
- Alert when network traffic flows from dev/test to production
- Alert when new production accounts are created without approval
- Alert when production access is granted without JIT approval
- Alert when environment configuration drift is detected
Example: Detecting Production Data in Dev
## DLP rule to detect PAN numbers in dev/test
if environment in ['dev', 'test'] and data matches PAN_PATTERN:
alert("PRODUCTION_DATA_IN_DEV",
f"PAN-like data detected in {environment} environment",
severity="CRITICAL")
block_transfer()
Special Scenarios
Multi-Tenant SaaS Environments
Challenge: A single SaaS application serves multiple tenants (customers). Each tenant's data must be isolated.
Separation Strategies:
- Database-level isolation: Separate databases per tenant (most secure, most expensive)
- Schema-level isolation: Separate schemas within a shared database
- Row-level isolation: Shared database with tenant_id column filtering (least secure, most efficient)
- Hybrid: Critical tenants get dedicated infrastructure; standard tenants share
Implementation:
- Enforce tenant isolation at application layer (all queries filtered by tenant_id)
- Enforce tenant isolation at database layer (RLS, Row Level Security in PostgreSQL)
- Prevent cross-tenant data access through automated testing
- Monitor for cross-tenant data leakage
Microservices Environment Separation
Challenge: Microservices architecture has many services, each with dev/test/staging/prod instances.
Separation Strategies:
- Separate clusters: Kubernetes clusters per environment (most secure)
- Separate namespaces: Namespaces within a cluster with network policies (efficient)
- Service mesh isolation: Istio/Linkerd mTLS and authorization policies between environments
Implementation:
- Network policies prevent cross-environment service communication
- Service mesh policies enforce environment-specific access
- Resource quotas prevent dev/test from consuming production resources
- Pod security policies enforce different security levels per environment
AI/ML Environment Separation
Challenge: AI/ML pipelines have training, validation, inference, and monitoring environments.
Separation Strategies:
- Training environment: Separate from inference, broad access for data scientists
- Validation environment: Production-like for model validation, no production data
- Inference environment: Production, strict access, monitoring for model drift
- Monitoring environment: Separate from inference, observability data only
Implementation:
- Training data must not include production inference data (feedback loops)
- Model artifacts must be signed and verified before inference deployment
- Inference environment must have strict access (no data scientists without approval)
- Model versioning must separate training artifacts from inference artifacts
Legacy System Environment Separation
Challenge: Legacy systems may not support modern environment separation.
Separation Strategies:
- Database-level: Separate database instances even if application runs on same server
- Network-level: VLAN separation for legacy systems
- Access-level: Different OS-level accounts for dev/test/prod access
- Time-based: Maintenance windows for production changes, no daytime access
- Proxy-level: API gateways or reverse proxies controlling access to legacy systems
Implementation:
- Compensating controls where technical separation is not possible
- Enhanced monitoring and logging for legacy systems
- Restricted access windows for production legacy systems
- Gradual modernization plan with environment separation as a goal
Tools, Technologies, and Solutions
Infrastructure as Code (IaC) Tools
| Tool | Best For | Pricing model |
|---|---|---|
| Terraform | Multi-cloud, modular, state management | Free (open source) / Enterprise |
| AWS CloudFormation | AWS-native, infrastructure as code | Free (AWS service) |
| Azure Resource Manager | Azure-native, template-based | Free (Azure service) |
| Google Cloud Deployment Manager | GCP-native, template-based | Free (GCP service) |
| Pulumi | Multi-language, modern IaC | Free (open source) / Enterprise |
| Ansible | Configuration management, agentless | Free (open source) / Tower |
| Chef | Configuration management, enterprise | Enterprise pricing |
| Puppet | Configuration management, enterprise | Enterprise pricing |
| SaltStack | Configuration management, event-driven | Free (open source) / Enterprise |
| Crossplane | Kubernetes-native, multi-cloud | Free (open source) |
Secrets Management Tools
| Tool | Best For | Pricing model |
|---|---|---|
| HashiCorp Vault | Enterprise secrets management, dynamic credentials | Free (open source) / Enterprise |
| AWS Secrets Manager | AWS-native, integration with AWS services | Pay per secret |
| Azure Key Vault | Azure-native, integration with Azure services | Pay per operation |
| Google Cloud Secret Manager | GCP-native, integration with GCP services | Pay per secret |
| CyberArk Conjur | Enterprise secrets management, DevOps | Enterprise pricing |
PAM (Privileged Access Management) Tools
| Tool | Best For | Pricing model |
|---|---|---|
| CyberArk | Enterprise PAM, complete | Enterprise pricing |
| BeyondTrust | Enterprise PAM, session management | Enterprise pricing |
| Delinea (Thycotic) | Enterprise PAM, secret server | Enterprise pricing |
| Azure AD Privileged Identity Management | Azure-native, JIT access | Included in Azure AD P2 |
| AWS IAM Identity Center | AWS-native, SSO + JIT | Free (AWS service) |
| Google Cloud IAM | GCP-native, conditional access | Free (GCP service) |
| Teleport | Modern PAM, infrastructure access | Free / Enterprise |
| StrongDM | Infrastructure access management | Commercial |
| Akeyless | SaaS PAM, vaultless | Commercial |
| Boundary by HashiCorp | Modern access management, open source | Free (open source) / Enterprise |
Monitoring and DLP Tools
| Tool | Best For | Pricing model |
|---|---|---|
| Splunk | SIEM, log analysis, security monitoring | Enterprise pricing |
| Elastic Security | SIEM, open source, scalable | Free / Enterprise |
| Microsoft Sentinel | Azure-native SIEM, SOAR | Pay per ingestion |
| Google Chronicle | GCP-native SIEM, threat intelligence | Enterprise pricing |
| Symantec DLP | Enterprise DLP, endpoint and network | Enterprise pricing |
| Microsoft Purview | Microsoft 365 DLP, compliance | Included in E5 |
| Digital Guardian | Enterprise DLP, endpoint focus | Enterprise pricing |
| Forcepoint DLP | Enterprise DLP, cloud and endpoint | Enterprise pricing |
| Nightfall AI | Cloud-native DLP, API-based | Commercial |
| Netwrix | Data security, DLP, access management | Commercial |
CI/CD Pipeline Tools
| Tool | Best For | Pricing model |
|---|---|---|
| Jenkins | Open source, highly customizable | Free (open source) |
| AWS CodePipeline | AWS-native, integrated with AWS services | Pay per pipeline |
| Google Cloud Build | GCP-native, container-focused | Pay per build |
| TeamCity | JetBrains, on-premise option | Free / Enterprise |
| ArgoCD | Kubernetes-native, GitOps | Free (open source) |
Policy and Procedure Templates
Environment Separation Policy (Template)
Template
Environment Separation Policy
Document ID: POL-ENV-SEP-001 Version: 1.0 Effective Date: [DATE] Owner: CISO / Infrastructure Security Lead Approved By: [Name], [Title] Review Cycle: Annual
1. Purpose
This policy establishes the requirement for separation of development, testing, staging, and production environments to prevent unauthorized access, data leakage, and production incidents.
2. Scope
This policy applies to:
- All software development environments
- All testing environments (integration, system, UAT, security)
- All staging/pre-production environments
- All production environments
- All database environments
- All network environments
- All cloud environments (AWS, Azure, GCP)
- All container and Kubernetes environments
- All AI/ML training, validation, and inference environments
- All infrastructure as code environments
3. Policy Statements
3.1 Environment Separation
- Development, testing, staging, and production environments must be logically or physically separated.
- Network traffic between environments must be controlled and monitored.
- Cross-environment access must be explicitly authorized and logged.
- Production environments must be the most restricted and monitored.
3.2 Access Control
- Access to production requires stronger authentication than non-production.
- Production access must use multi-factor authentication (MFA).
- Production access must be granted on a least-privilege basis.
- Developers must not have routine access to production.
- Just-in-time (JIT) access may be granted for production with approval and time limits.
- Break-glass access must be available for emergencies with full audit logging.
- Access must be reviewed and revoked when no longer needed.
3.3 Data Handling
- Production data must not be used in development or testing environments without proper masking, anonymization, or synthetic data generation.
- Production credentials (passwords, API keys, tokens) must not be used in non-production environments.
- Data transfer between environments must be logged and approved.
- Test data must be clearly labeled and managed (see A.8.33).
- Data masking must be verified before transfer to non-production environments.
3.4 Configuration Management
- Environment-specific configurations must be managed separately.
- Production configurations must be hardened and not include debug settings.
- Configuration changes must flow through change management.
- Infrastructure as Code (IaC) must be used for consistent environment provisioning.
- Configuration drift must be detected and remediated.
3.5 Deployment Management
- Code must flow through environments: dev → test → staging → production.
- Direct deployment from development to production is prohibited.
- Production deployments require approval from change advisory board or security sign-off.
- Deployment windows must be defined and communicated.
- Rollback capability must be maintained for all production deployments.
- Emergency deployments must be documented and reviewed post-incident.
3.6 Monitoring and Alerting
- Cross-environment access attempts must be logged and alerted.
- Production data detected in non-production must trigger critical alerts.
- Production credentials used in non-production must trigger critical alerts.
- Unauthorized environment creation must be detected and blocked.
- Environment access logs must be retained for the period in the retention schedule (at least 180 days in India for ICT logs covered by the CERT-In Directions).
3.7 Environment Lifecycle
- Environments must be provisioned through approved processes (IaC or service catalog).
- Temporary environments must be deprovisioned after use.
- Shadow environments (unauthorized) must be detected and eliminated.
- Environment naming conventions must clearly identify environment type.
4. Roles and Responsibilities
- CISO: Owns the policy, approves exceptions, reports to board
- Infrastructure Security Lead: Designs environment architecture, implements separation, monitors compliance
- DevOps Lead: Manages CI/CD pipeline, environment provisioning, deployment automation
- Development Lead: Ensures developers follow environment access rules
- Database Administrator: Manages database environments, data masking, refresh process
- Security Operations: Monitors cross-environment activity, investigates alerts
- Compliance Officer: Verifies regulatory compliance for environment separation
5. Exceptions
Exceptions require written CISO approval with documented risk acceptance and compensating controls.
6. Enforcement
Non-compliance may result in revoked access, audit findings, or disciplinary action.
7. Related Documents
- Information Security Policy (POL-INFO-001)
- Secure Development Life Cycle Policy (POL-SDLC-001)
- Change Management Policy (POL-CHANGE-001)
- Access Control Policy (POL-ACCESS-001)
- Data Protection Policy (POL-DATA-001)
- Test Data Management Policy (POL-TEST-DATA-001)
8. Revision History
| Version | Date | Author | Changes |
|---|---|---|---|
| 1.0 | [DATE] | [Name] | Initial version |
Environment Promotion Procedure (Template)
Template
Environment Promotion Procedure
Document ID: PROC-ENV-PROM-001 Version: 1.0 Effective Date: [DATE] Owner: DevOps Lead
1. Purpose
To define the procedure for promoting code, configuration, and data through environments (dev → test → staging → production).
2. Scope
All application deployments across all environments.
3. Promotion Paths
Dev → Test
- Trigger: Developer commits code, CI/CD pipeline runs
- Automated: Build, unit tests, SAST, SCA
- Approval: None (automated for standard changes)
- Deployment: Automated via CI/CD
- Validation: Automated tests run in test environment
Test → Staging
- Trigger: All tests pass in test environment
- Automated: Integration tests, security tests, performance baseline
- Approval: QA Lead approves staging deployment
- Deployment: Automated via CI/CD (after approval)
- Validation: UAT, performance tests, security tests
Staging → Production
- Trigger: UAT passes, security sign-off obtained
- Automated: Final security scan, configuration validation
- Approval: Change Advisory Board (CAB) or designated approver
- Deployment: Blue/green or canary deployment (automated with manual trigger)
- Validation: Smoke tests, monitoring, rollback readiness
4. Emergency Deployment
- Emergency deployments bypass normal approval for critical security patches or incidents.
- Requires: Security Lead + DevOps Lead approval
- Must be documented post-deployment
- Must be reviewed in next CAB meeting
- Full audit trail required
5. Rollback
- All production deployments must be rollback-capable within 15 minutes.
- Rollback requires: DevOps Lead + Security Lead approval
- Rollback must be documented and investigated
- Rollback does not require CAB approval (time-critical)
6. Records
- Deployment records (who, what, when, why)
- Approval records (CAB minutes, sign-off forms)
- Test results (automated and manual)
- Rollback records (if applicable)
- Post-deployment validation results
Risk Assessment and Treatment
Risk Assessment for Environment Separation
| Risk ID | Risk Description | Likelihood | Impact | Risk Level | Mitigation |
|---|---|---|---|---|---|
| R-001 | Developer has direct production access, makes unauthorized changes | Medium | Critical | Critical | Remove dev access, implement JIT, monitoring |
| R-002 | Production data copied to dev/test without masking | Medium | Critical | Critical | DLP, data masking process, monitoring, synthetic data policy |
| R-003 | Production credentials used in dev/test environments | Medium | Critical | Critical | Secrets management, credential scanning, monitoring |
| R-004 | Dev/test malware spreads to production via shared network | Low | High | High | Network segmentation, endpoint protection, monitoring |
| R-005 | Configuration drift causes production to behave like dev | Medium | High | High | IaC, configuration drift detection, automated compliance |
| R-006 | Unauthorized environment created (shadow IT) | Medium | Medium | Medium | Environment governance, automated discovery, policy enforcement |
| R-007 | Vendor has access to both dev and production | Medium | High | High | Vendor access controls, monitoring, time-limited access |
| R-008 | Debug mode enabled in production | Medium | High | High | Configuration validation, automated checks, deployment gates |
| R-009 | Production backup restored to dev/test without sanitization | Medium | High | High | Backup access controls, sanitization process, monitoring |
| R-010 | Cross-environment network traffic not monitored | Medium | Medium | Medium | Network monitoring, IDS/IPS, firewall logging |
| R-011 | Environment separation bypassed for "urgent" fix | High | High | Critical | Emergency process, post-incident review, management commitment |
| R-012 | Database replication from prod to dev/test | Low | Critical | High | Replication controls, masking, access controls |
| R-013 | Developer laptop with production credentials lost/stolen | Low | Critical | High | Endpoint encryption, credential rotation, MFA, remote wipe |
| R-014 | AI/ML training data includes production inference data | Medium | High | High | Data pipeline separation, training data validation |
| R-015 | Multi-tenant cross-tenant data leakage | Low | Critical | High | Tenant isolation, RLS, automated testing, monitoring |
Risk Treatment Options
| Risk | Treatment | Residual Risk |
|---|---|---|
| R-001 | JIT access + PAM + monitoring + access review | Low |
| R-002 | DLP + masking process + synthetic data + monitoring | Low |
| R-003 | Secrets management + scanning + monitoring | Low |
| R-004 | Network segmentation + endpoint protection + monitoring | Low |
| R-005 | IaC + drift detection + automated compliance | Low |
| R-006 | Environment governance + discovery + policy | Low |
| R-007 | Vendor access controls + monitoring + time limits | Low |
| R-008 | Config validation + automated checks + gates | Low |
| R-009 | Backup controls + sanitization + monitoring | Low |
| R-010 | Network monitoring + IDS/IPS + logging | Low |
| R-011 | Emergency process + review + management commitment | Low |
| R-012 | Replication controls + masking + access controls | Low |
| R-013 | Endpoint encryption + rotation + MFA + remote wipe | Low |
| R-014 | Pipeline separation + training data validation | Low |
| R-015 | Tenant isolation + RLS + testing + monitoring | Low |
Audit and Compliance Checklist
Pre-Audit Self-Assessment
| # | Question | Evidence | Status |
|---|---|---|---|
| 1 | Is there a documented Environment Separation Policy? | Policy document | ☐ |
| 2 | Are development, test, and production environments separated? | Network diagram, architecture docs | ☐ |
| 3 | Is network traffic between environments controlled? | Firewall rules, network policies | ☐ |
| 4 | Are different credentials required for each environment? | Access control records | ☐ |
| 5 | Is MFA required for production access? | MFA configuration | ☐ |
| 6 | Do developers have routine access to production? | Access review records | ☐ |
| 7 | Is just-in-time (JIT) access used for production? | JIT access records | ☐ |
| 8 | Is production data used in dev/test without masking? | Data handling records, DLP logs | ☐ |
| 9 | Are production credentials used in non-production? | Secrets scanning records | ☐ |
| 10 | Is Infrastructure as Code used for environment provisioning? | IaC repository, provisioning records | ☐ |
| 11 | Is configuration drift detected and remediated? | Drift detection records | ☐ |
| 12 | Is direct deployment from dev to production prohibited? | CI/CD configuration, deployment records | ☐ |
| 13 | Is production deployment approved by CAB or security sign-off? | Approval records, CAB minutes | ☐ |
| 14 | Are deployment logs maintained? | Deployment logs | ☐ |
| 15 | Is rollback capability maintained for production? | Rollback records, CI/CD config | ☐ |
| 16 | Are cross-environment access attempts logged and alerted? | SIEM logs, alerting rules | ☐ |
| 17 | Are unauthorized environments detected? | Discovery records, shadow IT reports | ☐ |
| 18 | Is environment access reviewed quarterly? | Access review records | ☐ |
| 19 | Are break-glass access events logged and reviewed? | Break-glass records | ☐ |
| 20 | Is data masking verified before transfer to non-production? | Masking verification records | ☐ |
| 21 | Are test environments clearly labeled and managed? | Environment inventory, naming records | ☐ |
| 22 | Is environment-specific configuration managed separately? | Configuration records, IaC | ☐ |
| 23 | Are debug settings disabled in production? | Configuration validation records | ☐ |
| 24 | Is environment separation monitored for compliance? | Monitoring records, compliance reports | ☐ |
| 25 | Are vendor environments separated from client production? | Vendor access records | ☐ |
| 26 | Are multi-tenant environments properly isolated? | Tenant isolation records | ☐ |
| 27 | Are AI/ML environments separated (training vs. inference)? | AI/ML environment records | ☐ |
| 28 | Is environment separation included in developer training? | Training records | ☐ |
| 29 | Are emergency deployments documented and reviewed? | Emergency deployment records | ☐ |
| 30 | Are environment access logs retained for the scheduled period (at least 180 days for CERT-In-covered logs)? | Log retention records | ☐ |
Auditor Interview Questions
Be prepared to answer:
- "How are your development, test, and production environments separated?"
- "Who has access to production? How is that different from development access?"
- "How do you prevent production data from being used in development or testing?"
- "Can developers deploy code directly to production?"
- "How do you handle emergency access to production?"
- "Are production credentials used in non-production environments?"
- "How do you detect configuration drift between environments?"
- "Are deployment logs maintained and reviewed?"
- "How do you handle vendor access to production?"
- "How do you detect and respond to unauthorized environment creation?"
Common Audit Findings and How to Avoid Them
| Finding | Cause | Prevention |
|---|---|---|
| "No environment separation documented" | Process gap | Network diagram, architecture documentation, policy |
| "Developers have production access" | Access control gap | Remove routine access, implement JIT, PAM |
| "Production data found in test environment" | Data handling gap | DLP, masking, synthetic data, monitoring |
| "Direct deployment from dev to prod" | CI/CD gap | Pipeline controls, deployment gates, approval workflow |
| "No MFA for production access" | Authentication gap | MFA enforcement for all production access |
| "Production credentials in dev environment" | Secrets management gap | Secrets scanning, vault, monitoring |
| "Debug mode enabled in production" | Configuration gap | Automated config validation, deployment checks |
| "No deployment logs" | Logging gap | CI/CD logging, deployment tracking |
| "Unauthorized environments exist" | Governance gap | Environment discovery, policy enforcement, governance |
| "No access review for production" | Review gap | Quarterly access review, automated reporting |
Metrics and KPIs
Figure · Measures
The measures that show A.8.31 is working
- Environment Separation Coverage100%Quarterly
- Production MFA Coverage100%Monthly
- Production Dev Access Rate0%Monthly
- JIT Access Usage> 90%Monthly
- Data Masking Compliance100%Monthly
Process Metrics
| Metric | Formula | Target | Frequency |
|---|---|---|---|
| Environment Separation Coverage | (# of systems with env separation / # of total systems) × 100 | 100% | Quarterly |
| Production MFA Coverage | (# of production accounts with MFA / # of production accounts) × 100 | 100% | Monthly |
| Production Dev Access Rate | (# of developers with routine prod access / # of developers) × 100 | 0% | Monthly |
| JIT Access Usage | (# of JIT access requests / # of production access needs) × 100 | > 90% | Monthly |
| Data Masking Compliance | (# of data transfers with masking / # of data transfers to non-prod) × 100 | 100% | Monthly |
| Secrets Scanning Coverage | (# of repos scanned for secrets / # of repos) × 100 | 100% | Monthly |
| IaC Coverage | (# of environments provisioned via IaC / # of environments) × 100 | > 95% | Quarterly |
| Drift Detection Coverage | (# of environments with drift detection / # of environments) × 100 | > 95% | Quarterly |
| Deployment Approval Rate | (# of deployments with approval / # of production deployments) × 100 | 100% | Monthly |
| Shadow Environment Detection | (# of unauthorized envs detected / # of total environments) × 100 | < 2% | Quarterly |
| Access Review Completion | (# of quarterly access reviews completed / # of required reviews) × 100 | 100% | Quarterly |
| Break-Glass Review Rate | (# of break-glass events reviewed / # of break-glass events) × 100 | 100% | Monthly |
| Rollback Success Rate | (# of successful rollbacks / # of rollback attempts) × 100 | 100% | Monthly |
| Environment Naming Compliance | (# of environments with standard naming / # of environments) × 100 | 100% | Quarterly |
| Cross-Environment Alert Response | Average time to respond to cross-env alerts | < 1 hour | Monthly |
Outcome Metrics
| Metric | Formula | Target | Frequency |
|---|---|---|---|
| Production Incidents from Dev/Test | Incidents caused by cross-environment activity per quarter | 0 | Quarterly |
| Production Data in Non-Production | Instances of production data detected in dev/test per quarter | 0 | Quarterly |
| Production Credentials in Non-Production | Instances of production credentials in dev/test per quarter | 0 | Quarterly |
| Unauthorized Production Changes | Changes made to production without approval per quarter | 0 | Quarterly |
| Environment Drift Instances | Configuration drift instances detected per quarter | < 5 | Quarterly |
| Compliance Audit Findings | Environment separation-related findings per audit | 0 | Per audit |
| Mean Time to Detect Shadow Environment | Average days to detect unauthorized environment | < 7 days | Quarterly |
| Deployment Success Rate | (# of successful deployments / # of deployments) × 100 | > 98% | Monthly |
| Emergency Deployment Rate | (# of emergency deployments / # of total deployments) × 100 | < 5% | Monthly |
| Cost of Environment Incidents | Financial impact of environment separation failures per quarter | Decreasing | Quarterly |
Dashboard Sample
┌─────────────────────────────────────────────────────────────────────┐
│ ENVIRONMENT SEPARATION DASHBOARD │
│ [Organization] — [Month Year] │
├─────────────────────────────────────────────────────────────────────┤
│ ENV SEPARATION: 100% ██████████████████████ Target: 100% │
│ PROD MFA: 100% ██████████████████████ Target: 100% │
│ DEV PROD ACCESS: 0% ░░░░░░░░░░░░░░░░░░░░░░ Target: 0% │
│ JIT USAGE: 95% ████████████████████░░ Target: 90% │
│ DATA MASKING: 100% ██████████████████████ Target: 100% │
│ SECRETS SCAN: 100% ██████████████████████ Target: 100% │
│ IaC COVERAGE: 97% ████████████████████░░ Target: 95% │
│ DRIFT DETECTED: 2/q ██░░░░░░░░░░░░░░░░░░░ Target: <5/q │
│ PROD INCIDENTS: 0 ░░░░░░░░░░░░░░░░░░░░░ Target: 0 │
│ DEPLOY SUCCESS: 99.2% ████████████████████░ Target: 98% │
│ SHADOW ENVS: 0 ░░░░░░░░░░░░░░░░░░░░░ Target: 0 │
│ EMERGENCY DEPLOY: 2% █░░░░░░░░░░░░░░░░░░░░ Target: <5% │
└─────────────────────────────────────────────────────────────────────┘
Common Pitfalls and How to Avoid Them
Pitfall 1: "We Can't Afford Separate Infrastructure"
Symptom: Small organizations use the same server for dev, test, and production.
Reality: Cloud computing makes environment separation affordable. A small dev environment costs a fraction of a production environment. The cost of a production incident from environment mixing far exceeds the cost of separation.
Solution:
- Use cloud-native separation (separate VPCs, resource groups, projects)
- Use serverless/containerization for lightweight dev/test environments
- Start with logical separation (network + access controls) even on shared hardware
- Use IaC to provision environments on-demand and decommission when not needed
Pitfall 2: "Developers Need Production Access for Troubleshooting"
Symptom: Developers have standing production access "just in case."
Reality: Production access should be rare, time-limited, and fully audited. Most troubleshooting can be done with logs, monitoring, and read-only access. If direct access is needed, JIT access is the solution.
Solution:
- Implement complete logging and monitoring so developers can troubleshoot without production access
- Use read-only dashboards and log access for most troubleshooting
- Implement JIT access for rare cases requiring direct access
- Use session recording for all production access
- Review and question every standing production access account
Pitfall 3: "We Use Production Data for Realistic Testing"
Symptom: Production data is copied to test for "realistic testing."
Reality: This is a data breach waiting to happen. Test environments typically have weaker security controls, broader access, and less monitoring. Production data in test environments violates DPDP Act 2023, GDPR, HIPAA, and PCI DSS.
Solution:
- Invest in high-quality synthetic data generation
- Use data masking with irreversible transformations
- Use data subsetting to reduce exposure
- Implement DLP to detect and block production data in test
- Train testers on using synthetic data effectively
Pitfall 4: "We Have the Same Password for All Environments"
Symptom: Shared credentials across environments for convenience.
Reality: If a dev environment is compromised, shared credentials give attackers direct production access. Credential leaks in dev (via code commits, logs, or screenshots) expose production.
Solution:
- Use environment-specific credentials managed by a vault
- Implement secrets rotation per environment
- Scan all code and logs for credential leaks
- Use different authentication mechanisms per environment (e.g., SSO for dev, MFA + PAM for prod)
Pitfall 5: "Environment Separation Is Just Network Separation"
Symptom: Organization separates networks but ignores access controls, data handling, and configuration.
Reality: Network separation is necessary but not sufficient. A developer with VPN access to both environments can bypass network separation. Production data on a laptop bridges the separation.
Solution:
- Implement defense in depth: network + access + data + configuration + monitoring
- Use PAM for production access even if network is separated
- Monitor for data exfiltration (DLP) even across separated networks
- Validate configuration differences between environments
Pitfall 6: "Our CI/CD Pipeline Bypasses Separation"
Symptom: CI/CD pipeline can deploy to any environment from any branch.
Reality: CI/CD pipelines must enforce environment separation. A pipeline that can deploy dev code to production is a single point of failure.
Solution:
- Implement branch-to-environment mapping (main branch → staging → production)
- Require manual approval for production deployment
- Implement deployment gates that validate environment-specific criteria
- Use separate pipeline stages for each environment with different permissions
- Implement blue/green or canary deployment for production
Pitfall 7: "We Don't Monitor for Shadow Environments"
Symptom: Developers create unauthorized cloud environments for testing.
Reality: Shadow environments lack security controls, monitoring, and governance. They may contain production data, credentials, or code.
Solution:
- Implement cloud governance tools (AWS Config, Azure Policy, GCP Organization Policy)
- Use cloud cost monitoring to detect unexpected environments
- Implement environment tagging policies and automated compliance checks
- Create a sanctioned environment provisioning process (service catalog)
- Conduct regular cloud security posture management (CSPM) scans
Pitfall 8: "Emergency Fixes Require Direct Production Access"
Symptom: Emergency process allows developers to bypass all controls for "urgent" fixes.
Reality: Emergency processes are necessary but must be tightly controlled. Otherwise, every fix becomes an "emergency."
Solution:
- Define clear criteria for emergency deployment (security patch, critical outage)
- Require dual approval for emergency access (security + operations)
- Implement time-limited emergency access (auto-expires)
- Full audit logging and session recording for all emergency access
- Mandatory post-incident review within 24 hours
- Track emergency deployment rate as a KPI (should be < 5% of deployments)
Illustrative Scenarios
Hypothetical scenarios: composite examples for guidance, not real organisations or verified outcomes.
Scenario 1: One Cloud Account for Everything
Situation. A UPI fintech runs development, test and production in one cloud account. Every developer can reach production, and production database credentials sit in development configuration. A migration script meant for development runs against production and deletes transaction records.
What applying 8.31 looks like.
- Separate cloud accounts for development, test, staging and production, with no standing cross-account access.
- Environment-specific secrets in a secrets manager; secret scanning in repositories.
- Synthetic data for development and test; masked data for staging (A.8.33); DLP checks for production data in lower environments.
- Deployments only through the pipeline, with approval for production (A.8.32); just-in-time, recorded production access.
- Red production banners and prompts so engineers always know where they are.
Lesson. Separation is cheapest at the start. Once one account holds everything, every mistake can reach production.
Scenario 2: Shared Database Servers in a Hospital
Situation. A hospital's electronic health record runs development, test and production databases on the same servers, and real patient records are used in user acceptance testing.
What applying 8.31 looks like.
- Production on its own network segment and database servers; lower environments with no route to production data.
- A masking pipeline for any production-derived test data; synthetic data elsewhere.
- Vendor access limited to development and test; production through just-in-time approval only.
- Development and test environments patched, monitored and backed up like any other system.
Lesson. Sensitive data in a test environment needs the same protection as production, or it should not be there.
Multi-Framework Mapping
NIST SP 800-53 Rev 5 Mapping
| NIST Control | Description | A.8.31 Mapping |
|---|---|---|
| SC-2 | Application partition | Environment separation for applications |
| SC-3 | Security function isolation | Isolation of security functions per environment |
| CM-2 | Baseline configuration | Environment-specific baselines |
| CM-4 | Security impact analysis | Impact analysis for cross-environment changes |
| CM-5 | Access restrictions for change | Restricting changes to production |
| CM-7 | Least functionality | Environment-specific functionality |
| AC-6 | Least privilege | Environment-specific least privilege |
| AU-6 | Audit review | Environment access audit |
| SC-7 | Boundary protection | Environment boundary protection |
| SC-32 | System partition | Partitioning systems by environment |
COBIT 2019 Mapping
| COBIT Practice | Description | A.8.31 Mapping |
|---|---|---|
| BAI06.01 | Evaluate, prioritise and authorise change requests | Authorised promotion to production |
| BAI07.04 | Establish a test environment | Separate, production-like test environments |
| BAI10.02 | Establish and maintain a configuration repository and baseline | Environment baselines |
| APO13.01 | Establish and maintain an ISMS | Environment separation within the ISMS |
PCI DSS v4.0.1 Mapping
| PCI DSS Requirement | Environment Separation Focus |
|---|---|
| 6.5.3 | Pre-production environments separated from production, with access controls |
| 6.5.4 | Roles and functions separated between production and pre-production |
| 6.5.5 | Live PANs not used in pre-production unless the environment is in CDE scope |
| 6.5.6 | Test data and test accounts removed before the system goes live |
DPDP Act 2023: Environment Implications
| DPDP Act Section | Environment Separation Implication |
|---|---|
| Section 8(5) | Reasonable security safeguards, environments must have appropriate safeguards |
| Section 8(4) | Appropriate technical and organisational measures, separation is a design principle |
| Section 6(1) | Data minimisation (consent covers only data necessary for the purpose), production data must not be in non-production |
| Sections 4–5 | Purpose limitation (processing only for the lawful purpose in the notice), data must be used only for its intended purpose |
OWASP SAMM Mapping
| SAMM Practice | Maturity Level | A.8.31 Mapping |
|---|---|---|
| Operations - Environment Management | Level 1-3 | Environment separation, configuration, monitoring |
| Implementation - Secure Build | Level 1-3 | Build environment separation |
| Verification - Security Testing | Level 1-3 | Test environment separation |
CIS Controls v8 Mapping
| CIS Safeguard | Implementation Group | A.8.31 Mapping |
|---|---|---|
| 16.8 Separate production and non-production systems | IG2 | Environment separation |
| 4.x Secure configuration | IG1 | Environment baselines |
| 6.x Access control management | IG1 | Environment-specific access |
Regulatory and Industry Context
India
| Regulation | Environment Separation Relevance | Key Mandates |
|---|---|---|
| DPDP Act 2023 | Critical | Section 6(1): data minimisation, no production personal data in non-production; Section 8(5) (Reasonable security safeguards) |
| IT Act 2000 | High | Section 43A: Reasonable security practices include environment separation |
| RBI | Critical for banks | MD IT Governance, Risk, Controls and Assurance Practices (2023): change management and controlled environments |
| SEBI CSCRF (2024) | Critical for markets | Secure development, change and testing for regulated entities |
| IRDAI Guidelines | Critical for insurance | Information security requires environment separation for customer data |
| CERT-In | High | Security best practices include environment separation |
| Digital India | Critical for government | Citizen data must be protected through environment separation |
International
| Regulation | Relevance | Key Mandates |
|---|---|---|
| PCI DSS v4.0.1 | Critical for card data | 6.5.3–6.5.6: pre-production separation, roles, no live PANs, test data removal |
| GDPR | High | Art 32: Security of processing requires environment separation |
| HIPAA | Critical for health | Security Rule: Technical safeguards require environment separation for PHI |
| SOX | High for public companies | IT general controls require environment separation for financial systems |
| NIST CSF 2.0 | High | PR.IR-01, PR.PS-01 |
| CCPA/CPRA | High | Security requirements require environment separation |
| LGPD | High | Security measures require environment separation |
| PDPA | High | Protection obligations require environment separation |
Roles and Responsibilities (RACI)
RACI Matrix for Environment Separation
| Activity | CISO | Infra Security Lead | DevOps Lead | Network Admin | DBA | Dev Lead | Security Ops | Compliance Officer |
|---|---|---|---|---|---|---|---|---|
| Define separation policy | A | R | C | C | I | I | C | C |
| Design network architecture | I | R/A | C | C | I | I | C | I |
| Implement network separation | I | A | C | R | I | I | C | I |
| Manage access controls | I | A | C | C | C | C | R | I |
| Implement data masking | I | C | I | I | R/A | I | C | C |
| Manage secrets | I | A | R | I | I | C | C | I |
| Implement CI/CD gates | I | C | R/A | I | I | C | C | I |
| Provision environments | I | C | R/A | C | I | C | I | I |
| Monitor cross-env activity | I | A | C | C | I | I | R | I |
| Implement PAM/JIT | I | A | C | C | C | I | R | I |
| Conduct access reviews | I | A | C | C | C | C | R | I |
| Audit environment separation | R/A | C | I | I | I | I | C | C |
| Manage IaC | I | C | R/A | C | I | C | I | I |
| Handle incidents | A | C | C | C | C | I | R | I |
| Compliance verification | I | C | I | I | I | I | C | R/A |
| Training | A | R | C | I | I | C | C | I |
Legend: R = Responsible, A = Accountable, C = Consulted, I = Informed
Role Descriptions
| Role | Key Responsibilities | Required Skills |
|---|---|---|
| CISO | Policy ownership, exception approval, incident oversight, board reporting | Security leadership, risk management, governance |
| Infrastructure Security Lead | Architecture design, separation implementation, access control design, monitoring strategy | Network security, cloud security, architecture, PAM |
| DevOps Lead | CI/CD pipeline, environment provisioning, deployment automation, IaC management | DevOps, CI/CD, cloud infrastructure, automation |
| Network Administrator | Network segmentation, firewall rules, VLANs, VPN, connectivity | Networking, firewalls, routing, switching |
| Database Administrator | Database environment separation, data masking, data refresh, backup management | Database administration, data masking, SQL |
| Development Lead | Developer access management, dev environment governance, code deployment | Software development, team management, security awareness |
| Security Operations | Cross-environment monitoring, incident detection, access review, alerting | SIEM, monitoring, incident response, threat detection |
| Compliance Officer | Regulatory compliance, audit support, compliance verification, DPDP Act compliance | Regulatory knowledge, compliance frameworks, audit |
Documentation and Evidence Requirements
Recommended Documentation
ISO 27001 does not mandate specific documents for this control. If you select it, keep the documented information you need to show it is planned and working (clause 7.5), and set retention in your own retention schedule. Typical records:
| Document | Purpose | Suggested retention | Owner |
|---|---|---|---|
| Environment Separation Policy | Governance framework | Per retention schedule | CISO |
| Network Architecture Diagram | Environment separation design | Current version + your retention period | Infrastructure Security Lead |
| Environment Inventory | List of all environments and their purpose | Current version + your retention period | DevOps Lead |
| Access Control Matrix | Who has access to what environment | Current version + your retention period | Infrastructure Security Lead |
| Environment Configuration Standards | Configuration baseline per environment | Current version + your retention period | DevOps Lead |
| Data Masking Procedure | How production data is masked for non-production | Current version + your retention period | DBA |
| Environment Promotion Procedure | How code/config flows through environments | Current version + your retention period | DevOps Lead |
| JIT Access Procedure | How just-in-time access is granted | Current version + your retention period | Infrastructure Security Lead |
| Break-Glass Procedure | Emergency access process | Current version + your retention period | Infrastructure Security Lead |
| IaC Repository | Infrastructure as Code for environment provisioning | Ongoing | DevOps Lead |
| Deployment Logs | Record of all deployments | Per retention schedule | DevOps Lead |
| Access Logs | Record of all environment access | Per retention schedule | Security Operations |
| Cross-Environment Activity Logs | Record of cross-environment traffic | Per retention schedule | Security Operations |
| Access Review Records | Quarterly access review evidence | Per retention schedule | Security Operations |
| Drift Detection Reports | Configuration drift findings | Per retention schedule | DevOps Lead |
| Incident Records | Environment separation incident documentation | Per retention schedule | Security Operations |
| Audit Records | Internal and external audit findings | Per retention schedule | CISO |
| Training Records | Staff training on environment separation | Per retention schedule | HR / Security |
| Shadow Environment Detection Records | Unauthorized environment findings | Per retention schedule | Security Operations |
| DLP Alerts | Data loss prevention alerts | 1 year | Security Operations |
| Environment Naming Convention | Standard naming rules | Current version | DevOps Lead |
| Emergency Deployment Records | Emergency deployment documentation | Per retention schedule | DevOps Lead |
| Rollback Records | Rollback execution records | Per retention schedule | DevOps Lead |
| Compliance Verification Records | Compliance attestation and verification | Per retention schedule | Compliance Officer |
Evidence for Audit
| Audit Question | Evidence Required |
|---|---|
| "Show me the environment separation policy" | Approved Environment Separation Policy |
| "How are environments separated?" | Network architecture diagram, firewall rules |
| "Who has access to production?" | Access control matrix, access review records |
| "How do you prevent production data in dev/test?" | Data masking procedure, DLP logs, synthetic data evidence |
| "Can developers deploy to production?" | CI/CD configuration, deployment logs, approval records |
| "How do you handle emergency access?" | Break-glass procedure, break-glass records |
| "Are deployment logs maintained?" | Deployment logs, CI/CD records |
| "How do you detect cross-environment traffic?" | SIEM logs, network monitoring, alerting rules |
| "Are environments provisioned consistently?" | IaC repository, environment provisioning records |
| "How do you handle vendor environment access?" | Vendor access records, environment separation for vendors |
Continuous Improvement
Improvement Cycle
Plan → Implement → Measure → Review → Improve
Plan: Set targets for separation coverage, access control, data masking, deployment success, incident reduction.
Implement: Deploy separation, access controls, data masking, monitoring, CI/CD gates, PAM.
Measure: Track KPIs, conduct surveys, analyze audit results, monitor incidents.
Review: Monthly metrics review, quarterly access review, annual complete review.
Improve: Update architecture, refine controls, adopt new tools, enhance training, automate compliance.
Improvement Triggers
| Trigger | Action |
|---|---|
| New application or system | Design environment separation from the start |
| Cloud migration | Re-architect separation for cloud-native model |
| New regulation | Update compliance requirements, data handling procedures |
| Security incident | Root cause analysis, update controls if separation gap contributed |
| Audit finding | Update process, architecture, or monitoring to address finding |
| Technology change | Evaluate new tools for environment management, PAM, DLP |
| Organizational change | Update access controls, review roles, retrain staff |
| Vendor change | Review vendor environment access, update separation controls |
| Scale growth | Re-architect separation for increased scale |
| Industry benchmark | Compare metrics, set improvement targets |
Maturity Advancement Path
| From Level | To Level | Key Actions | Typical Timeline |
|---|---|---|---|
| 1 (Ad-hoc) | 2 (Managed) | Inventory environments, implement basic network separation, create policy | 1–2 months |
| 2 (Managed) | 3 (Defined) | Standardize separation for all systems, implement access controls, data masking, CI/CD gates | 3–4 months |
| 3 (Defined) | 4 (Quantified) | Metrics tracked, PAM/JIT implemented, IaC deployed, drift detection, automated compliance | 3–4 months |
| 4 (Quantified) | 5 (Optimizing) | Dynamic environment provisioning, predictive monitoring, automated remediation, zero-trust environments | 6–12 months |
FAQ
Q1: Do we need physical separation or is logical separation enough?
A: Logical separation is sufficient for most organizations. Physical separation is typically required only for the highest security environments (classified systems, critical infrastructure). Cloud-native logical separation (VPCs, network policies, IAM) is the industry standard and fully compliant with ISO 27001.
Q2: How do we handle small teams that can't afford separate environments?
A: Use cloud-native separation with lightweight environments. AWS/Azure/GCP offer free tiers and lightweight dev/test environments. Containers (Docker, Kubernetes) allow multiple environments on shared hardware. The cost of a single production incident from environment mixing exceeds years of cloud environment costs.
Q3: Can developers ever access production?
A: Yes, but only through controlled mechanisms: read-only dashboards for troubleshooting, log access for debugging, and just-in-time (JIT) access for critical incidents with full audit logging. Standing production access for developers should be eliminated.
Q4: What is the difference between staging and production?
A: Staging is a production-like environment for final validation before go-live. It should mirror production configuration, data volume, and architecture but with masked data. Production is the live environment serving real users. Staging should be as close to production as possible to catch issues that only appear at scale.
Q5: How do we prevent configuration drift between environments?
A: Use Infrastructure as Code (Terraform, CloudFormation) for consistent provisioning. Use configuration management tools (Ansible, Chef) for consistent configuration. Implement automated drift detection. Enforce environment-specific configuration standards. Use container images that are the same across environments (only environment variables differ).
Q6: How do we handle data refreshes from production to staging?
A: Implement a secure data refresh pipeline: extract from production, mask/anonymize in a secure staging area, verify no real PII remains, transfer to staging, log the transfer, delete staging area data. Use tools like Delphix, Oracle Data Masking, or custom scripts. Never allow direct production-to-staging database replication without masking.
Q7: What is a "shadow environment" and how do we detect them?
A: Shadow environments are unauthorized environments created by developers or teams outside the approved process. Detect them using cloud cost monitoring (unexpected charges), cloud governance tools (AWS Config, Azure Policy), network scanning, and regular cloud security posture management (CSPM) scans.
Q8: How do we handle multi-tenant SaaS environment separation?
A: Separate tenant data at the application layer (tenant_id filtering), database layer (RLS, Row Level Security), or infrastructure layer (separate databases per tenant). Implement tenant isolation testing. Monitor for cross-tenant data access. Use API gateways to enforce tenant boundaries.
Q9: How do we handle AI/ML environment separation?
A: Separate training environments (broad access, large datasets) from inference environments (strict access, production data). Separate model artifacts (signed, versioned) from training code. Use different pipelines for training data and inference data. Monitor training environments for data leakage.
Q10: What is blue/green deployment and how does it relate to environment separation?
A: Blue/green deployment maintains two identical production environments (blue and green). Traffic flows to one while the other is updated. After validation, traffic switches to the updated environment. This reduces deployment risk and allows instant rollback. It requires strong separation between the active and standby environments.
Q11: How do we handle vendor access to our environments?
A: Vendors should have access only to dev/test environments for development. Production access should be time-limited, JIT, and fully audited. Vendor code should be reviewed and tested before staging deployment. Vendor personnel should be screened and trained. See A.8.30 for detailed vendor security guidance.
Q12: What is the role of CI/CD in environment separation?
A: CI/CD pipelines enforce environment separation by controlling what can be deployed where. Branch-to-environment mapping ensures only approved code reaches production. Deployment gates require approvals for staging and production. Automated testing in each environment prevents bad code from reaching production.
Q13: How do we measure environment separation effectiveness?
A: Track production incidents from cross-environment activity, production data in non-production, production credentials in non-production, unauthorized environment creation, deployment success rate, and compliance audit findings. See Section 13 for a full KPI framework.
Q14: What if we need to debug an issue that only occurs in production?
A: Use production logging, monitoring, and tracing (APM tools like Datadog, New Relic, Dynatrace) for debugging without direct access. If direct access is needed, use JIT access with session recording. Reproduce the issue in staging with similar data and load. Never enable debug mode in production.
Q15: How do we handle legacy systems that can't be separated?
A: For legacy systems that cannot be technically separated, implement compensating controls: time-based access (maintenance windows only), enhanced monitoring, restricted access lists, manual change controls, and enhanced logging. Plan for modernization with environment separation as a key requirement.
References and Further Reading
Standards and Guidelines
- ISO/IEC 27001:2022: Information Security, Cybersecurity and Privacy Protection, Information Security Management Systems, Requirements. ISO, 2022.
- ISO/IEC 27002:2022: Information Security, Cybersecurity and Privacy Protection, Information Security Controls. ISO, 2022.
- NIST SP 800-53 Rev 5: Security and Privacy Controls for Information Systems and Organizations. NIST, 2020.
- NIST SP 800-125: Guide to Security for Full Virtualization Technologies. NIST, 2011.
- NIST SP 800-125B: Secure Virtual Network Configuration for Virtual Machine (VM) Protection. NIST, 2016.
- PCI DSS v4.0.1: Payment Card Industry Data Security Standard. PCI SSC, 2024.
- CIS Controls v8: Center for Internet Security, 2021.
- OWASP Software Assurance Maturity Model (SAMM) v2.0. OWASP, 2020.
- BSIMM: Building Security In Maturity Model (use the latest edition).
- COBIT 2019: Control Objectives for Information and Related Technologies. ISACA, 2019.
Cloud Security Resources
- AWS Well-Architected Framework: Security Pillar: AWS, 2024.
- Azure Well-Architected Framework: Security: Microsoft, 2024.
- Google Cloud Architecture Framework: Security: Google, 2024.
- AWS Security Reference Architecture: AWS, 2024.
- Azure Security Benchmark: Microsoft, 2024.
- Google Cloud Security Foundations Guide: Google, 2024.
- Kubernetes Security Guide: CNCF, 2024.
- CIS Kubernetes Benchmark: Center for Internet Security, 2024.
Data Masking and Protection
- "Data Masking: A Complete Guide": Various, Oracle, 2023.
- Delphix Data Masking Best Practices: https://www.delphix.com/
- IBM InfoSphere Optim Data Masking: https://www.ibm.com/
- OWASP Data Masking Cheat Sheet: https://cheatsheetseries.owasp.org/
Indian Regulatory Resources
- Digital Personal Data Protection Act 2023: Government of India, 2023.
- RBI Master Direction on Information Technology Governance, Risk, Controls and Assurance Practices: Reserve Bank of India, 7 November 2023 (in force from 1 April 2024); and RBI Cyber Security Framework in Banks: circular of 2 June 2016.
- SEBI Cybersecurity and Cyber Resilience Framework (CSCRF) for SEBI Regulated Entities, circular of 20 August 2024.
- IRDAI Guidelines on Information and Cybersecurity: Insurance Regulatory and Development Authority of India, 2023.
- IT Act 2000 (as amended): Ministry of Electronics and Information Technology, India.
- MeitY Cloud Security Guidelines: Ministry of Electronics and Information Technology, India.
- DSCI Cloud Security Framework: Data Security Council of India, https://dsci.in
Industry Research
-
IBM Cost of a Data Breach Report 2024: IBM Security and Ponemon Institute, 2024.
-
Verizon Data Breach Investigations Report 2024: Verizon, 2024.
-
Capital One Data Breach Analysis: CISA, 2019.
-
GitLab Database Incident Postmortem: GitLab, 2017.