# Blue-Green Deployments
## Overview
Blue-green deployment is a deployment strategy that provides zero-downtime deployments with manual validation gates and rollback capabilities. The pipeline templates implement a sophisticated blue-green deployment system specifically designed for Azure Web App container deployments.
[TOC]
## What is Blue-Green Deployment?
Blue-green deployment is a technique that reduces downtime and risk by running two identical production environments called Blue and Green. At any time, only one of the environments is live, with the other serving as a staging environment.
### Key Benefits
- **Zero Downtime**: Traffic is switched instantly between environments
- **Risk Mitigation**: Full validation in production-like environment before go-live
- **Manual Rollback**: Quick rollback capability available through manual intervention
- **Manual Control**: Human validation gates ensure quality before production release
## Architecture Overview
The blue-green deployment system consists of five main components organized as dependent Azure DevOps jobs:
```mermaid
graph TD;
A[Staging Deployment
Deploy to staging slot] --> B[Pre-Swap Validation
Manual validation gate
Default: 15 min timeout
Timeout action: Resume]
B -->|Approved| C[Production Swap
Swap staging to production]
B -->|Rejected| D[Deployment Cancelled]
C --> E[Post-Swap Validation
Manual validation gate
Default: 36 hour timeout
Timeout action: Resume]
E -->|Approved| F[Finalization
Success notification & cleanup]
E -->|Rejected| G[Automatic Rollback
Execute rollback workflow]
style A fill:#e1f5fe
style B fill:#fff3e0
style C fill:#e8f5e8
style D fill:#ffebee
style E fill:#fff3e0
style F fill:#e8f5e8
style G fill:#ffebee
```
## Configuration
### Basic Configuration Structure
Blue-green deployment is configured at the top-level pipeline parameters and passed down to the orchestrator:
```yaml
# Top-level pipeline (e.g., azure-dotnet-api-v2.yml)
parameters:
- name: projectId
type: string
- name: applicationName
type: string
- name: blueGreen
displayName: 'Blue-Green Deployment Configuration'
type: object
default:
enabled: false
preSwapValidation:
timeout: 15 # Minutes
onTimeout: 'resume' # 'resume' or 'reject'
notifyUsers: |
team@company.com
instructions: |
๐งช **STAGING ENVIRONMENT VALIDATION**
The container has been deployed to the staging slot and health checks have passed.
Please review the deployment logs and monitoring before proceeding to production:
- Verify container health check is successful
- Review deployment logs for any warnings or errors
- Check monitoring dashboards for expected metrics
โ
Click "Resume" if validation passes
โ Click "Reject" to cancel deployment
postSwapValidation:
timeout: 2160 # Minutes (36 hours)
onTimeout: 'resume'
notifyUsers: |
team@company.com
instructions: |
๐ **PRODUCTION ENVIRONMENT VALIDATION**
The deployment has been swapped to production. Please validate:
- Container health check shows healthy status
- Monitoring dashboards show normal metrics
- No critical errors in application logs
โ ๏ธ This will auto-approve in 36 hours unless rejected.
stages:
- template: /v2/orchestrators/azure-app-orchestrator-v2.yml
parameters:
projectId: ${{ parameters.projectId }}
applicationName: ${{ parameters.applicationName }}
blueGreen: ${{ parameters.blueGreen }}
# ... other parameters
```
### Configuration Parameters
| Parameter | Type | Default | Description |
| ------------------------------------------- | ------- | ------------ | ------------------------------------------------------------------------------------- |
| `blueGreen.enabled` | boolean | false | Enable blue-green deployment for this pipeline |
| `blueGreen.preSwapValidation.timeout` | number | 15 | Pre-swap validation timeout in minutes (maximum: 2880 - 48 hours) |
| `blueGreen.preSwapValidation.onTimeout` | string | 'resume' | Action on timeout: 'resume' or 'reject' |
| `blueGreen.preSwapValidation.notifyUsers` | string | - | Email addresses for validation notifications |
| `blueGreen.preSwapValidation.instructions` | string | Default text | Custom instructions for validators |
| `blueGreen.postSwapValidation.timeout` | number | 2160 | Post-swap validation timeout in minutes (default: 36 hours, maximum: 2880 - 48 hours) |
| `blueGreen.postSwapValidation.onTimeout` | string | 'resume' | Action on timeout: 'resume' or 'reject' |
| `blueGreen.postSwapValidation.notifyUsers` | string | - | Email addresses for validation notifications |
| `blueGreen.postSwapValidation.instructions` | string | Default text | Custom instructions for validators |
## Deployment Flow
The blue-green deployment follows a strict sequence of dependent jobs:
1. **Staging Deployment** - Deploy new version to staging slot and validate
2. **Pre-Swap Validation** - Manual validation of staging environment (default: 15 min timeout)
3. **Production Swap** - Swap staging and production slots
4. **Post-Swap Validation** - Manual validation of production environment (default: 36 hours timeout)
5. **Finalization OR Rollback** - Success notification and cleanup, or automatic rollback if validation fails
### Job Details
#### Staging Deployment
- **Type**: Agent job
- **Purpose**: Deploy container to staging slot and validate deployment
#### Pre-Swap Validation
- **Type**: Agentless server job
- **Purpose**: Manual validation before production release
- **Features**: Email notifications, configurable timeout (max: 48 hours), auto-approve on timeout
#### Production Swap
- **Type**: Agent job
- **Purpose**: Swap staging slot to production and validate swap
#### Post-Swap Validation
- **Type**: Agentless server job
- **Purpose**: Manual validation of live production environment
- **Features**: Email notifications, configurable timeout (max: 48 hours), triggers automatic rollback if rejected
#### Finalization or Rollback
- **Finalization** (if approved): Success notification and cleanup
- **Automatic Rollback** (if rejected): Executes service-specific rollback workflow
## Manual Validation Gates
The pipeline includes two critical manual validation points with configurable timeouts and email notifications.
### Validation Configuration
Both validation gates support:
- **Email Notifications**: Configurable recipient list
- **Custom Instructions**: Tailored validation guidance
- **Timeout Settings**: Default or custom timeout periods (max: 48 hours)
- **Auto-Action**: Auto-approve or auto-reject on timeout
### Pre-Swap Validation
Validates the staging deployment before production release:
- **Environment**: Staging slot with production configuration
- **Default Timeout**: 15 minutes
- **Purpose**: Ensure deployment health before going live
**Validation Checklist:**
- [ ] Container health check is successful
- [ ] Deployment completed without errors
- [ ] No critical warnings in deployment logs
- [ ] Monitoring shows healthy container metrics
- [ ] Required environment variables are set correctly
### Post-Swap Validation
Validates the live production environment with automatic rollback capability:
- **Environment**: Live production environment
- **Default Timeout**: 36 hours
- **Purpose**: Validate production functionality and trigger rollback if needed
- **Critical Feature**: **Rejection triggers automatic rollback**
**Validation Checklist:**
- [ ] Production swap completed successfully
- [ ] Container health check shows healthy status
- [ ] Monitoring dashboards show normal metrics
- [ ] No critical errors in application logs
- [ ] Performance meets SLA requirements
- [ ] External integrations are working
### Notification Configuration
```yaml
notifyUsers: |
team@company.com,
qa-team@company.com
```
Notifications include deployment details, custom instructions, approval/rejection options, and timeout information.
## Rollback Mechanism
The blue-green deployment system provides automatic rollback when post-swap validation is rejected:
1. **Detection**: Post-swap validation gate is rejected by the approver
2. **Automatic Trigger**: Pipeline automatically starts the rollback job
3. **Rollback Execution**: Executes service-specific rollback workflow to restore previous version
**Key Features:**
- Zero manual intervention required for rollback execution
- Service-specific rollback workflows tailored to each service type
- Immediate rollback reduces production impact time
- Consistent and reliable rollback procedure
## Implementation Examples
### Basic Implementation
```yaml
# azure-pipelines.yml in consuming repository
extends:
template: azure-dotnet-api-v2.yml@templates
parameters:
projectId: 'my-api-project'
applicationName: 'my-production-api'
# ... other parameters
blueGreen:
enabled: true
preSwapValidation:
timeout: 30
notifyUsers: |
qa-team@company.com
instructions: |
๐งช **STAGING VALIDATION REQUIRED**
Please validate the staging deployment:
1. Verify container health check is successful
2. Check deployment logs for errors
3. Validate monitoring dashboards show healthy metrics
postSwapValidation:
timeout: 1440 # 24 hours
notifyUsers: |
team@company.com
instructions: |
๐ **PRODUCTION VALIDATION**
The application is now live in production.
Monitor for 24 hours.
```
### Different Service Types
The blue-green configuration works across different pipeline templates:
- **API Services**: `azure-dotnet-api-v2.yml`
- **Web Applications**: `azure-react-web-v2.yml`
### Support and Resources
- **Azure App Service Slots**: [Understanding deployment slot swaps](https://learn.microsoft.com/en-us/azure/app-service/deploy-staging-slots?tabs=portal#understand-what-happens-during-a-swap)
- **Manual Validation Task**: [Azure DevOps Manual Validation documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/reference/manual-validation-v1?view=azure-pipelines)