Staging Environment - VNet Integration & Performance Improvements

Full specification for upgrading the FEG staging environment with Azure Virtual Network integration, private endpoints for Redis, upgraded SWA tiers, and Container App sizing improvements.

Last updated: 2026-04-01

---

Table of Contents

1. Problem Statement
2. Current Architecture (Dev)
3. Target Architecture (Staging)
4. Detailed Changes
   • 4.1 VNet & Subnet Provisioning
   • 4.2 Container Apps Environment with VNet
   • 4.3 Redis Private Endpoint
   • 4.4 Private DNS Zone for Redis
   • 4.5 SWA Upgrade to Standard
   • 4.6 Container App Sizing
5. Network Topology
6. Pulumi Code Changes
   • 6.1 feg_slot_be/infra Changes
   • 6.2 feg_common/infra Changes
   • 6.3 Pulumi.staging.yaml Updates
7. Deployment Order
8. Configuration Checklist
9. Cost Estimate
10. Rollback Plan
11. Constraints & Decisions

---

1. Problem Statement

The dev team reports two issues:

• Frontend asset loading slowness: SWA Free tier has limited CDN edge PoPs and 100 GB/month bandwidth
• Intermittent backend slowness: CMS scales to zero (26s cold start), all traffic traverses public internet (Container Apps to Redis, Container Apps to MongoDB Atlas), and container sizing is minimal (0.25 vCPU / 0.5 Gi)

For the new staging environment, we want fast, consistent performance. The agreed approach is:

1. VNet integration for the Container Apps Environment (external ingress retained - apps still get public FQDNs)
2. Private endpoint for Redis Enterprise (traffic stays on Azure backbone)
3. SWA Standard SKU for better CDN and custom domain support
4. Increased container sizing (0.5 vCPU / 1 Gi, minReplicas: 1 for pack-alpha/rgs/external, minReplicas: 0 for CMS)
5. ACR stays Basic, Redis stays Balanced_B0 (no tier changes)

---

2. Current Architecture (Dev)

Traffic flow today (all public internet):

Browser -> SWA (West Europe CDN) -> Container App (Germany West Central, public IP)
                                        |
                                        +-> Redis Enterprise (public endpoint, TLS, port 10000)
                                        +-> MongoDB Atlas (public endpoint, TLS)

---

3. Target Architecture (Staging)

Browser -> SWA Standard (West Europe, improved CDN) -> Container App (Germany West Central, public FQDN)
                                                            |
                                                            | (VNet - internal traffic)
                                                            |
                                                            +-> Redis Enterprise (private endpoint, TLS, port 10000)
                                                            +-> MongoDB Atlas (public endpoint, TLS) *

* MongoDB Atlas private endpoint is out of scope for this iteration. Atlas Private Link requires M10+ dedicated clusters and Atlas configuration changes beyond Azure infra.

Key architectural decisions:

• Container Apps Environment is VNet-integrated but NOT internal (internal: false). Apps still get public FQDNs - no need for Application Gateway or Front Door as a reverse proxy
• Redis gets a private endpoint inside the VNet, so Container App -> Redis traffic stays on Azure backbone
• A Private DNS Zone (privatelink.redisenterprise.cache.azure.net) is created and linked to the VNet so the existing Redis hostname resolves to the private IP within the VNet
• MongoDB Atlas continues over public internet (private link upgrade deferred to production)

---

4. Detailed Changes

4.1 VNet & Subnet Provisioning

Create a new VNet in Germany West Central with two subnets:

Why /23 for CAE subnet: Azure Container Apps requires a minimum of /23 (512 IPs) for the infrastructure subnet. This subnet is fully managed by Azure - it provisions internal load balancers, Envoy proxies, and node IPs within it.

Why a separate /24 for endpoints: Private endpoints need their own subnet. Keeping them separate from the CAE infrastructure subnet avoids conflicts and follows Azure best practices.

Subnet delegations:

• snet-cae-staging - Delegation required: Microsoft.App/environments. Azure Container Apps Environment with VNet integration requires explicit subnet delegation to this service.
• snet-endpoints-staging - NO delegation needed; private endpoints do not require subnet delegation

4.2 Container Apps Environment with VNet

The new staging Container Apps Environment will be created with VNet configuration:

vnetConfiguration: {
  infrastructureSubnetId: snetCae.id,
  internal: false,  // Apps still get public FQDNs
}

What this changes:

• All Container App replicas run inside the VNet subnet
• Inter-service calls (e.g., pack-alpha -> feg-rgs) go through the internal Envoy mesh within the VNet (no public internet hop)
• Outbound traffic from Container Apps to resources with private endpoints stays within the Azure backbone
• Apps retain public FQDNs and are directly accessible from the internet (no Application Gateway needed)

What this does NOT change:

• Public ingress still works identically
• CI/CD deployment via az containerapp update works identically
• No changes to application code

4.3 Redis Private Endpoint

Create a private endpoint for the Redis Enterprise cluster in the endpoints subnet:

Target sub-resource: redisEnterprise (for Redis Enterprise clusters)

Result:

• Private endpoint gets a private IP (e.g., 10.0.2.4) in the endpoints subnet
• Traffic from Container Apps to Redis goes: Container App -> VNet -> Private Endpoint NIC -> Redis (all on Azure backbone)
• TLS is still used (no change to clientProtocol: Encrypted)
• Port remains 10000

4.4 Private DNS Zone for Redis

For the private endpoint to work transparently (without changing app connection strings), we need a Private DNS Zone:

How it works:

1. App connects to feg-dev.germanywestcentral.redis.azure.net (same connection string as today)
2. Azure public DNS returns a CNAME: feg-dev.germanywestcentral.redis.azure.net -> feg-dev.privatelink.redisenterprise.cache.azure.net
3. The private DNS zone (linked to VNet) resolves feg-dev.privatelink.redisenterprise.cache.azure.net -> 10.0.2.x (private IP)
4. Traffic stays entirely within Azure backbone

No application code or config changes needed - the existing Redis hostname/port/password all remain the same.

4.5 SWA Upgrade to Standard

Cost: ~$9/month per SWA = ~$18/month total for 2 SWAs.

How geo-distribution actually works:

Azure SWA Standard uses Azure Traffic Manager (not Azure Front Door) for geo-distribution. Traffic Manager resolves each user's DNS request to the nearest regional SWA hosting node. There is no traditional CDN edge cache - assets are served from the regional origin node closest to the user.

Verified via curl and dig:

• Custom domain resolves to a flat A record (20.2.51.235)
• Default SWA hostname resolves through azurestaticapps6.trafficmanager.net -> regional node -> same IP
• No X-Cache or X-Azure-Ref headers in responses - confirms no AFD edge caching layer

This is a meaningful improvement over a single-region origin (dev had no geo-routing), but it is not a traditional CDN with edge caching. For true edge caching in front of SWA, Azure Front Door Premium ("Enterprise-grade edge") would be needed - that is out of scope for staging.

Verification: curl -sI https://feg-fe-staging.pixentech.com/ - confirm HTTP 200 and check cache-control header. No X-Cache header is expected (Traffic Manager, not AFD).

4.6 Container App Sizing

CMS keeps minReplicas: 0 as agreed - cold starts are acceptable for CMS in staging.

---

5. Network Topology

                          Internet
                             |
                    [Public FQDN Ingress]
                             |
    +========================|========================+
    |  VNet: feg-slot-vnet-staging (10.0.0.0/16)      |
    |                        |                         |
    |  +--  snet-cae-staging (10.0.0.0/23) -----------+|
    |  |                                               |
    |  |  Container Apps Environment                   |
    |  |    - pack-alpha-staging (public FQDN)         |
    |  |    - feg-rgs-staging    (public FQDN)         |
    |  |    - feg-cms-staging    (public FQDN)         |
    |  |    - feg-external-staging (public FQDN)       |
    |  |                                               |
    |  +-----------------------------------------------+
    |                        |
    |                   (VNet traffic)
    |                        |
    |  +-- snet-endpoints-staging (10.0.2.0/24) ------+|
    |  |                                               |
    |  |  Private Endpoint: pe-redis-staging           |
    |  |    -> feg-dev Redis Enterprise (10.0.2.x)     |
    |  |                                               |
    |  +-----------------------------------------------+
    |                                                   |
    +===================================================+
                             |
                   (public internet, TLS)
                             |
                    MongoDB Atlas (public)

---

6. Pulumi Code Changes

6.1 feg_slot_be/infra Changes

Files to modify:

• infra/src/config.ts - Add VNet, subnet, and Redis private endpoint config variables
• infra/src/index.ts - Add VNet, subnets, private endpoint, private DNS zone resources; pass VNet config to Container Apps Environment
• infra/src/types.ts (create) - Not needed, feg_slot_be/infra doesn't have a types file

New config variables (in config.ts):

// ── VNet integration (staging/prod) ─────────────────────────────
/** Whether to create a VNet and integrate the Container Apps Environment */
export const enableVnet = cfg.get('enableVnet') === 'true';

/** VNet address space (default: 10.0.0.0/16) */
export const vnetAddressPrefix = cfg.get('vnetAddressPrefix') ?? '10.0.0.0/16';

/** Container Apps Environment infrastructure subnet (minimum /23) */
export const caeSubnetPrefix = cfg.get('caeSubnetPrefix') ?? '10.0.0.0/23';

/** Private endpoints subnet */
export const endpointsSubnetPrefix = cfg.get('endpointsSubnetPrefix') ?? '10.0.2.0/24';

// ── Redis private endpoint ──────────────────────────────────────
/** Whether to create a private endpoint for Redis */
export const enableRedisPrivateEndpoint = cfg.get('enableRedisPrivateEndpoint') === 'true';

/**
 * Full Azure resource ID of the Redis Enterprise cluster to link.
 * Required when enableRedisPrivateEndpoint is true.
 * Example: /subscriptions/.../providers/Microsoft.Cache/redisEnterprise/feg-dev
 */
export const redisResourceId = cfg.get('redisResourceId');

New resources in index.ts (conditionally created when enableVnet === true):

1. VNet (Microsoft.Network/virtualNetworks)
2. Subnet: snet-cae (10.0.0.0/23) - for Container Apps
3. Subnet: snet-endpoints (10.0.2.0/24) - for private endpoints
4. Container Apps Environment - modified to include vnetConfiguration.infrastructureSubnetId
5. Private Endpoint for Redis (if enableRedisPrivateEndpoint === true)
6. Private DNS Zone (privatelink.redisenterprise.cache.azure.net)
7. VNet Link for the DNS zone
8. Private DNS Zone Group on the private endpoint (auto-creates A record)

Container Apps Environment change:

// Current (dev):
const containerAppsEnv = new app.ManagedEnvironment(`feg-slot-env-${env}`, {
  environmentName: `feg-slot-env-${env}`,
  resourceGroupName: resourceGroup.name,
  appLogsConfiguration: { ... },
  zoneRedundant: env === 'prod',
  workloadProfiles: [{ name: 'Consumption', workloadProfileType: 'Consumption' }],
});

// New (staging with VNet):
const containerAppsEnv = new app.ManagedEnvironment(`feg-slot-env-${env}`, {
  environmentName: `feg-slot-env-${env}`,
  resourceGroupName: resourceGroup.name,
  appLogsConfiguration: { ... },
  zoneRedundant: env === 'prod',
  workloadProfiles: [{ name: 'Consumption', workloadProfileType: 'Consumption' }],
  // VNet integration (staging/prod only)
  ...(config.enableVnet && vnet && snetCae
    ? {
        vnetConfiguration: {
          infrastructureSubnetId: snetCae.id,
          internal: false,
        },
      }
    : {}),
});

Exports to add:

// VNet outputs (for feg_common/infra or diagnostics)
if (config.enableVnet) {
  exports.vnetId = vnet.id;
  exports.caeSubnetId = snetCae.id;
  exports.endpointsSubnetId = snetEndpoints.id;
}

6.2 feg_common/infra Changes

No code changes needed for feg_common/infra.

The common infra project creates Container Apps that are deployed into the Container Apps Environment. Since the Environment itself is VNet-integrated (done by feg_slot_be/infra), all Container Apps automatically inherit the VNet. The containerAppsEnvironmentId reference in the staging config will point to the new VNet-integrated environment.

The Redis private endpoint is also transparent - the same hostname resolves to a private IP inside the VNet via the Private DNS Zone. No connection string changes.

6.3 Pulumi.staging.yaml Updates

feg_slot_be/infra/Pulumi.staging.yaml - add these new keys:

# ── VNet integration ──────────────────────────────────────────
feg-slot-infra:enableVnet: 'true'
feg-slot-infra:vnetAddressPrefix: '10.0.0.0/16'
feg-slot-infra:caeSubnetPrefix: '10.0.0.0/23'
feg-slot-infra:endpointsSubnetPrefix: '10.0.2.0/24'
# ── Redis private endpoint ────────────────────────────────────
feg-slot-infra:enableRedisPrivateEndpoint: 'true'
feg-slot-infra:redisResourceId: '/subscriptions/5cd9c9d4-48a1-4ae3-88b5-39caac27bd1e/resourceGroups/feg-slot-rg-dev/providers/Microsoft.Cache/redisEnterprise/feg-dev'
# NOTE: If staging gets its own Redis, update this resource ID
# ── SWA ────────────────────────────────────────────────────
feg-slot-infra:deployFrontend: 'true'
feg-slot-infra:frontendSku: Standard
feg-slot-infra:deployCmsFrontend: 'true'
feg-slot-infra:cmsFrontendSku: Standard

feg_common/infra/Pulumi.staging.yaml - update CMS minReplicas:

# Override CMS to keep scale-to-zero
feg-common-infra:cmsMinReplicas: '0'

feg_slot_be/infra/Pulumi.dev.yaml - NO changes (dev stays without VNet).

---

7. Deployment Order

Since staging is a new environment (new Pulumi stack), the order is:

Step 1: Initialize Pulumi staging stack for feg_slot_be/infra
          cd feg_slot_be/infra
          pulumi stack init staging
          # (Pulumi.staging.yaml already exists with config)

Step 2: Deploy feg_slot_be/infra (staging)
          pulumi up --stack staging
          Creates: Resource Group, ACR (shared), Identity, RBAC,
                   Log Analytics, VNet + subnets,
                   Container Apps Environment (VNet-integrated),
                   pack-alpha Container App,
                   Redis private endpoint + DNS zone,
                   SWA Standard x2

Step 3: Capture outputs from Step 2
          pulumi stack output --stack staging
          Note: containerAppsEnvironmentId, managedIdentityId, managedIdentityClientId

Step 4: Update feg_common/infra/Pulumi.staging.yaml with outputs from Step 3
          Set: containerAppsEnvironmentId, managedIdentityId, managedIdentityClientId

Step 5: Initialize and deploy feg_common/infra (staging)
          cd feg_common/infra
          pulumi stack init staging
          # Set secrets:
          pulumi config set --secret feg-common-infra:rgsMongodbUri <value>
          pulumi config set --secret feg-common-infra:redisPassword <value>
          # ... (other secrets)
          pulumi up --stack staging
          Creates: feg-rgs-staging, feg-cms-staging, feg-external-staging

Step 6: Trigger CI pipelines to push real images to ACR with staging tags
          (until then, use placeholder images or manually push)

Step 7: Verify connectivity
          - curl each Container App FQDN
          - Verify Redis connectivity (check app logs for successful connection)
          - Verify MongoDB connectivity

---

8. Configuration Checklist

Before pulumi up for feg_slot_be/infra (staging):

• Pulumi.staging.yaml has enableVnet: "true"
• Pulumi.staging.yaml has enableRedisPrivateEndpoint: "true"
• Pulumi.staging.yaml has correct redisResourceId (if sharing dev Redis) or a new staging Redis is created separately
• Pulumi.staging.yaml has deployFrontend: "true" and frontendSku: Standard
• Pulumi.staging.yaml has deployCmsFrontend: "true" and cmsFrontendSku: Standard
• Secrets are set: pulumi config set --secret feg-slot-infra:rgsUrl <value>
• ACR SKU stays "Basic" (shared across all environments)

Before pulumi up for feg_common/infra (staging):

• containerAppsEnvironmentId is set from feg_slot_be/infra outputs
• managedIdentityId is set from feg_slot_be/infra outputs
• managedIdentityClientId is set from feg_slot_be/infra outputs
• redisHost is set (same as dev: feg-dev.germanywestcentral.redis.azure.net)
• redisPort is set to 10000 (Redis Enterprise)
• redisTls is set to "true"
• redisClusterMode is set to "false" (even though OSSCluster, apps use non-cluster client)
• All secrets set: rgsMongodbUri, cmsMongodbUri, externalMongodbUri, cmsJwtSecret, redisPassword
• cmsMinReplicas is set to "0" to keep CMS at scale-to-zero

Post-deploy verification:

• Container Apps are running (check az containerapp list -g feg-slot-rg-staging)
• VNet exists and subnets are configured (az network vnet show -n feg-slot-vnet-staging -g feg-slot-rg-staging)
• Private endpoint is connected (az network private-endpoint show -n pe-redis-staging -g feg-slot-rg-staging)
• DNS resolution test: from within a Container App, feg-dev.germanywestcentral.redis.azure.net resolves to a 10.0.2.x private IP
• Redis connectivity confirmed in app startup logs (no ECONNREFUSED or ETIMEDOUT)
• SWA is accessible and assets load with good TTFB (< 200ms for cached assets)
• CDN latency verified from Czech/Slovak IP: TTFB < 50ms for cached frontend assets (Prague AFD edge)
• CDN latency verified from India IP: TTFB < 50ms for cached frontend assets (Mumbai AFD edge)
• Health probes are passing for all Container Apps

---

9. Cost Estimate

Additional Monthly Cost (Staging vs. Dev)

Note: If staging gets its own Redis Enterprise instance, add ~$60/month for Balanced_B0.

---

10. Rollback Plan

Since staging is a new environment with a new Pulumi stack, rollback is straightforward:

# Destroy all staging resources
cd feg_common/infra && pulumi destroy --stack staging
cd feg_slot_be/infra && pulumi destroy --stack staging

No impact on dev - staging is a completely separate resource group (feg-slot-rg-staging).

The only shared resource is ACR (fegslotacr), which is not modified - only new image tags are pushed.

---

11. Constraints & Decisions

---

Appendix A: Azure Resource Dependencies

Resource Group (feg-slot-rg-staging)
  |
  +-- VNet (feg-slot-vnet-staging)
  |     |
  |     +-- Subnet: snet-cae-staging (10.0.0.0/23)
  |     |     |
  |     |     +-- Container Apps Environment (feg-slot-env-staging)
  |     |           |
  |     |           +-- pack-alpha-staging
  |     |           +-- feg-rgs-staging     (from feg_common/infra)
  |     |           +-- feg-cms-staging     (from feg_common/infra)
  |     |           +-- feg-external-staging(from feg_common/infra)
  |     |
  |     +-- Subnet: snet-endpoints-staging (10.0.2.0/24)
  |           |
  |           +-- Private Endpoint: pe-redis-staging
  |                 |
  |                 +-- NIC (auto, private IP)
  |                 +-- Private DNS Zone Group -> DNS A record
  |
  +-- Private DNS Zone: privatelink.redisenterprise.cache.azure.net
  |     |
  |     +-- VNet Link -> feg-slot-vnet-staging
  |     +-- A Record: feg-dev -> 10.0.2.x (auto from zone group)
  |
  +-- ACR (fegslotacr, shared)
  +-- Managed Identity (feg-slot-id-staging)
  +-- Role Assignment (AcrPull)
  +-- Log Analytics (feg-slot-logs-staging)
  +-- SWA: feg-slot-fe-staging (Standard)
  +-- SWA: feg-cms-fe-staging (Standard)

Appendix B: Pulumi Resource Mapping

All other resources are unchanged from the current Pulumi code.