Getting multiple Azure Stack subscriptions to connect with each other is not as simple as in Azure. A customer of mine was planning to migrate their resources in a hybrid environment (Azure and Azure Stack) and for this they had some requirements for connecting the different subscriptions in the hybrid environment:

  • Every subscription has 1 Virtual Network.
  • The resources in Azure and Azure Stack must be able to connect the office and datacenter resources.
  • There are in total 6 subscriptions, 3 on Azure and 3 on Azure Stack:
  • Azure
    • DEVTST
    • ACCPRD
    • OPS
  • Azure Stack
    • DEVTST
    • ACCPRD
    • OPS
  • It must be possible from the OPS subscription, either from Azure or Azure Stack to connect to resources in the other subscriptions. The OPS subscription will have to CI/CD tooling to create/update resources, and monitoring.

I have tested different scenarios for the network design in a hub/spoke architecture, but Azure Stack has some limitations which make it difficult to implement. The first one is that Azure Stack currently does not support VNET peering. And because a VNET peering connection is non-transitive, it means that it will not forward the traffic to another VNET, like for example from an Azure VNET peering connection to an S2S VPN connection to Azure Stack. This is because the traffic flows through the peering configuration, rather than the gateway.

Another limitation is that Azure Stack uses 1 public IP for all their VPN connections in every Azure Stack subscription. So if for example I have created a S2S VPN connection between the OPS subscription in Azure Stack with the OPS subcription in Azure, it is not possible to create a S2S VPN connection between DEV/TST subscription in Azure Stack with the OPS subscription in Azure. This is because the DEV/TST subscription in Azure Stack will have the same public IP as the OPS subscription in Azure Stack.

To resolve this I have used S2S VPN's and User Defined Routes to connect the different VNETs with each other. Every Azure Stack subscription has an S2S VPN connection to the corresponding Azure subsciption. I've used the OPS Azure subscription like an central point where all the network traffic flows through. So for example traffic from OPS Azure Stack to DEVTST Azure flows from:

OPS Azure Stack > OPS Azure > DEVTST Azure > DEVTST Azure Stack

This is my setup: alt text

  • Azure
    • DEVTST
      • VNET: 10.160.128.0/17
      • VPN S2S Connection:
        • OPS
          • Local network: 10.129.128.0/17, 10.162.128.0/17
        • DEVTST
          • Local network: 10.142.0.0/17
      • Route table
        • Route: 10.129.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
    • OPS
      • VNET: 10.162.128.0/17
      • VPN S2S Connection:
        • OPS
          • Local network: 10.129.128.0/17
        • DEVTST
          • Local network: 10.142.0.0/17, 10.160.128.0/17
        • ACCPRD
          • Local network: 10.129.0.0/17, 10.162.0.0/17
      • Route table
        • Route: 10.129.0.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.142.0.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
    • ACCPRD
      • VNET: 10.162.0.0/17
      • VPN S2S Connection:
        • OPS
          • Local network: 10.129.128.0/17, 10.162.128.0/17
        • ACCPRD
          • Local network: 10.129.0.0/17
      • Route table
        • Route: 10.129.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
  • Azure Stack
    • DEVTST
      • VNET: 10.142.0.0/17
      • VPN S2S Connection:
        • DEVTST
          • Local network: 10.129.128.0/17, 10.160.128.0/17, 10.162.128.0/17
      • Route table
        • Route: 10.162.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.129.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
    • OPS
      • VNET: 10.162.128.0/17
      • VPN S2S Connection:
        • DEVTST
          • Local network: 10.129.0.0/17, 10.142.0.0/17, 10.160.128.0/17, 10.162.0.0/17, 10.162.128.0/17
      • Route table
        • Route: 10.162.0.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.160.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.129.0.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.142.0.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
    • ACCPRD
      • VNET: 10.162.0.0/17
      • VPN S2S Connection:
        • ACCPRD
          • Local network: 10.129.128.0/17, 10.162.0.0/17, 10.162.128.0/17
      • Route table
        • Route: 10.129.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet
        • Route: 10.162.128.0/17 - Next hop type: Virtual network gateway - Subnet: GatewaySubnet