Recently my company was looking at AWS for hosting their kubernetes and openshift platforms. The first thing to do when starting with AWS is to create an account structure strategy, that is secure, manageable and flexible. After looking at several setups we have chosen to use an multi account approach. This has several benefits instead of using an single account approach:

  • Security (not stepping on each other toes, separation of duties)
  • Access to AWS services without too much barriers
  • Ability to make mistakes and fail fast without collateral damage
  • Smaller blast radius for security incidents
  • Operations team to become Cloud Architects
  • Costs and resources tracked to individuals
  • All platform IT will be centrally organized
  • Automated, self-service and designed for scale
  • Can give the department/squads a balance between security and freedom

This blog I will explain how we create and configure or update new accounts in AWS with infrastructure as code. For this I will be using Azure DevOps and Powershell. There are some requirement that have to be met so the users can get access and use the accounts:

  • IAM: All users can only get access to AWS through Azure AD (SAML), for this we used the AWS Multi-Account approach: https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/aws-multi-accounts-tutorial. For this you need an Azure AD Application as described in the link. Every AWS account is connected to the same Azure AD application. Every custom role in an AWS Account is added to the Azure AD application as an AppRole, this is necessary so you can link a role to an AAD user or AAD group. Users get access by adding users to Azure AD Groups. The Groups are added in the Azure AD Application
  • Accounts: We have chosen for a account strategy based on solution/tribe and environment, with the prefix VCO-AWS, the name of the solution/tribe and then the environment. Every AWS Account name must end with 1 of the following environtment abbreviations:
    • DEV
    • TST
    • ACC
    • PRD
    • SANDBOX

Examples for the AWS Account names: - VCO-AWS-FIT-DEVTST - VCO-AWS-MIDDLEWARE-SANDBOX - VCO-AWS-MIDDLEWARE-PRD Every AWS Account name cannot have more than 27 characters, this is because it gives an error when creating an AppRole in the Azure AD application. The total string for the AppRole then exceeds the maximum characters allowed.

  • Roles & Policies
    • Every custom role created has the prefix “FIT-“, so it is easy to find and recognize the custom roles created by our department.
    • We have custom roles, 1 for the AWS users and 1 for the administrators. Every custom role has 1 or more custom policies
    • Every AWS Account gets the custom roles with global custom policies, except for when the AWS Account needs special custom policies. The global custom policies are saved in the GIT repository
      • PRD-Users.json
      • PRD-Blacklist.json
      • PRD-Full.json
      • OTA-Users.json
      • OTA-Blacklist.json
      • OTA-Full.json
    • If an AWS Account name ends with “DEV” or “TST” or “ACC” or “SANDBOX”, the 3 OTA-… json files are being used. If an AWS account name ends with “PRD”. The PRD-… json files are being used.
    • If an AWS Account has special rights, these custom policies will have to be saved in the code repository as json files. The name for the JSON file has to be the same as the policy name in the AWS account. For example:
      • FIT-MIDDLEWARE-SANDBOX-Users.json
      • FIT-MIDDLEWARE-SANDBOX-Blacklist.json
    • The scripts checks if there are custom policies for the AWS account in the GIT repository, if not then it uses the global custom policies.
  • AAD Groups
    • The script creates an Azure AD group with prefix CGP- and the solution/tribe name. Users then can be added to Azure Active Directory group, and the group can be added to the Azure AD Application to give access to AWS. You can link the AAD group with an AWS role in the Azure AD Application.
  • Microsoft Graph
    • For creating creating/updating AppRoles in the Azure AD Application & creating/updating the Azure AD Groups, I have created another Azure AD Application to connect to the Microsoft Graph. This Azure AD application has minimum required rights to connect to the AWS SSO Azure AD Application and minimum rights to Azure AD to create/update Groups.

The solution consists of the following files:

  • AWS-Manage-AccountStructure.ps1
  • The global custom policies and account specific custom policies in json format
  • VCO-AWS-SSO.xml (necessary for creating the SAML SSO connection with Azure AD. You can get this xml file from the Azure AD application created to setup SSO)

Here is a picture of the flow: alt text

  1. Azure DevOps > Build deployment > Run Build
  2. The Build runs a PowerShell script:
  • If ENV variable is added when running the PowerShell script > it uses the variable as Account name
    • Logs in with an service account in the Master Account
    • Checks if the Account name is longer than 27 characters (the Account name cannot exceed 27 characters, because that will cause errors when adding the Approles in the Azure AD application)
    • The new Account name gets the prefix “VCO-AWS”
    • Checks is account already exists
      • If the account already exists, it uses Use-STSRole to use the OrganizationAccountAccessRole in the Account, then it updates the Account’s the SAML provider, the custom Roles and Policies, Creates an AAD Approle and AADgroup.
      • If the account does not exists in AWS, the new account is created, the Use-STSRole is used in the Master Account to use the OrganizationAccountAccessRole in the Account. Then it creates the Account’s the SAML provider, the custom Roles and Policies, Creates an AAD Approle and AADgroup.
  • If no ENV variable is added when running the PowerShell script > it checks every existing account and updates the account / roles
    • It loops to each account (except the master account), assumes the OrganizationAccountAccessRole in the account and updates:
      • SAMLprovider
      • Roles and Policies
      • AADAppRole
      • AADGroup
      • Email alias

Prerequisites

  • Configure an custom Azure DevOps build agent with the following powershell modules:
    • Az modules
    • AWS Powershell modules
  • An IAM user in the Master account. This IAM user is used to connect to and modify the other accounts. The IAM must have the following:
    • Only programmatic access
    • The following permissions:
{
	"Version": "2012-10-17",
	"Statement": [
	{
		"Sid": "VisualEditor0",
		"Effect": "Allow",
		"Action": [
			"organizations:*",
			"iam:GetAccountPasswordPolicy",
			"organizations:ListRoots",
			"iam:GetServiceLastAccessedDetailsWithEntities",
			"iam:GenerateServiceLastAccessedDetails",
			"iam:ListPoliciesGrantingServiceAccess",
			"organizations:CreateAccount",
			"organizations:DeleteOrganization",
			"iam:GetServiceLastAccessedDetails",
			"iam:SetSecurityTokenServicePreferences",
			"organizations:EnableAWSServiceAccess",
			"organizations:ListCreateAccountStatus",
			"organizations:DescribeOrganization",
			"organizations:CreateGovCloudAccount",
			"organizations:EnableAllFeatures",
			"iam:CreateAccountAlias",
			"iam:GetAccountAuthorizationDetails",
			"iam:DeleteAccountAlias",
			"organizations:CreatePolicy",
			"organizations:DescribeCreateAccountStatus",
			"organizations:CreateOrganization",
			"iam:ListPolicies",
			"iam:DeleteAccountPasswordPolicy",
			"iam:ListSAMLProviders",
			"organizations:DisableAWSServiceAccess",
			"iam:ListRoles",
			"sts:*",
			"organizations:ListAWSServiceAccessForOrganization",
			"organizations:ListPolicies",
			"iam:GetContextKeysForCustomPolicy",
			"organizations:ListHandshakesForOrganization",
			"organizations:LeaveOrganization",
			"organizations:ListHandshakesForAccount",
			"organizations:ListAccounts",
			"iam:UpdateAccountPasswordPolicy",
			"iam:ListOpenIDConnectProviders",
			"iam:ListAccountAliases",
			"iam:ListUsers",
			"iam:ListGroups",
			"iam:GetAccountSummary"
		],
		"Resource": "*"
	},
	{
		"Sid": "VisualEditor1",
		"Effect": "Allow",
		"Action": "iam:*",
		"Resource": [
			"arn:aws:iam::*:saml-provider/*",
			"arn:aws:iam::*:policy/*",
			"arn:aws:iam::*:oidc-provider/*",
			"arn:aws:iam::*:instance-profile/*",
			"arn:aws:iam::*:user/*",
			"arn:aws:iam::*:role/*",
			"arn:aws:iam::*:server-certificate/*",
			"arn:aws:iam::*:group/*"
		]
	}
	]
}
  • Access key and secret created. The access key and secret are being used in the script for managing the accounts.
  • An Azure AD Application to manage/modify the Azure AD AWS SSO Application. The Azure AD Application must have the following:
    • A client secret. The Client ID and client secret are necessary for the powershell script.
    • Must have the following API permissions:
      • Application.ReadWrite.All – type = Application
      • Group.ReadWrite.All – type = Application
      • User.ReadWrite.All – type = Application
  • All code and pipelines can be found on GitHub

Build Application

Make sure you have the prerequisites ready:

  1. Azure DevOps custom build agent
  2. IAM user with specified permissions. The Access key and secret are necessary for the script.
  3. An Azure AD application to manage the Azure AD AWS SSO Application. The Client ID and client secret are necessary for the script.

Azure AD AWS SSO Application

First create the Azure AD AWS SSO Application. More information can be found here: https://docs.microsoft.com/en-us/azure/active-directory/saas-apps/aws-multi-accounts-tutorial

The Application must be configured with the following:

  • Redirect URIs = https://signin.aws.amazon.com/saml (type = Web) alt text
  • Application ID URI = https://signin.aws.amazon.com/saml#1
  • Custom Scope:
    • Scope name = user_impersonation
    • Who can consent= Admins and users
    • Admin consent display name = Access Amazon Web Services (AWS)
    • Admin consent description = Allow the application to access Amazon Web Services (AWS) on behalf of the signed-in user.
    • User consent display name = Access Amazon Web Services (AWS)
    • User consent description = Access Amazon Web Services (AWS) alt text

The following Single sign-on settings: alt text

  • After the application has been created, you need to download the Federation Metadata XML and add it to the code repository. This XML file is needed and the script uses it to create the IdP provider in every AWS Account. This connects to AWS Account to the Azure AD Application.

Azure DevOps – Build pipeline

First clone the GitHub repo and import into your own Azure DevOps repo. You can import the build definition from the GitHub repository. The Build pipeline step looks like this: alt text Linked to the build pipeline is 1 variable group and has the following values: alt text

  • varAAD = the domain for the email address that will be used when creating a new AWS account. Example: abc.com
  • varAccessKey = the Access Key from the IAM user that is created in the Master Account
  • varClientID = the ClientID from the Azure AD Application that is created to connect to Graph and manage the other Azure AD Application (Azure AD AWS SSO Application)
  • varClientSecret = the ClientID from the Azure AD Application that is created to connect to Graph and manage the other Azure AD Application (Azure AD AWS SSO Application)
  • varPassword = the Office 365 user’s password to connect to Exchange Online and add the new AWS account email as alias to the users mailbox
  • varSecretKey = the Secret Key from the IAM user that is created in the Master Account
  • varServicePrincipal = the ObjectID from the Azure AD AWS SSO Application)
  • varTenantName = The Tenant name from the Azure AD tenant. Example: ebpibv
  • varUserName = the Office 365 user’s username to connect to Exchange Online and add the new AWS account email as alias to the users mailbox.

How to use the solution

After everything is setup, it is easy to use the solution: alt text

  1. Go to the newly created Build Pipeline > Click on Queue
  2. Add variable ‘Account’ with value ‘All’ to run the script to check/update every account. alt text
  3. Click on Queue. The script now is going to run and updates every AWS Account.
  4. Or add variable ‘Account’ with value ‘AccountName‘ without the prefix ‘VCO-AWS-‘, for example:
  • If the AccountName is ‘VCO-AWS-FIT-ACCPRD, the value should be ”FIT-ACCPRD’
  • If the AccountName is ‘VCO-AWS-PORTAL-DEVTST, the value should be ”PORTAL-DEVTST’ alt text
  1. Click on Queue. The script now runs. If the account already exists, the script updates the account. if the account does not exists, it creates the account in AWS. All code and pipelines can be found on GitHub