AKS + Private Link Service + Private Endpoint

This walkthrough shows how to setup a Private Link Service with an AKS cluster and create a Private Endpoint in a separate Vnet.

While many tutorials might give you a full ARM template, this is designed as a walkthrough which completely uses the CLI so you can understand what’s happening at every step of the process.

It focuses on an “uninteresting” workload and uses podinfo as the sample app. This is because it’s easy to deploy and customize with a sample Helm chart.

This is inspired and leans heavily on the Azure Docs for creating a Private Link Service.

Prerequisites

1
- [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/)
2

3
- [jq](https://stedolan.github.io/jq/)

This walkthrough assumes you let Azure create the Vnet when creating the AKS cluster. If you manually created the Vnet, then the general steps are the same, except you must enter the AKS_MC_VNET, AKS_MC_SUBNET env vars manually.

Setup Steps

First, create a sample AKS cluster and install Podinfo on it.

1
# Set these values
2
AKS_NAME=
3
AKS_RG=
4
LOCATION=
5

6
# Create the AKS cluster
7
az aks create -n $AKS_NAME -g $AKS_RG
8

9
# Get the MC Resource Group
10
AKS_MC_RG=$(az aks show -n $AKS_NAME -g $AKS_RG | jq -r '.nodeResourceGroup')
11
echo $AKS_MC_RG
12

13
# Get the Vnet Name
14
AKS_MC_VNET=$(az network vnet list -g $AKS_MC_RG | jq -r '.[0].name')
15
echo $AKS_MC_VNET
16

17
AKS_MC_SUBNET=$(az network vnet subnet list -g $AKS_MC_RG --vnet-name $AKS_MC_VNET | jq -r '.[0].name')
18
echo $AKS_MC_SUBNET
19

20
AKS_MC_LB_INTERNAL=kubernetes-internal
21

22
AKS_MC_LB_INTERNAL_FE_CONFIG=$(az network lb rule list -g $AKS_MC_RG --lb-name=$AKS_MC_LB_INTERNAL | jq -r '.[0].frontendIpConfiguration.id')
23
echo $AKS_MC_LB_INTERNAL_FE_CONFIG
24

25
# Deploy a sample app using an Internal LB
26
helm upgrade --install --wait podinfo-internal-lb \
27
--set-string service.annotations."service\.beta\.kubernetes\.io\/azure-load-balancer-internal"=true \
28
--set service.type=LoadBalancer \
29
--set ui.message=podinfo-internal-lb \
30
podinfo/podinfo

Install Steps - Create the Private Link Service

These steps will be done in the MC_ resource group.

1
# Disable the private link service network policies
2
az network vnet subnet update \
3
--name $AKS_MC_SUBNET \
4
--resource-group $AKS_MC_RG \
5
--vnet-name $AKS_MC_VNET \
6
--disable-private-link-service-network-policies true
7

8
# Create the PLS
9
PLS_NAME=aks-pls
10
az network private-link-service create \
11
--resource-group $AKS_MC_RG \
12
--name $PLS_NAME \
13
--vnet-name $AKS_MC_VNET \
14
--subnet $AKS_MC_SUBNET \
15
--lb-name $AKS_MC_LB_INTERNAL \
16
--lb-frontend-ip-configs $AKS_MC_LB_INTERNAL_FE_CONFIG

Install Steps - Create the Private Endpoint

These steps will be done in our private-endpoint-rg resource group.

1
PE_RG=private-endpoint-rg
2
az group create \
3
--name $PE_RG \
4
--location $LOCATION
5

6
PE_VNET=pe-vnet
7
PE_SUBNET=pe-subnet
8

9
az network vnet create \
10
--resource-group $PE_RG \
11
--name $PE_VNET \
12
--address-prefixes 10.0.0.0/16 \
13
--subnet-name $PE_SUBNET \
14
--subnet-prefixes 10.0.0.0/24
15

16
# Disable the private link service network policies
17
az network vnet subnet update \
18
--name $PE_SUBNET \
19
--resource-group $PE_RG \
20
--vnet-name $PE_VNET \
21
--disable-private-endpoint-network-policies true
22

23
PE_CONN_NAME=pe-conn
24
PE_NAME=pe
25
az network private-endpoint create \
26
--connection-name $PE_CONN_NAME \
27
--name $PE_NAME \
28
--private-connection-resource-id $PLS_ID \
29
--resource-group $PE_RG \
30
--subnet $PE_SUBNET \
31
--manual-request false \
32
--vnet-name $PE_VNET
33

34
# We need the NIC ID to get the newly created Private IP
35
PE_NIC_ID=$(az network private-endpoint show -g $PE_RG --name $PE_NAME -o json | jq -r '.networkInterfaces[0].id')
36
echo $PE_NIC_ID
37

38
# Get the Private IP from the NIC
39
PE_IP=$(az network nic show --ids $PE_NIC_ID -o json | jq -r '.ipConfigurations[0].privateIpAddress')
40
echo $PE_IP

Validation Steps - Create a VM

Lastly, validate that this works by creating a VM in the Vnet with the Private Endpoint.

1
VM_NAME=ubuntu
2
az vm create \
3
--resource-group $PE_RG \
4
--name ubuntu \
5
--image UbuntuLTS \
6
--public-ip-sku Standard \
7
--vnet-name $PE_VNET \
8
--subnet $PE_SUBNET \
9
--admin-username $USER \
10
--ssh-key-values ~/.ssh/id_rsa.pub
11

12
VM_PIP=$(az vm list-ip-addresses -g $PE_RG -n $VM_NAME | jq -r '.[0].virtualMachine.network.publicIpAddresses[0].ipAddress')
13
echo $VM_PIP
14

15
# SSH into the host
16
ssh $VM_IP
17

18
$ curl COPY_THE_VALUE_FROM_PE_IP:9898
19

20
# The output should look like:
21
$ curl 10.0.0.5:9898
22
{
23
"hostname": "podinfo-6ff68cbf88-cxcvv",
24
"version": "6.0.3",
25
"revision": "",
26
"color": "#34577c",
27
"logo": "/images/2022/cuddle_clap.gif",
28
"message": "podinfo-internal-lb",
29
"goos": "linux",
30
"goarch": "amd64",
31
"runtime": "go1.16.9",
32
"num_goroutine": "9",
33
"num_cpu": "2"
34
}

Multiple PLS/PE

To test a specific use case, I wanted to create multiple PLS and PE’s. This set of instructions lets you easily loop through and create multiple instances.

1
# podinfo requires a high numbered port, eg 9000+
2

3
SUFFIX=9000
4
helm upgrade --install --wait podinfo-$SUFFIX \
5
--set-string service.annotations."service\.beta\.kubernetes\.io\/azure-load-balancer-internal"=true \
6
--set service.type=LoadBalancer \
7
--set service.httpPort=$SUFFIX \
8
--set service.externalPort=$SUFFIX \
9
--set ui.message=podinfo-$SUFFIX \
10
podinfo/podinfo
11

12
# This might be easier to hard-code
13
AKS_MC_LB_INTERNAL_FE_CONFIG=$(az network lb rule list -g $AKS_MC_RG --lb-name=$AKS_MC_LB_INTERNAL -o json | jq -r ".[] | select( .backendPort == $SUFFIX) | .frontendIpConfiguration.id")
14
echo $AKS_MC_LB_INTERNAL_FE_CONFIG
15

16
PLS_NAME=aks-pls-$SUFFIX
17
PE_CONN_NAME=pe-conn-$SUFFIX
18
PE_NAME=pe-$SUFFIX
19

20
az network private-link-service create \
21
--resource-group $AKS_MC_RG \
22
--name $PLS_NAME \
23
--vnet-name $AKS_MC_VNET \
24
--subnet $AKS_MC_SUBNET \
25
--lb-name $AKS_MC_LB_INTERNAL \
26
--lb-frontend-ip-configs $AKS_MC_LB_INTERNAL_FE_CONFIG
27

28
PLS_ID=$(az network private-link-service show \
29
--name $PLS_NAME \
30
--resource-group $AKS_MC_RG \
31
--query id \
32
--output tsv)
33
echo $PLS_ID
34

35
az network private-endpoint create \
36
--connection-name $PE_CONN_NAME \
37
--name $PE_NAME \
38
--private-connection-resource-id $PLS_ID \
39
--resource-group $PE_RG \
40
--subnet $PE_SUBNET \
41
--manual-request false \
42
--vnet-name $PE_VNET
43

44
PE_NIC_ID=$(az network private-endpoint show -g $PE_RG --name $PE_NAME -o json | jq -r '.networkInterfaces[0].id')
45
echo $PE_NIC_ID
46

47
PE_IP=$(az network nic show --ids $PE_NIC_ID -o json | jq -r '.ipConfigurations[0].privateIpAddress')
48
echo $PE_IP
49

50
echo "From your Private Endpoint VM run: curl $PE_IP:$SUFFIX"

I created this article to help myself (and hopefully you!) to clearly understand all of the resources and how they interact to create a Private Link Service and Private Endpoint fronting a private service inside an AKS cluster. This has been highly enlightening for me and I hope it has for you too.