{"slug":"multi-cluster-ingress","title":"Multi-cluster Ingress","tags":["tailscale","containers"],"agent_summary":"Last validated: Jan 5, 2026","trigger_phrases":[],"runnable":false,"markdown":"\r\n# Multi-cluster Ingress\r\n\r\nLast validated: Jan 5, 2026\r\n\r\nKubernetes Operator multi-cluster Ingressis currently [in alpha](https://tailscale.com/docs/reference/tailscale-release-stages#alpha). To try it, follow the steps below to enable it for your network using Tailscale v1.84 or later.\r\n\r\nThe following example shows the process for exposing an application, deployed across two different clusters, to your tailnet. A single MagicDNS name routes each Tailscale client to their closest cluster.\r\n\r\n- A [`ProxyGroup`](https://tailscale.com/docs/features/kubernetes-operator#optional-pre-creating-a-proxygroup) in each cluster will manage a set of highly available proxies.\r\n- A Tailscale `Ingress` in each cluster will configure proxies to route tailnet traffic to the in-cluster app instance.\r\n- A single Tailscale Service ensures that tailnet traffic for the app is routed to a proxy in the closest cluster.\r\n\r\nThis tutorial covers exposing multi-cluster applications using Tailscale's application layer ingress via `Ingress` resource.\r\nYou can use the same approach to expose multi-cluster applications using Tailscale's network layer ingress via `Service` resource.\r\n\r\n## [Setup](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#setup)\r\n\r\n1. Ensure you have two clusters available, preferably in different geographical regions, so you can test clients in different regions being routed to the closest cluster.\r\n\r\n2. Update [`tagOwners`](https://tailscale.com/docs/reference/syntax/policy-file#tag-owners) in your tailnet policy file to let the Kubernetes Operator to create `ProxyGroup` proxies with the tag `tag:ingress-proxies` and Tailscale Services with the tag `tag:internal-apps`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```json\r\n     \"tagOwners\": {\r\n       \"tag:k8s-operator\": [],\r\n       \"tag:internal-apps\":          [\"tag:k8s-operator\"],\r\n       \"tag:ingress-proxies\":        [\"tag:k8s-operator\"],\r\n       ...\r\n     },\r\n```\r\n\r\n\r\n\r\n\r\n\r\nYou can use the [visual policy editor](https://tailscale.com/docs/features/visual-editor) to manage your tailnet policy file. Refer to the [visual editor reference](https://tailscale.com/docs/reference/visual-editor) for guidance on using the visual editor.\r\n\r\n3. Update [`autoApprovers`](https://tailscale.com/docs/reference/syntax/policy-file#autoapprovers) in your tailnet policy file to let the `ProxyGroup` proxies to advertise Tailscale Services with the tag `tag:internal-apps`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```json\r\n     \"autoApprovers\": {\r\n       \"services\": {\r\n         \"tag:ingress-proxies\": [\"tag:internal-apps\"],\r\n       },\r\n     }\r\n```\r\n\r\n\r\n\r\n\r\n\r\nYou can use the [visual policy editor](https://tailscale.com/docs/features/visual-editor) to manage your tailnet policy file. Refer to the [visual editor reference](https://tailscale.com/docs/reference/visual-editor) for guidance on using the visual editor.\r\n\r\n4. Update [grants](https://tailscale.com/docs/reference/syntax/policy-file#grants) to allow tailnet clients access to the cluster apps you want to expose.\r\nThere will be a Tailscale Service created for each app. The Service's ACL tag can be used to grant permissions to access the app.\r\nFor example, to grant the Tailscale user group `group:engineering` access to apps tagged with `tag:internal-apps`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```json\r\n     \"grants\": [\\\r\n       {\\\r\n         \"src\":    [\"group:engineering\"],\\\r\n         \"dst\":    [\"tag:internal-apps:80\",\"tag:internal-apps:443\"],\\\r\n         \"ip\": [\"tcp:80\",\"tcp:443\"],\\\r\n       },\\\r\n       {\\\r\n         \"src\": [\"group:engineering\"],\\\r\n         \"ip\": [\"icmp:22\"],\\\r\n         \"dst\": [\"tag:ingress-proxies\"],\\\r\n       },\\\r\n       ...\\\r\n     ]\r\n```\r\n\r\n\r\n\r\n\r\n\r\nYou can use the [visual policy editor](https://tailscale.com/docs/features/visual-editor) to manage your tailnet policy file. Refer to the [visual editor reference](https://tailscale.com/docs/reference/visual-editor) for guidance on using the visual editor.\r\n\r\n\r\n\r\n\r\n\r\nThe requirement to grant access to the devices to access a Tailscale Service is a temporary limitation.\r\n\r\n5. Ensure your [client accepts routes](https://tailscale.com/docs/features/subnet-routers#use-your-subnet-routes-from-other-devices). Clients other than Linux accept routes by default.\r\n\r\n6. Ensure your tailnet has [regional routing](https://tailscale.com/docs/how-to/set-up-high-availability#regional-routing) enabled.\r\n\r\n\r\n## [In each cluster](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#in-each-cluster)\r\n\r\n01. [Create OAuth client credentials](https://tailscale.com/docs/features/kubernetes-operator#setting-up-the-kubernetes-operator) for the Kubernetes Operator with write scopes for `Services`, `Devices Core`, and `Auth Keys`.\r\n\r\n02. Add `https://pkgs.tailscale.com/helmcharts` to your local Helm repositories:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```shell\r\n    helm repo add tailscale https://pkgs.tailscale.com/helmcharts\r\n    ```\r\n\r\n03. Update your local Helm cache:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```shell\r\n    helm repo update\r\n    ```\r\n\r\n04. Install the operator:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```shell\r\n    helm upgrade --install operator tailscale/tailscale-operator \\\r\n        -n tailscale --create-namespace \\\r\n        --set oauth.clientId=<id> \\\r\n        --set oauth.clientSecret=<key> \\\r\n        --wait\r\n    ```\r\n\r\n05. Apply the following `ProxyGroup` and [`ProxyClass`](https://tailscale.com/docs/features/kubernetes-operator/how-to/customize#cluster-resource-customization-using-proxyclass) resources to create a set of Tailscale devices that will act as proxies:\r\n\r\n\r\n\r\n    While initially deploying `Ingress` resources, we highly recommend you use Let's Encrypt's [staging environment](https://letsencrypt.org/docs/staging-environment/) to avoid production's tighter rate limits.\r\n    The following example uses Let's Encrypt's staging environment, but you can unset `useLetsEncryptStagingEnvironment` once you are ready to provision production HTTPS certificates.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```yaml\r\n    apiVersion: tailscale.com/v1alpha1\r\n    kind: ProxyGroup\r\n    metadata:\r\n      name: ingress-proxies\r\n    spec:\r\n      type: ingress\r\n      hostnamePrefix: eu-west\r\n      replicas: 2\r\n      tags: [\"tag:ingress-proxies\"]\r\n      proxyClass: letsencrypt-staging\r\n    ---\r\n    apiVersion: tailscale.com/v1alpha1\r\n    kind: ProxyClass\r\n    metadata:\r\n      name: letsencrypt-staging\r\n    spec:\r\n      useLetsEncryptStagingEnvironment: true\r\n    ```\r\n\r\n06. (Optional) Wait for the ProxyGroup to become ready:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```shell\r\n    kubectl wait proxygroup ingress-proxies --for=condition=ProxyGroupReady=true\r\n    ```\r\n\r\n\r\n\r\n    For the above `ProxyGroup` the operator creates a `StatefulSet` with two replicas.\r\n    Each replica `Pod` runs a Tailscale device with a [tag](https://tailscale.com/docs/features/tags)`tag:ingress-proxies` and hostname with prefix `eu-west-`\r\n\r\n07. (Optional) if you don't have an existing workload to route traffic to, deploy `nginx` as a sample application:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```yaml\r\n    apiVersion: v1\r\n    kind: Pod\r\n    metadata:\r\n      labels:\r\n        run: nginx\r\n      name: nginx\r\n    spec:\r\n      containers:\r\n  - name: nginx\r\n    image: nginx\r\n---\r\napiVersion: v1\r\nkind: Service\r\nmetadata:\r\nlabels:\r\n    run: nginx\r\nname: nginx\r\nspec:\r\nports:\r\n  - port: 80\r\n    protocol: TCP\r\n    targetPort: 80\r\nselector:\r\n    run: nginx\r\n```\r\n\r\n08. Apply the following `Ingress` resource to expose nginx with the DNS name `nginx.<your-tailnet-domain>`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```yaml\r\n    apiVersion: networking.k8s.io/v1\r\n    kind: Ingress\r\n    metadata:\r\n      name: nginx\r\n      annotations:\r\n        tailscale.com/proxy-group: ingress-proxies\r\n        tailscale.com/tags: \"tag:internal-apps\"\r\n    spec:\r\n      tls:\r\n  - hosts:\r\n    - nginx\r\n  rules:\r\n  - http:\r\n      paths:\r\n      - backend:\r\n          service:\r\n            name: nginx\r\n            port:\r\n              number: 80\r\n        pathType: Prefix\r\n        path: /\r\n  ingressClassName: tailscale\r\n```\r\n\r\n09. (Optional) Wait for the HTTPS endpoints to be ready:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```shell\r\n    kubectl wait --timeout=80s  ingress nginx --for=jsonpath='{.status.loadBalancer.ingress[0].ports[0].port}'=443\r\n    ```\r\n\r\n\r\n\r\n    For the above `Ingress` resource, the Kubernetes Operator ensures that a Tailscale Service named `svc:nginx` exists for the tailnet and that proxies route tailnet traffic for the Tailscale Service to the Kubernetes `Service``nginx`.\r\n    The Tailscale Service's name is determined from `ingress.spec.tls[0].hosts[0]` field.\r\n    The Tailscale Service will be created if it does not already exist. `Ingress` resources in multiple clusters can define backends for a single Tailscale Service by using the same `ingress.spec.tls[0].hosts[0]` field.\r\n\r\n10. Repeat the steps from this section for your second cluster (optionally changing `proxyGroup.spec.hostnamePrefix` field value).\r\n\r\n\r\nYou can expose any number of `Ingress` resources on the same `ProxyGroup` (limited by resource consumption).\r\nYou can not create multiple `Ingress` resources for the same Tailscale Service in the same cluster.\r\n\r\n## [Test](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#test)\r\n\r\n1. Check the MagicDNS name for the created Ingress:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\nkubectl get ingress\r\nNAME          CLASS       HOSTS   ADDRESS                 PORTS   AGE\r\nnginx         tailscale   *       nginx.tailxyz.ts.net    443     3h18m\r\n```\r\n\r\n2. Test that traffic flows for the MagicDNS name:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\ncurl -ksS https://nginx.tailxyz.ts.net\r\n<!DOCTYPE html>\r\n<html>\r\n<head>\r\n<title>Welcome to nginx!</title>\r\n...\r\n```\r\n\r\n\r\n## [(Optional) Cluster-specific DNS names](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#optional-cluster-specific-dns-names)\r\n\r\nWhen using a single DNS name for `Ingress` resources deployed across multiple clusters, clients will automatically route to their closest cluster.\r\n\r\nIf you want to also create cluster-specific DNS names, you can deploy additional `Ingress` resources that are specific to each cluster.\r\n\r\nApply a cluster-specific `Ingress` such as:\r\n\r\n```yaml\r\napiVersion: networking.k8s.io/v1\r\nkind: Ingress\r\nmetadata:\r\n  name: nginx-eu\r\n  annotations:\r\n    tailscale.com/proxy-group: ingress-proxies\r\n    tailscale.com/tags: \"tag:internal-apps\"\r\nspec:\r\n  tls:\r\n  - hosts:\r\n    - nginx-eu\r\n  rules:\r\n  - http:\r\n      paths:\r\n      - backend:\r\n          service:\r\n            name: nginx\r\n            port:\r\n              number: 80\r\n        pathType: Prefix\r\n        path: /\r\n  ingressClassName: tailscale\r\n```\r\n\r\n## [TLS certificates](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#tls-certificates)\r\n\r\nThe cluster app behind the `Ingress` is exposed to tailnet over HTTPS with a MagicDNS name constructed using the value of `ingress.spec.tls[0].hosts[0]` field and the tailnet domain.\r\nA single TLS certificate is issued for the `Ingress` and shared amongst the `ProxyGroup` replicas.\r\nTLS certificates are issued from Let's Encrypt's production environment by default.\r\n\r\nLet's Encrypt imposes [rate limits](https://letsencrypt.org/docs/rate-limits/) to certificate issuance.\r\n\r\nTailscale's `ts.net` domain is in [public suffix list](https://publicsuffix.org/).\r\nThis means that Let's Encrypt considers a tailnet's top level domain (for example, `tailxyz.ts.net`) a registered domain and rate limits are applied to each tailnet individually.\r\n\r\nFor example:\r\n\r\n- [50 certificates per week per domain](https://letsencrypt.org/docs/rate-limits/#certificates-per-registered-domain). This means no more than 50 unique Ingresses per week per tailnet.\r\nNote that any other certs issued for tailnet devices also count toward this limit.\r\n\r\n- [5 certificates for the same hostname per week](https://letsencrypt.org/docs/rate-limits/#duplicate-certificate-limit).\r\nThis means that the number of clusters that can expose the same app with the same DNS name is limited to 5 per week.\r\n\r\n\r\nWe highly recommend testing using Let's Encrypt's staging environment to avoid tighter rate limits until you are ready to deploy to production.\r\n\r\n### [Using Let's Encrypt's staging environment](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#using-lets-encrypts-staging-environment)\r\n\r\nYou can use the staging environment with a `ProxyGroup` and `ProxyClass` such as the following:\r\n\r\n```yaml\r\napiVersion: tailscale.com/v1alpha1\r\nkind: ProxyGroup\r\nmetadata:\r\n  name: ingress-proxies\r\nspec:\r\n  type: ingress\r\n  replicas: 2\r\n  tags: [\"tag:ingress-proxies\"]\r\n  proxyClass: letsencrypt-staging\r\n---\r\napiVersion: tailscale.com/v1alpha1\r\nkind: ProxyClass\r\nmetadata:\r\n  name: letsencrypt-staging\r\nspec:\r\n  useLetsEncryptStagingEnvironment: true\r\n```\r\n\r\nFor the above configuration, the operator will create a `ProxyGroup` that always uses Let's Encrypt's staging endpoint to issue certificates for any Tailscale `Ingress` DNS names.\r\n\r\n### [Advertising an HTTP endpoint](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#advertising-an-http-endpoint)\r\n\r\nYou can optionally enable an HTTP endpoint on port 80 in addition to the HTTPS endpoint on port 443.\r\nThis may be helpful if you want the `Ingress` to still be available when an HTTPS certificate cannot be issued due to rate limits or other failure cases.\r\n\r\nTo enable an HTTP endpoint, add a `tailscale.com/http-endpoint: enabled` annotation to your `Ingress`.\r\n\r\nNote that:\r\n\r\n- If an `Ingress` does not have an HTTP endpoint enabled, the proxies that advertise this `Ingress` will only be considered healthy once the certificate issuance has succeeded.\r\nFor example, if you have `Ingress` resources that expose app `nginx` with DNS name `nginx.tailxyx.ts.net` in clusters `us-east`, `us-west` and `eu-west` and certificate issuance only\r\nsucceeds in `us-west` and `eu-west`, tailnet traffic for `nginx.tailxyz.ts.net` will never be routed to `us-east`.\r\nThe proxies in `us-east` will retry issuance and once it succeeds, they will be considered healthy and tailnet traffic will start getting routed to `us-east`.\r\n\r\n- If an `Ingress` does have an HTTP endpoint enabled, the proxies that advertise this `Ingress` will be considered healthy, even if certificate issuance has failed.\r\nSo, if you have `Ingress` resources that expose app `nginx` in clusters in `us-east`, `us-west` and `eu-west`, and certificate issuance only\r\nsucceeds in `us-west` and `eu-west`, tailnet traffic will still be routed to all three clusters, but only the HTTP endpoint is guaranteed to be healthy.\r\nThe proxies in `us-east` will retry issuance and once it succeeds, all clients will be able to reach the app by using HTTPS in any cluster.\r\n\r\n\r\n## [Debugging](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#debugging)\r\n\r\n### [Debugging `Ingress` resource failure](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#debugging-ingress-resource-failure)\r\n\r\n1. Take a look at [operator's logs](https://tailscale.com/docs/reference/troubleshooting/containers/kubernetes-operator#operator-logs).\r\n\r\n2. Take a look at the proxies:\r\nFor a `ProxyGroup` named `ingress-proxies`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\nkubectl get pod -n tailscale -l tailscale.com/parent-resource=\"ingress-proxies\",tailscale.com/parent-resource-type=\"proxygroup\"\r\nNAME                READY   STATUS    RESTARTS   AGE\r\ningress-proxies-0   1/1     Running   0          2d22h\r\ningress-proxies-1   1/1     Running   0          2d22h\r\n```\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\nkubectl logs ingress-proxies-0 -n tailscale -c tailscale\r\nboot: 2025/03/28 14:20:43 Starting tailscaled\r\n...\r\n```\r\n\r\n\r\n### [Validate to which cluster a tailnet client's requests are routed](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#validate-to-which-cluster-a-tailnet-clients-requests-are-routed)\r\n\r\n1. Find the MagicDNS name by which Ingress is exposed to tailnet.\r\nThis will be the DNS name of the corresponding Tailscale Service.\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\nkubectl get ingress\r\nNAME          CLASS       HOSTS   ADDRESS                 PORTS   AGE\r\nnginx         tailscale   *       nginx.tailxyz.ts.net    443     3h18m\r\n```\r\n\r\n2. Find the tailnet IP address of the Tailscale Service:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\ndig nginx.tailxyz.ts.net +short\r\n100.100.126.127\r\n```\r\n\r\n\r\n\r\nIf the Tailscale Service's IP address can not be resolved, its creation might have failed. Check the operator's logs.\r\n\r\n3. Find which proxy the tailnet client uses as a backend for the Tailscale Service.\r\nEach tailnet client's requests for a Tailscale Service will be routed to a proxy in the closest cluster.\r\nYou can run the following commands on a client to find which proxy it uses as a backend for a Tailscale Service:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n```shell\r\ntailscale status --json | jq '.Peer | to_entries[] | select(.value.AllowedIPs[] | contains(\"100.100.126.127/32\")) | .value |\r\n{DNSName, ID}'\r\n{\r\n     \"DNSName\": \"ingress-proxies-eu-0.tailxyz.ts.net.\",\r\n     \"ID\": \"n9Ch5VvNug11CNTRL\"\r\n}\r\n```\r\n\r\n\r\n\r\nThe first part of DNS name is either `proxyGroup.spec.hostnamePrefix` or the `ProxyGroup` name - you can use that to identify which cluster the tailnet client's traffic will be routed to.\r\n\r\nIf `tailscale status --json` does not contain any results for the given tailnet IP address then the client likely does not have permissions to access the Tailscale Service.\r\n\r\n\r\n## [Best practices](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#best-practices)\r\n\r\n- Use Let's Encrypt's staging environment for initial testing.\r\n\r\n- Set a different `proxygroup.spec.hostnamePrefix` field values for `ProxyGroup` resources in different clusters (or simply name the `ProxyGroup` resources differently).\r\nThis will ensure that proxies in different clusters can be easily identified by the Tailscale hostname.\r\n\r\n\r\n## [Current limitations](https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress\\#current-limitations)\r\n\r\n- Exposing a new `Ingress` resource takes up to a minute to become available. We are working to make this faster.\r\n\r\n- The current access model requires explicitly defining access for each app in ACLs.\r\nWe are working on allowing tag based access.\r\n\r\n- When exposing `Ingress` resources with the same DNS name concurrently, you may hit transient failures due to DNS challenge errors.\r\nWe are working on fixing this.\r\n\r\n- Currently Let's Encrypt's staging endpoint can only be enabled for the `ProxyGroup` as a whole and not for individual `Ingress` resources.\r\n\r\n- Currently you must ensure that multi-cluster `Ingress` resources for the same app all have the same tag and all either expose an HTTP endpoint or not.\r\nApplying `Ingress` resources for the same app, but with different tags or different HTTP endpoint settings, might result in the operator instances in the different clusters continuously attempting to reconcile the Tailscale Service.\r\n\r\n\r\n![Project Logo](https://cdn.brandfetch.io/tailscale.com/fallback/lettermark/theme/dark/h/256/w/256/icon?c=1bfwsmEH20zzEfSNTed)\r\n\r\nAsk AI\r\n\r\nreCAPTCHA\r\n\r\nRecaptcha requires verification.\r\n\r\nprotected by **reCAPTCHA**\r\n","html":"<h1>Multi-cluster Ingress</h1>\n<p>Last validated: Jan 5, 2026</p>\n<p>Kubernetes Operator multi-cluster Ingressis currently <a href=\"https://tailscale.com/docs/reference/tailscale-release-stages#alpha\">in alpha</a>. To try it, follow the steps below to enable it for your network using Tailscale v1.84 or later.</p>\n<p>The following example shows the process for exposing an application, deployed across two different clusters, to your tailnet. A single MagicDNS name routes each Tailscale client to their closest cluster.</p>\n<ul>\n<li>A <a href=\"https://tailscale.com/docs/features/kubernetes-operator#optional-pre-creating-a-proxygroup\"><code>ProxyGroup</code></a> in each cluster will manage a set of highly available proxies.</li>\n<li>A Tailscale <code>Ingress</code> in each cluster will configure proxies to route tailnet traffic to the in-cluster app instance.</li>\n<li>A single Tailscale Service ensures that tailnet traffic for the app is routed to a proxy in the closest cluster.</li>\n</ul>\n<p>This tutorial covers exposing multi-cluster applications using Tailscale's application layer ingress via <code>Ingress</code> resource.\r\nYou can use the same approach to expose multi-cluster applications using Tailscale's network layer ingress via <code>Service</code> resource.</p>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#setup\">Setup</a></h2>\n<ol>\n<li>\n<p>Ensure you have two clusters available, preferably in different geographical regions, so you can test clients in different regions being routed to the closest cluster.</p>\n</li>\n<li>\n<p>Update <a href=\"https://tailscale.com/docs/reference/syntax/policy-file#tag-owners\"><code>tagOwners</code></a> in your tailnet policy file to let the Kubernetes Operator to create <code>ProxyGroup</code> proxies with the tag <code>tag:ingress-proxies</code> and Tailscale Services with the tag <code>tag:internal-apps</code>:</p>\n</li>\n</ol>\n<pre><code class=\"language-json\">     \"tagOwners\": {\r\n       \"tag:k8s-operator\": [],\r\n       \"tag:internal-apps\":          [\"tag:k8s-operator\"],\r\n       \"tag:ingress-proxies\":        [\"tag:k8s-operator\"],\r\n       ...\r\n     },\n</code></pre>\n<p>You can use the <a href=\"https://tailscale.com/docs/features/visual-editor\">visual policy editor</a> to manage your tailnet policy file. Refer to the <a href=\"https://tailscale.com/docs/reference/visual-editor\">visual editor reference</a> for guidance on using the visual editor.</p>\n<ol start=\"3\">\n<li>Update <a href=\"https://tailscale.com/docs/reference/syntax/policy-file#autoapprovers\"><code>autoApprovers</code></a> in your tailnet policy file to let the <code>ProxyGroup</code> proxies to advertise Tailscale Services with the tag <code>tag:internal-apps</code>:</li>\n</ol>\n<pre><code class=\"language-json\">     \"autoApprovers\": {\r\n       \"services\": {\r\n         \"tag:ingress-proxies\": [\"tag:internal-apps\"],\r\n       },\r\n     }\n</code></pre>\n<p>You can use the <a href=\"https://tailscale.com/docs/features/visual-editor\">visual policy editor</a> to manage your tailnet policy file. Refer to the <a href=\"https://tailscale.com/docs/reference/visual-editor\">visual editor reference</a> for guidance on using the visual editor.</p>\n<ol start=\"4\">\n<li>Update <a href=\"https://tailscale.com/docs/reference/syntax/policy-file#grants\">grants</a> to allow tailnet clients access to the cluster apps you want to expose.\r\nThere will be a Tailscale Service created for each app. The Service's ACL tag can be used to grant permissions to access the app.\r\nFor example, to grant the Tailscale user group <code>group:engineering</code> access to apps tagged with <code>tag:internal-apps</code>:</li>\n</ol>\n<pre><code class=\"language-json\">     \"grants\": [\\\r\n       {\\\r\n         \"src\":    [\"group:engineering\"],\\\r\n         \"dst\":    [\"tag:internal-apps:80\",\"tag:internal-apps:443\"],\\\r\n         \"ip\": [\"tcp:80\",\"tcp:443\"],\\\r\n       },\\\r\n       {\\\r\n         \"src\": [\"group:engineering\"],\\\r\n         \"ip\": [\"icmp:22\"],\\\r\n         \"dst\": [\"tag:ingress-proxies\"],\\\r\n       },\\\r\n       ...\\\r\n     ]\n</code></pre>\n<p>You can use the <a href=\"https://tailscale.com/docs/features/visual-editor\">visual policy editor</a> to manage your tailnet policy file. Refer to the <a href=\"https://tailscale.com/docs/reference/visual-editor\">visual editor reference</a> for guidance on using the visual editor.</p>\n<p>The requirement to grant access to the devices to access a Tailscale Service is a temporary limitation.</p>\n<ol start=\"5\">\n<li>\n<p>Ensure your <a href=\"https://tailscale.com/docs/features/subnet-routers#use-your-subnet-routes-from-other-devices\">client accepts routes</a>. Clients other than Linux accept routes by default.</p>\n</li>\n<li>\n<p>Ensure your tailnet has <a href=\"https://tailscale.com/docs/how-to/set-up-high-availability#regional-routing\">regional routing</a> enabled.</p>\n</li>\n</ol>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#in-each-cluster\">In each cluster</a></h2>\n<ol>\n<li>\n<p><a href=\"https://tailscale.com/docs/features/kubernetes-operator#setting-up-the-kubernetes-operator\">Create OAuth client credentials</a> for the Kubernetes Operator with write scopes for <code>Services</code>, <code>Devices Core</code>, and <code>Auth Keys</code>.</p>\n</li>\n<li>\n<p>Add <code>https://pkgs.tailscale.com/helmcharts</code> to your local Helm repositories:</p>\n<pre><code class=\"language-shell\">helm repo add tailscale https://pkgs.tailscale.com/helmcharts\n</code></pre>\n</li>\n<li>\n<p>Update your local Helm cache:</p>\n<pre><code class=\"language-shell\">helm repo update\n</code></pre>\n</li>\n<li>\n<p>Install the operator:</p>\n<pre><code class=\"language-shell\">helm upgrade --install operator tailscale/tailscale-operator \\\r\n    -n tailscale --create-namespace \\\r\n    --set oauth.clientId=&#x3C;id> \\\r\n    --set oauth.clientSecret=&#x3C;key> \\\r\n    --wait\n</code></pre>\n</li>\n<li>\n<p>Apply the following <code>ProxyGroup</code> and <a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/customize#cluster-resource-customization-using-proxyclass\"><code>ProxyClass</code></a> resources to create a set of Tailscale devices that will act as proxies:</p>\n<p>While initially deploying <code>Ingress</code> resources, we highly recommend you use Let's Encrypt's <a href=\"https://letsencrypt.org/docs/staging-environment/\">staging environment</a> to avoid production's tighter rate limits.\r\nThe following example uses Let's Encrypt's staging environment, but you can unset <code>useLetsEncryptStagingEnvironment</code> once you are ready to provision production HTTPS certificates.</p>\n<pre><code class=\"language-yaml\">apiVersion: tailscale.com/v1alpha1\r\nkind: ProxyGroup\r\nmetadata:\r\n  name: ingress-proxies\r\nspec:\r\n  type: ingress\r\n  hostnamePrefix: eu-west\r\n  replicas: 2\r\n  tags: [\"tag:ingress-proxies\"]\r\n  proxyClass: letsencrypt-staging\r\n---\r\napiVersion: tailscale.com/v1alpha1\r\nkind: ProxyClass\r\nmetadata:\r\n  name: letsencrypt-staging\r\nspec:\r\n  useLetsEncryptStagingEnvironment: true\n</code></pre>\n</li>\n<li>\n<p>(Optional) Wait for the ProxyGroup to become ready:</p>\n<pre><code class=\"language-shell\">kubectl wait proxygroup ingress-proxies --for=condition=ProxyGroupReady=true\n</code></pre>\n<p>For the above <code>ProxyGroup</code> the operator creates a <code>StatefulSet</code> with two replicas.\r\nEach replica <code>Pod</code> runs a Tailscale device with a <a href=\"https://tailscale.com/docs/features/tags\">tag</a><code>tag:ingress-proxies</code> and hostname with prefix <code>eu-west-</code></p>\n</li>\n<li>\n<p>(Optional) if you don't have an existing workload to route traffic to, deploy <code>nginx</code> as a sample application:</p>\n<pre><code class=\"language-yaml\">apiVersion: v1\r\nkind: Pod\r\nmetadata:\r\n  labels:\r\n    run: nginx\r\n  name: nginx\r\nspec:\r\n  containers:\n</code></pre>\n</li>\n</ol>\n<ul>\n<li>name: nginx\r\nimage: nginx</li>\n</ul>\n<hr>\n<p>apiVersion: v1\r\nkind: Service\r\nmetadata:\r\nlabels:\r\nrun: nginx\r\nname: nginx\r\nspec:\r\nports:</p>\n<ul>\n<li>port: 80\r\nprotocol: TCP\r\ntargetPort: 80\r\nselector:\r\nrun: nginx</li>\n</ul>\n<pre><code>\r\n08. Apply the following `Ingress` resource to expose nginx with the DNS name `nginx.&#x3C;your-tailnet-domain>`:\r\n\r\n\r\n\r\n\r\n\r\n\r\n\r\n    ```yaml\r\n    apiVersion: networking.k8s.io/v1\r\n    kind: Ingress\r\n    metadata:\r\n      name: nginx\r\n      annotations:\r\n        tailscale.com/proxy-group: ingress-proxies\r\n        tailscale.com/tags: \"tag:internal-apps\"\r\n    spec:\r\n      tls:\r\n  - hosts:\r\n    - nginx\r\n  rules:\r\n  - http:\r\n      paths:\r\n      - backend:\r\n          service:\r\n            name: nginx\r\n            port:\r\n              number: 80\r\n        pathType: Prefix\r\n        path: /\r\n  ingressClassName: tailscale\n</code></pre>\n<ol start=\"9\">\n<li>\n<p>(Optional) Wait for the HTTPS endpoints to be ready:</p>\n<pre><code class=\"language-shell\">kubectl wait --timeout=80s  ingress nginx --for=jsonpath='{.status.loadBalancer.ingress[0].ports[0].port}'=443\n</code></pre>\n<p>For the above <code>Ingress</code> resource, the Kubernetes Operator ensures that a Tailscale Service named <code>svc:nginx</code> exists for the tailnet and that proxies route tailnet traffic for the Tailscale Service to the Kubernetes <code>Service``nginx</code>.\r\nThe Tailscale Service's name is determined from <code>ingress.spec.tls[0].hosts[0]</code> field.\r\nThe Tailscale Service will be created if it does not already exist. <code>Ingress</code> resources in multiple clusters can define backends for a single Tailscale Service by using the same <code>ingress.spec.tls[0].hosts[0]</code> field.</p>\n</li>\n<li>\n<p>Repeat the steps from this section for your second cluster (optionally changing <code>proxyGroup.spec.hostnamePrefix</code> field value).</p>\n</li>\n</ol>\n<p>You can expose any number of <code>Ingress</code> resources on the same <code>ProxyGroup</code> (limited by resource consumption).\r\nYou can not create multiple <code>Ingress</code> resources for the same Tailscale Service in the same cluster.</p>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#test\">Test</a></h2>\n<ol>\n<li>Check the MagicDNS name for the created Ingress:</li>\n</ol>\n<pre><code class=\"language-shell\">kubectl get ingress\r\nNAME          CLASS       HOSTS   ADDRESS                 PORTS   AGE\r\nnginx         tailscale   *       nginx.tailxyz.ts.net    443     3h18m\n</code></pre>\n<ol start=\"2\">\n<li>Test that traffic flows for the MagicDNS name:</li>\n</ol>\n<pre><code class=\"language-shell\">curl -ksS https://nginx.tailxyz.ts.net\r\n&#x3C;!DOCTYPE html>\r\n&#x3C;html>\r\n&#x3C;head>\r\n&#x3C;title>Welcome to nginx!&#x3C;/title>\r\n...\n</code></pre>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#optional-cluster-specific-dns-names\">(Optional) Cluster-specific DNS names</a></h2>\n<p>When using a single DNS name for <code>Ingress</code> resources deployed across multiple clusters, clients will automatically route to their closest cluster.</p>\n<p>If you want to also create cluster-specific DNS names, you can deploy additional <code>Ingress</code> resources that are specific to each cluster.</p>\n<p>Apply a cluster-specific <code>Ingress</code> such as:</p>\n<pre><code class=\"language-yaml\">apiVersion: networking.k8s.io/v1\r\nkind: Ingress\r\nmetadata:\r\n  name: nginx-eu\r\n  annotations:\r\n    tailscale.com/proxy-group: ingress-proxies\r\n    tailscale.com/tags: \"tag:internal-apps\"\r\nspec:\r\n  tls:\r\n  - hosts:\r\n    - nginx-eu\r\n  rules:\r\n  - http:\r\n      paths:\r\n      - backend:\r\n          service:\r\n            name: nginx\r\n            port:\r\n              number: 80\r\n        pathType: Prefix\r\n        path: /\r\n  ingressClassName: tailscale\n</code></pre>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#tls-certificates\">TLS certificates</a></h2>\n<p>The cluster app behind the <code>Ingress</code> is exposed to tailnet over HTTPS with a MagicDNS name constructed using the value of <code>ingress.spec.tls[0].hosts[0]</code> field and the tailnet domain.\r\nA single TLS certificate is issued for the <code>Ingress</code> and shared amongst the <code>ProxyGroup</code> replicas.\r\nTLS certificates are issued from Let's Encrypt's production environment by default.</p>\n<p>Let's Encrypt imposes <a href=\"https://letsencrypt.org/docs/rate-limits/\">rate limits</a> to certificate issuance.</p>\n<p>Tailscale's <code>ts.net</code> domain is in <a href=\"https://publicsuffix.org/\">public suffix list</a>.\r\nThis means that Let's Encrypt considers a tailnet's top level domain (for example, <code>tailxyz.ts.net</code>) a registered domain and rate limits are applied to each tailnet individually.</p>\n<p>For example:</p>\n<ul>\n<li>\n<p><a href=\"https://letsencrypt.org/docs/rate-limits/#certificates-per-registered-domain\">50 certificates per week per domain</a>. This means no more than 50 unique Ingresses per week per tailnet.\r\nNote that any other certs issued for tailnet devices also count toward this limit.</p>\n</li>\n<li>\n<p><a href=\"https://letsencrypt.org/docs/rate-limits/#duplicate-certificate-limit\">5 certificates for the same hostname per week</a>.\r\nThis means that the number of clusters that can expose the same app with the same DNS name is limited to 5 per week.</p>\n</li>\n</ul>\n<p>We highly recommend testing using Let's Encrypt's staging environment to avoid tighter rate limits until you are ready to deploy to production.</p>\n<h3><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#using-lets-encrypts-staging-environment\">Using Let's Encrypt's staging environment</a></h3>\n<p>You can use the staging environment with a <code>ProxyGroup</code> and <code>ProxyClass</code> such as the following:</p>\n<pre><code class=\"language-yaml\">apiVersion: tailscale.com/v1alpha1\r\nkind: ProxyGroup\r\nmetadata:\r\n  name: ingress-proxies\r\nspec:\r\n  type: ingress\r\n  replicas: 2\r\n  tags: [\"tag:ingress-proxies\"]\r\n  proxyClass: letsencrypt-staging\r\n---\r\napiVersion: tailscale.com/v1alpha1\r\nkind: ProxyClass\r\nmetadata:\r\n  name: letsencrypt-staging\r\nspec:\r\n  useLetsEncryptStagingEnvironment: true\n</code></pre>\n<p>For the above configuration, the operator will create a <code>ProxyGroup</code> that always uses Let's Encrypt's staging endpoint to issue certificates for any Tailscale <code>Ingress</code> DNS names.</p>\n<h3><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#advertising-an-http-endpoint\">Advertising an HTTP endpoint</a></h3>\n<p>You can optionally enable an HTTP endpoint on port 80 in addition to the HTTPS endpoint on port 443.\r\nThis may be helpful if you want the <code>Ingress</code> to still be available when an HTTPS certificate cannot be issued due to rate limits or other failure cases.</p>\n<p>To enable an HTTP endpoint, add a <code>tailscale.com/http-endpoint: enabled</code> annotation to your <code>Ingress</code>.</p>\n<p>Note that:</p>\n<ul>\n<li>\n<p>If an <code>Ingress</code> does not have an HTTP endpoint enabled, the proxies that advertise this <code>Ingress</code> will only be considered healthy once the certificate issuance has succeeded.\r\nFor example, if you have <code>Ingress</code> resources that expose app <code>nginx</code> with DNS name <code>nginx.tailxyx.ts.net</code> in clusters <code>us-east</code>, <code>us-west</code> and <code>eu-west</code> and certificate issuance only\r\nsucceeds in <code>us-west</code> and <code>eu-west</code>, tailnet traffic for <code>nginx.tailxyz.ts.net</code> will never be routed to <code>us-east</code>.\r\nThe proxies in <code>us-east</code> will retry issuance and once it succeeds, they will be considered healthy and tailnet traffic will start getting routed to <code>us-east</code>.</p>\n</li>\n<li>\n<p>If an <code>Ingress</code> does have an HTTP endpoint enabled, the proxies that advertise this <code>Ingress</code> will be considered healthy, even if certificate issuance has failed.\r\nSo, if you have <code>Ingress</code> resources that expose app <code>nginx</code> in clusters in <code>us-east</code>, <code>us-west</code> and <code>eu-west</code>, and certificate issuance only\r\nsucceeds in <code>us-west</code> and <code>eu-west</code>, tailnet traffic will still be routed to all three clusters, but only the HTTP endpoint is guaranteed to be healthy.\r\nThe proxies in <code>us-east</code> will retry issuance and once it succeeds, all clients will be able to reach the app by using HTTPS in any cluster.</p>\n</li>\n</ul>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#debugging\">Debugging</a></h2>\n<h3><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#debugging-ingress-resource-failure\">Debugging <code>Ingress</code> resource failure</a></h3>\n<ol>\n<li>\n<p>Take a look at <a href=\"https://tailscale.com/docs/reference/troubleshooting/containers/kubernetes-operator#operator-logs\">operator's logs</a>.</p>\n</li>\n<li>\n<p>Take a look at the proxies:\r\nFor a <code>ProxyGroup</code> named <code>ingress-proxies</code>:</p>\n</li>\n</ol>\n<pre><code class=\"language-shell\">kubectl get pod -n tailscale -l tailscale.com/parent-resource=\"ingress-proxies\",tailscale.com/parent-resource-type=\"proxygroup\"\r\nNAME                READY   STATUS    RESTARTS   AGE\r\ningress-proxies-0   1/1     Running   0          2d22h\r\ningress-proxies-1   1/1     Running   0          2d22h\n</code></pre>\n<pre><code class=\"language-shell\">kubectl logs ingress-proxies-0 -n tailscale -c tailscale\r\nboot: 2025/03/28 14:20:43 Starting tailscaled\r\n...\n</code></pre>\n<h3><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#validate-to-which-cluster-a-tailnet-clients-requests-are-routed\">Validate to which cluster a tailnet client's requests are routed</a></h3>\n<ol>\n<li>Find the MagicDNS name by which Ingress is exposed to tailnet.\r\nThis will be the DNS name of the corresponding Tailscale Service.</li>\n</ol>\n<pre><code class=\"language-shell\">kubectl get ingress\r\nNAME          CLASS       HOSTS   ADDRESS                 PORTS   AGE\r\nnginx         tailscale   *       nginx.tailxyz.ts.net    443     3h18m\n</code></pre>\n<ol start=\"2\">\n<li>Find the tailnet IP address of the Tailscale Service:</li>\n</ol>\n<pre><code class=\"language-shell\">dig nginx.tailxyz.ts.net +short\r\n100.100.126.127\n</code></pre>\n<p>If the Tailscale Service's IP address can not be resolved, its creation might have failed. Check the operator's logs.</p>\n<ol start=\"3\">\n<li>Find which proxy the tailnet client uses as a backend for the Tailscale Service.\r\nEach tailnet client's requests for a Tailscale Service will be routed to a proxy in the closest cluster.\r\nYou can run the following commands on a client to find which proxy it uses as a backend for a Tailscale Service:</li>\n</ol>\n<pre><code class=\"language-shell\">tailscale status --json | jq '.Peer | to_entries[] | select(.value.AllowedIPs[] | contains(\"100.100.126.127/32\")) | .value |\r\n{DNSName, ID}'\r\n{\r\n     \"DNSName\": \"ingress-proxies-eu-0.tailxyz.ts.net.\",\r\n     \"ID\": \"n9Ch5VvNug11CNTRL\"\r\n}\n</code></pre>\n<p>The first part of DNS name is either <code>proxyGroup.spec.hostnamePrefix</code> or the <code>ProxyGroup</code> name - you can use that to identify which cluster the tailnet client's traffic will be routed to.</p>\n<p>If <code>tailscale status --json</code> does not contain any results for the given tailnet IP address then the client likely does not have permissions to access the Tailscale Service.</p>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#best-practices\">Best practices</a></h2>\n<ul>\n<li>\n<p>Use Let's Encrypt's staging environment for initial testing.</p>\n</li>\n<li>\n<p>Set a different <code>proxygroup.spec.hostnamePrefix</code> field values for <code>ProxyGroup</code> resources in different clusters (or simply name the <code>ProxyGroup</code> resources differently).\r\nThis will ensure that proxies in different clusters can be easily identified by the Tailscale hostname.</p>\n</li>\n</ul>\n<h2><a href=\"https://tailscale.com/docs/features/kubernetes-operator/how-to/multi-cluster-ingress#current-limitations\">Current limitations</a></h2>\n<ul>\n<li>\n<p>Exposing a new <code>Ingress</code> resource takes up to a minute to become available. We are working to make this faster.</p>\n</li>\n<li>\n<p>The current access model requires explicitly defining access for each app in ACLs.\r\nWe are working on allowing tag based access.</p>\n</li>\n<li>\n<p>When exposing <code>Ingress</code> resources with the same DNS name concurrently, you may hit transient failures due to DNS challenge errors.\r\nWe are working on fixing this.</p>\n</li>\n<li>\n<p>Currently Let's Encrypt's staging endpoint can only be enabled for the <code>ProxyGroup</code> as a whole and not for individual <code>Ingress</code> resources.</p>\n</li>\n<li>\n<p>Currently you must ensure that multi-cluster <code>Ingress</code> resources for the same app all have the same tag and all either expose an HTTP endpoint or not.\r\nApplying <code>Ingress</code> resources for the same app, but with different tags or different HTTP endpoint settings, might result in the operator instances in the different clusters continuously attempting to reconcile the Tailscale Service.</p>\n</li>\n</ul>\n<p><img src=\"https://cdn.brandfetch.io/tailscale.com/fallback/lettermark/theme/dark/h/256/w/256/icon?c=1bfwsmEH20zzEfSNTed\" alt=\"Project Logo\"></p>\n<p>Ask AI</p>\n<p>reCAPTCHA</p>\n<p>Recaptcha requires verification.</p>\n<p>protected by <strong>reCAPTCHA</strong></p>\n"}