How To Configure Pub/Sub Auth Subscription

Explains how to subscribe to pubsubhub to receive update notifications for a YouTube channel

As a case study project I wanted to dynamically update a web page showcasing my YouTube channel content. The goal was to automate the process of updating the web page with new video uploads, without requiring any manual intervention or redeploying the website.

Part 3 of the real time web notification series explains how I configured a Google Pub/Sub subscription for push authentication.

A detailed walkthrough is provided for developers on how to configure Pub/Sub authentication for a Cloud Run service acting as a webhook. By following these steps, you'll deploy a secure Cloud Run service that processes messages pushed from a Pub/Sub topic, ensuring proper authentication to avoid common issues like 403 Forbidden errors.

Check out the videos page to view a summary of the latest YouTube video content uploaded to my channel.

Prerequisites

A Google Cloud project with billing enabled.
The gcloud CLI installed and authenticated with gcloud auth login.
Basic familiarity with Google Cloud Pub/Sub and Cloud Run.
A container image ready to deploy (e.g., a webhook application).

Overview

We'll set up a Cloud Run service triggered by Pub/Sub via a push subscription. Since the Cloud Run service will require authentication, we'll configure Pub/Sub to use its service account with the necessary permissions to invoke the service. Here are the steps:

Create or Identify a Pub/Sub Topic
Deploy the Cloud Run Service
Retrieve the Cloud Run Service URL
Create a Pub/Sub Push Subscription
Grant IAM Permissions
Verify the Setup

Step 1: Create or Identify a Pub/Sub Topic

Pub/Sub topics are where messages are published. If you don’t already have a topic, create one:

gcloud pubsub topics create my-topic

Replace my-topic with your desired topic name. If you already have a topic, skip this step and note its name for later use.

Step 2: Deploy the Cloud Run Service

Deploy your Cloud Run service with authentication required. This service will act as the webhook to process Pub/Sub messages.

Run the following command:

gcloud run deploy pubsub-webhook \
 --image gcr.io/my-project/my-image \
 --region us-central1 \
 --no-allow-unauthenticated \
 --service-account pubsub-webhook-service@my-project.iam.gserviceaccount.com

Parameters explained:

--image: Your container image (e.g., gcr.io/my-project/my-image).
--region: The region for deployment (e.g., us-central1).
--no-allow-unauthenticated: Ensures only authenticated requests can invoke the service.
--service-account: The service account the Cloud Run service runs as (e.g., pubsub-webhook-service@my-project.iam.gserviceaccount.com). This is for outbound permissions (e.g., calling other APIs), not inbound authentication from Pub/Sub.

Info

Replace placeholders with your project-specific values. If you don’t have a custom service account, you can use the default Compute Engine service account (<project-number>-compute@developer.gserviceaccount.com), but ensure it has the necessary permissions for your application’s logic.

Step 3: Retrieve the Cloud Run Service URL

After deployment, get the service URL, which Pub/Sub will use as the push endpoint:

CLOUD_RUN_URL=$(gcloud run services describe pubsub-webhook \
  --region us-central1 \
  --format='value(status.url)')
echo $CLOUD_RUN_URL

This outputs something like https://pubsub-webhook-<hash>.uc.run.app.
Store this URL for the next step.

Step 4: Create a Pub Sub Push Subscription

Create a push subscription to deliver messages from the topic to the Cloud Run service. Pub/Sub will authenticate requests using its service account.

First, find your project number:

PROJECT_NUMBER=$(gcloud projects describe $(gcloud config get-value project) \
  --format='value(projectNumber)')

Then, create the subscription:

gcloud pubsub subscriptions create my-subscription \
  --topic my-topic \
  --push-endpoint=$CLOUD_RUN_URL \
  --push-auth-service-account=service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com

Parameters Explained:

--topic: The topic you created or identified (e.g., my-topic).
--push-endpoint: The Cloud Run service URL from Step 3.
--push-auth-service-account: Specifies the Pub/Sub service account (service-<project-number>@gcp-sa-pubsub.iam.gserviceaccount.com) to authenticate requests. This account will generate an OIDC token sent with each push request.

Replace my-subscription with your desired subscription name.

Step 5: Grant IAM Permissions

Pub/Sub’s service account needs the roles/run.invoker role on the Cloud Run service to invoke it. Grant this permission:

gcloud run services add-iam-policy-binding pubsub-webhook \
  --region us-central1 \
  --member=serviceAccount:service-${PROJECT_NUMBER}@gcp-sa-pubsub.iam.gserviceaccount.com \
  --role=roles/run.invoker

Key Points:

--member: The Pub/Sub service account identified by your project number.
--role: roles/run.invoker allows invocation of the Cloud Run service.

info

This policy persists across service revisions but is lost if you delete and recreate the service with the same name. Reapply it in such cases.

Step 6: Verify The Setup

Test the configuration by publishing a message and checking the logs.

Publish a Test Message:

gcloud pubsub topics publish my-topic --message="Hello, Cloud Run!"

Check Cloud Run Logs:

gcloud logging read "resource.type=cloud_run_revision AND resource.labels.service_name=pubsub-webhook" \
  --limit=10

If the message appears in the logs, the setup is successful.
If not, see the Troubleshooting section below.

Why This Works

Cloud Run Authentication: By setting --no-allow-unauthenticated, only callers with valid credentials can invoke the service.
Pub/Sub Push: The push subscription sends HTTP requests to the Cloud Run URL, including an OIDC token from the specified service account.
IAM Roles: Granting roles/run.invoker to the Pub/Sub service account ensures it’s authorized to call the service.

Using the Pub/Sub service account (service-<project-number>@gcp-sa-pubsub.iam.gserviceaccount.com) simplifies setup, as it can generate tokens for itself without additional permissions.

Troubleshooting

403 Forbidden Error:

Cause: The Pub/Sub service account lacks roles/run.invoker.
Fix: Verify the IAM policy:

gcloud run services get-iam-policy pubsub-webhook --region us-central1

Ensure the binding exists and reapply Step 5 if needed.

Push Endpoint Mismatch

Cause: The subscription’s --push-endpoint doesn’t match the Cloud Run URL.
Fix: Check the subscription:

gcloud pubsub subscriptions describe my-subscription

Update if necessary:

gcloud pubsub subscriptions update my-subscription --push-endpoint=$CLOUD_RUN_URL

Logs Not Showing

Cause: The webhook logic might not log messages, or there’s an error.
Fix: Add logging to your application and retry the test.

Additional Notes

Service Account Distinction: The --service-account in Step 2 is for the Cloud Run service’s outbound actions (e.g., calling APIs). The --push-auth-service-account in Step 4 is for Pub/Sub’s inbound authentication.
Custom Service Account: If you prefer a different service account for Pub/Sub authentication, use it in Step 4, but grant it roles/run.invoker and ensure the Pub/Sub service account has roles/iam.serviceAccountTokenCreator on it.
Environment Variables: If your webhook needs configuration (e.g., PROJECT_ID), add them during deployment:

--set-env-vars "PROJECT_ID=my-project"