Optional Components

• 1: Branchprotector
• 2: Exporter
• 3: gcsupload
• 4: Gerrit
• 5: HMAC
• 6: jenkins-operator
• 7: status-reconciler
• 8: tot
• 8.1: fallbackcheck
• 9: Gangway (Prow API)
• 10: Sub

1 - Branchprotector

branchprotector configures github branch protection according to a specified policy.

Policy configuration

Extend the primary prow config.yaml document to include a top-level branch-protection key that looks like the following:

branch-protection:
  orgs:
    kubernetes:
      repos:
        test-infra:
          # Protect all branches in kubernetes/test-infra
          protect: true
          # Always allow the org's oncall-team to push
          restrictions:
            teams: ["oncall-team"]
          # Ensure that the extra-process-followed github status context passes.
          # In addition, adds any required prow jobs (aka always_run: true)
          required_status_checks:
            contexts: ["extra-process-followed"]

presubmits:
  kubernetes/test-infra:
  - name: fancy-job-name
    context: fancy-job-name
    always_run: true
    spec:  # podspec that runs job

This config will:

• Enable protection for every branch in the kubernetes/test-infra repo.
• Require extra-process-followed and fancy-job-name status contexts to pass before allowing a merge
  • Although it will always allow oncall-team to merge, even if required contexts fail.
  • Note that fancy-job-name is pulled in automatically from the presubmits config for the repo, if one exists.

Updating

• Send PR with config.yaml changes
• Merge PR
• Done!

Make changes to the policy by modifying config.yaml in your favorite text editor and then send out a PR. When the PR merges prow pushes the updated config . The branchprotector applies the new policies the next time it runs (within 24hrs).

Advanced configuration

Fields

See branch_protection.go and GitHub’s protection api for a complete list of fields allowed inside branch-protection and their meanings. The format is:

branch-protection:
  # default policy here
  orgs:
    foo:
      # this is the foo org policy
      protect: true  # enable protection
      enforce_admins: true  # rules apply to admins
      required_linear_history: true  # enforces a linear commit Git history
      allow_force_pushes: true  # permits force pushes to the protected branch
      allow_deletions: true  # allows deletion of the protected branch
      required_pull_request_reviews:
        dismiss_stale_reviews: false # automatically dismiss old reviews
        dismissal_restrictions: # allow review dismissals
          users:
          - her
          - him
          teams:
          - them
          - those
        require_code_owner_reviews: true  # require a code owner approval
        required_approving_review_count: 1 # number of approvals
      required_status_checks:
        strict: false # require pr branch to be up to date
        contexts: # checks which must be green to merge
        - foo
        - bar
      restrictions: # restrict who can push to the repo
        apps:
        - github-prow-app
        users:
        - her
        - him
        teams:
        - them
        - those

Scope

It is possible to define a policy at the branch-protection, org, repo or branch level. For example:

branch-protection:
  # Protect unless overridden
  protect: true
  # If protected, always require the cla status context
  required_status_checks:
    contexts: ["cla"]
  orgs:
    unprotected-org:
      # Disable protection unless overridden (overrides parent setting of true)
      protect: false
      repos:
        protected-repo:
          protect: true
          # Inherit protect-by-default config from parent
          # If protected, always require the tested status context
          required_status_checks:
            contexts: ["tested"]
          branches:
            secure:
              # Protect the secure branch (overrides inhereted parent setting of false)
              protect: true
              # Require the foo status context
              required_status_checks:
                contexts: ["foo"]
    different-org:
      # Inherits protect-by-default: true setting from above

The general rule for how to compute child values is:

• If the child value is null or missing, inherit the parent value.
• Otherwise:
  • List values (like contexts), create a union of the parent and child lists.
  • For bool/int values (like protect), the child value replaces the parent value.

So in the example above:

• The secure branch in unprotected-org/protected-repo
  • enables protection (set a branch level)
  • requires foo tested cla status contexts (the latter two are appended by ancestors)
• All other branches in unprotected-org/protected-repo
  • disable protection (inherited from org level)
• All branches in all other repos in unprotected-org
  • disable protection (set at org level)
• All branches in all repos in different-org
  • Enable protection (inherited from branch-protection level)
  • Require the cla context to be green to merge (appended by parent)

Developer docs

Run unit tests

go test ./cmd/branchprotector

Run locally

go run ./cmd/branchprotector --help, which will tell you about the current flags.

Do a dry run (which will not make any changes to github) with something like the following command:

go run ./cmd/branchprotector \
  --config-path=/path/to/config.yaml \
  --github-token-path=/path/to/my-github-token

This will say how the binary will actually change github if you add a --confirm flag.

Deploy local changes to dev cluster

Run things like the following:

# Build image locally and push it to <YOUR_REGISTRY>
make push-single-image PROW_IMAGE=cmd/branchprotector REGISTRY=<YOUR_REGISTRY>

This will build an image with your local changes, and push it to <YOUR_REGISTRY>.

Or, if you just want to build an image but not to push, run the following:

# Build image locally
make build-single-image PROW_IMAGE=cmd/branchprotector

This will build an image with your local changes, without pushing it to anywhere.

Deploy cronjob to production

branchprotector image is automatically built as part of prow, see “How to update the cluster” for more details.

Branchprotector runs as a prow periodic job, for example ci-test-infra-branchprotector.

2 - Exporter

The prow-exporter exposes metrics about prow jobs while the metrics are not directly related to a specific prow-component.

Metrics

For example, the metric prow_job_labels is similar to kube_pod_labels defined in kubernetes/kube-state-metrics. A typical usage of prow_job_labels is to join it with other metrics using a Prometheus matching operator.

Note that job_name is .spec.job instead of .metadata.name as taken in kube_pod_labels. The gauge value is always 1 because we have another metric prowjobs for the number jobs by name. The metric here shows only the existence of such a job with the label set in the cluster.

3 - gcsupload

gcsupload uploads artifacts to cloud storage at a path resolved from the job configuration.

gcsupload can be configured by either passing in flags or by specifying a full set of options as JSON in the $GCSUPLOAD_OPTIONS environment variable, which has the following form:

{
    "bucket": "kubernetes-jenkins",
    "sub_dir": "",
    "items": [
        "/logs/artifacts/"
    ],
    "path_strategy": "legacy",
    "default_org": "kubernetes",
    "default_repo": "kubernetes",
    "gcs_credentials_file": "/secrets/gcs/service-account.json",
    "dry_run": "false"
}

In addition to this configuration for the tool, the $JOB_SPEC environment variable should be present to provide the contents of the Prow downward API for jobs. This data is used to resolve the exact location in GCS to which artifacts and logs will be pushed.

The path strategy field can be one of "legacy", "single", and "explicit". This field determines how the organization and repository of the code under test is encoded into the GCS path for the test artifacts:

For historical reasons, the "legacy" or "single" strategies may already be in use for some; however, for new deployments it is strongly advised to use the "explicit" strategy.

4 - Gerrit

Gerrit is a Prow-gerrit adapter for handling CI on gerrit workflows. It can poll gerrit changes from multiple gerrit instances, and trigger presubmits on Prow upon new patchsets on Gerrit changes, and postsubmits when Gerrit changes are merged.

Deployment Usage

When deploy the gerrit component, you need to specify --config-path to your prow config, and optionally --job-config-path to your prowjob config if you have split them up.

Set --gerrit-projects to the gerrit projects you want to poll against.

Example: If you want prow to interact with gerrit project foo and bar on instance gerrit-1.googlesource.com and also project baz on instance gerrit-2.googlesource.com, then you can set:

--gerrit-projects=gerrit-1.googlesource.com=foo,bar
--gerrit-projects=gerrit-2.googlesource.com=baz

--cookiefile allows you to specify a git https cookie file to interact with your gerrit instances, leave it empty for anonymous access to gerrit API.

--last-sync-fallback should point to a persistent volume that saves your last poll to gerrit.

Underlying infra

Also take a look at gerrit related packages for implementation details.

You might also want to deploy Crier which reports job results back to gerrit.

5 - HMAC

hmac is a tool to update the HMAC token, GitHub webhooks and HMAC secret for the orgs/repos as per the managed_webhooks configuration changes in the Prow config file.

Prerequisites

To run this tool, you’ll need:

1. A github account that has admin permission to the orgs/repos.

2. A personal access token for the github account. Note the token must be granted admin:repo_hook and admin:org_hook scopes.

3. Permissions to read&write the hmac secret in the Prow cluster.

How to run this tool

There are two ways to run this tool:

1. Run it on local:

go run ./cmd/hmac \
  --config-path=/path/to/prow/config \
  --github-token-path=/path/to/oauth/secret \
  --kubeconfig=/path/to/kubeconfig \
  --kubeconfig-context=[context of the cluster to connect] \
  --hmac-token-secret-name=[hmac secret name in Prow cluster] \
  --hmac-token-key=[key of the hmac tokens in the secret] \
  --hook-url http://an.ip.addr.ess/hook \
  --dryrun=true  # Remove it to actually update hmac tokens and webhooks

2. Run it as a Prow job:

The recommended way to run this tool would be running it as a postsubmit job. One example Prow job configured for k8s Prow can be found here.

How it works

Given a new managed_webhooks configuration in the Prow core config file, the tool can reconcile the current state of HMAC tokens, secrets and webhooks to meet the new configuration.

Configuration example

Below is a typical example for the managed_webhooks configuration:

managed_webhooks:
  # Whether this tool should respect the legacy global token.
  # This has to be true if any of the managed repo/org is using the legacy global token that is manually created.   
  respect_legacy_global_token: true
  # Controls whether org/repo invitation for prow bot should be automatically
  # accepted or not. Only admin level invitations related to orgs and repos
  # in the managed_webhooks config will be accepted and all other invitations
  # will be left pending.
  auto_accept_invitation: true
  # Config for orgs and repos that have been onboarded to this Prow instance.
  org_repo_config:
    qux:
      token_created_after: 2017-10-02T15:00:00Z
    foo/bar:
      token_created_after: 2018-10-02T15:00:00Z
    foo/baz:
      token_created_after: 2019-10-02T15:00:00Z

Workflow example

Suppose the current org_repo_config in the managed_webhooks configuration is

qux:
  token_created_after: 2017-10-02T15:00:00Z
foo/bar:
  token_created_after: 2018-10-02T15:00:00Z
foo/baz:
  token_created_after: 2019-10-02T15:00:00Z

There can be 3 scenarios to modify the configuration, as explained below:

Rotate an existing HMAC token

User updates the token_created_after for foo/baz to a later time, as shown below:

qux:
  token_created_after: 2017-10-02T15:00:00Z
foo/bar:
  token_created_after: 2018-10-02T15:00:00Z
foo/baz:
  token_created_after: 2020-03-02T15:00:00Z

The hmac tool will generate a new HMAC token for the foo/baz repo, add the new token to the secret, and update the webhook for the repo. And after the update finishes, it will delete the old token.

Onboard a new repo

User adds a new repo foo/bax in the managed_webhooks configuration, as shown below:

qux:
  token_created_after: 2017-10-02T15:00:00Z
foo/bar:
  token_created_after: 2018-10-02T15:00:00Z
foo/baz:
  token_created_after: 2019-10-02T15:00:00Z
foo/bax:
  token_created_after: 2020-03-02T15:00:00Z

The hmac tool will generate an HMAC token for the foo/bax repo, add the token to the secret, and add the webhook for the repo.

Remove an existing repo

User deletes the repo foo/baz from the managed_webhooks configuration, as shown below:

qux:
  token_created_after: 2017-10-02T15:00:00Z
foo/bar:
  token_created_after: 2018-10-02T15:00:00Z

The hmac tool will delete the HMAC token for the foo/baz repo from the secret, and delete the corresponding webhook for this repo.

Note the 3 types of config changes can happen together, and hmac tool is able to handle all the changes in one single run.

6 - jenkins-operator

jenkins-operator is a controller that enables Prow to use Jenkins as a backend for running jobs.

Jenkins configuration

A Jenkins master needs to be provided via --jenkins-url in order for the operator to make requests to. By default, --dry-run is set to true so the operator will not make any mutating requests to Jenkins, GitHub, and Kubernetes, but you most probably want to set it to false. The Jenkins operator expects to read the Prow configuration by default in /etc/config/config.yaml which can be configured with --config-path.

The following stanza is config that can be optionally set in the Prow config file:

jenkins_operators:
- max_concurrency: 150
  max_goroutines: 20
  job_url_template: 'https://storage-for-logs/{{if eq .Spec.Type "presubmit"}}pr-logs/pull{{else if eq .Spec.Type "batch"}}pr-logs/pull{{else}}logs{{end}}{{if ne .Spec.Refs.Repo "origin"}}/{{.Spec.Refs.Org}}_{{.Spec.Refs.Repo}}{{end}}{{if eq .Spec.Type "presubmit"}}/{{with index .Spec.Refs.Pulls 0}}{{.Number}}{{end}}{{else if eq .Spec.Type "batch"}}/batch{{end}}/{{.Spec.Job}}/{{.Status.BuildID}}/'
  report_template: '[Full PR test history](https://pr-history/{{if ne .Spec.Refs.Repo "origin"}}{{.Spec.Refs.Org}}_{{.Spec.Refs.Repo}}/{{end}}{{with index .Spec.Refs.Pulls 0}}{{.Number}}{{end}}).'

• max_concurrency is the maximum number of Jenkins builds that can run in parallel, otherwise the operator is not going to start new builds. Defaults to 0, which means no limit.
• max_goroutines is the maximum number of goroutines that the operator will spin up to handle all Jenkins builds. Defaulted to 20.
• job_url_template is a Golang-templated URL that shows up in the Details button next to the GitHub job status context. A ProwJob is provided as input to the template.
• report_template is a Golang-templated message that shows up in GitHub in case of a job failure. A ProwJob is provided as input to the template.

Security

Various flavors of authentication are supported:

• basic auth, using --jenkins-user and --jenkins-token-file.
• OpenShift bearer token auth, using --jenkins-bearer-token-file.
• certificate-based auth, using --cert-file, --key-file, and optionally --ca-cert-file.

Basic auth and bearer token are mutually exclusive options whereas cert-based auth is complementary to both of them.

If CSRF protection is enabled in Jenkins, --csrf-protect=true needs to be used on the operator’s side to allow Prow to work correctly.

Logs

Apart from a controller, the Jenkins operator also runs a http server to serve Jenkins logs. You can configure the Prow frontend to show Jenkins logs with the following Prow config:

deck:
  external_agent_logs:
  - agent: jenkins
    url_template: 'http://jenkins-operator/job/{{.Spec.Job}}/{{.Status.BuildID}}/consoleText'

Deck uses url_template to contact jenkins-operator when a user clicks the Build log button of a Jenkins job (agent: jenkins). jenkins-operator forwards the request to Jenkins and serves back the response.

NOTE: Deck will display the Build log button on the main page when the agent is not kubernetes regardless the external agent log was configured on the server side. Deck has no way to know if the server side configuration is consistent when rendering jobs on the main page.

Job configuration

Below follows the Prow configuration for a Jenkins job:

presubmits:
  org/repo:
  - name: pull-request-unit
    agent: jenkins
    always_run: true
    context: ci/prow/unit
    rerun_command: "/test unit"
    trigger: "((?m)^/test( all| unit),?(\\s+|$))"

You can read more about the different types of Prow jobs elsewhere. What is interesting for us here is the agent field which needs to be set to jenkins in order for jobs to be dispatched to Jenkins and name which is the name of the job inside Jenkins.

The following parameters must be added within each Jenkins job:

• BUILD_ID
• PROW_JOB_ID

Sharding

Sharding of Jenkins jobs is supported via Kubernetes labels and label selectors. This enables Prow to work with multiple Jenkins masters. Three places need to be configured in order to use sharding:

• --label-selector in the Jenkins operator.
• label_selector in jenkins_operators in the Prow config.
• labels in the job config.

For example, one would set the following options:

• --label-selector=master=jenkins-master in a Jenkins operator.

This option forces the operator to list all ProwJobs with master=jenkins-master.

• label_selector: master=jenkins-master in the Prow config.

jenkins_operators:
- label_selector: master=jenkins-master
  max_concurrency: 150
  max_goroutines: 20

jenkins_operators in the Prow config can be read by multiple running operators and based on label_selector, each operator knows which config stanza does it need to use. Thus, --label-selector and label_selector need to match exactly.

• labels: jenkins-master in the job config.

presubmits:
  org/repo:
  - name: pull-request-unit
    agent: jenkins
    labels:
      master: jenkins-master
    always_run: true
    context: ci/prow/unit
    rerun_command: "/test unit"
    trigger: "((?m)^/test( all| unit),?(\\s+|$))"

Labels in the job config are set in ProwJobs during their creation.

Kubernetes client

The Jenkins operator acts as a Kubernetes client since it manages ProwJobs backed by Jenkins builds. It is expected to run as a pod inside a Kubernetes cluster and so it uses the in-cluster client config.

GitHub integration

The operator needs to talk to GitHub for updating commit statuses and adding comments about failed tests. Note that this functionality may potentially move into its own service, then the Jenkins operator will not need to contact the GitHub API. The required options are already defaulted:

• github-token-path set to /etc/github/oauth. This is the GitHub bot oauth token that is used for updating job statuses and adding comments in GitHub.
• github-endpoint set to https://api.github.com.

Prometheus support

The following Prometheus metrics are exposed by the operator:

• jenkins_requests is the number of Jenkins requests made.
  • verb is the type of request (GET, POST)
  • handler is the path of the request, usually containing a job name (eg. job/test-pull-request-unit).
  • code is the status code of the request (200, 404, etc.).
• jenkins_request_retries is the number of Jenkins request retries made.
• jenkins_request_latency is the time for a request to roundtrip between the operator and Jenkins.
• resync_period_seconds is the time the operator takes to complete one reconciliation loop.
• prowjobs is the number of Jenkins prowjobs in the system.
  • job_name is the name of the job.
  • type is the type of the prowjob: presubmit, postsubmit, periodic, batch
  • state is the state of the prowjob: triggered, pending, success, failure, aborted, error

If a push gateway needs to be used it can be configured in the Prow config:

push_gateway:
  endpoint: http://prometheus-push-gateway
  interval: 1m

7 - status-reconciler

status-reconciler ensures that changes to blocking presubmits in Prow configuration while PRs are in flight do not cause those PRs to get stuck.

When the set of blocking presubmits changes for a repository, one of three cases occurs:

• a new blocking presubmit exists and should be triggered for every trusted pull request in flight
• an existing blocking presubmit is removed and should have its' status retired
• an existing blocking presubmit is renamed and should have its' status migrated

The status-reconciler watches the job configuration for Prow and ensures that the above actions are taken as necessary.

To exclude repos from being reconciled, passing flag --denylist, this can be done repeatedly. This is useful when moving a repo from prow instance A to prow instance B, while unwinding jobs from prow instance A, the jobs are not expected to be blindly lablled succeed by prow instance A.

Note that status-reconciler is edge driven (not level driven) so it can’t be used retrospectively.

8 - tot

This is a placeholder page. Some contents needs to be filled.

8.1 - fallbackcheck

Ensure your GCS bucket layout is what tot expects to use. Useful when you want to transition from versioning your GCS buckets away from Jenkins build numbers to build numbers vended by prow.

fallbackcheck checks the existence of latest-build.txt files as per the documented GCS layout. It ignores jobs that have no GCS buckets.

Install

go get sigs.k8s.io/prow/cmd/tot/fallbackcheck

Run

fallbackcheck -bucket GCS_BUCKET -prow-url LIVE_DECK_DEPLOYMENT

For example:

fallbackcheck -bucket https://gcsweb-ci.svc.ci.openshift.org/gcs/origin-ci-test/ -prow-url https://deck-ci.svc.ci.openshift.org/

9 - Gangway (Prow API)

Architecture

See the design doc.

Gangway uses gRPC to serve several endpoints. These can be seen in the gangway.proto file, which describes the gRPC endpoints. The proto describes the interface at a high level, and is converted into low-level Golang types into gangway.pb.go and gangway_grpc.pb.go. These low-level Golang types are then used in the gangway.go file to implement the high-level intent of the proto file.

As Gangway only understands gRPC natively, if you want to use a REST client against it you must deploy Gangway. For example, on GKE you can use Cloud Endpoints and deploy Gangway behind a reverse proxy called “ESPv2”. This ESPv2 container will forward HTTP requests made to it to the equivalent gRPC endpoint in Gangway and back again.

Configuration setup

Server-side configuration

Gangway has its own security check to see whether the client is allowed to, for example, trigger the job that it wants to trigger (we don’t want to let any random client trigger any Prow Job that Prow knows about). In the central Prow config under the gangway section, prospective Gangway users can list themselves in there. For an example, see the section filled out for Gangway’s own integration tests and search for allowed_jobs_filters.

Client-side configuration

The table below lists the supported endpoints.

See gangway.proto and the Gangway Google client.

Tutorial

See the example.

10 - Sub

Sub is a Prow component that can trigger new Prow jobs (PJs) using Pub/Sub messages. The message does not need to have the full PJ defined; instead you just need to have the job name and some other key pieces of information (more on this below). The rest of the data needed to create a full-blown PJ is derived from the main Prow configuration (or inrepoconfig).

Deployment Usage

Sub can listen to Pub/Sub subscriptions (known as “pull subscriptions”).

When deploy the sub component, you need to specify --config-path to your prow config, and optionally --job-config-path to your prowjob config if you have split them up.

Notable options:

• --dry-run: Dry run for testing. Uses API tokens but does not mutate.
• --grace-period: On shutdown, try to handle remaining events for the specified duration.
• --port: On shutdown, try to handle remaining events for the specified duration.
• --github-app-id and --github-app-private-key-path=/etc/github/cert: Used to authenticate to GitHub for cloning operations as a GitHub app. Mutually exclusive with --cookiefile.
• --cookiefile: Used to authenticate git when cloning from https://... URLs. See http.cookieFile in man git-config.
• --in-repo-config-cache-size: Used to cache Prow configurations fetched from inrepoconfig-enabled repos.

flowchart TD

    classDef yellow fill:#ff0
    classDef cyan fill:#0ff
    classDef pink fill:#f99

    subgraph Service Cluster
        PCM[Prow Controller Manager]:::cyan
        Prowjob:::yellow
        subgraph Sub
            staticconfig["Static Config
                (/etc/job-config)"]
            inrepoconfig["Inrepoconfig
                (git clone <inrepoconfig>)"]
            YesOrNo{"Is my-prow-job-name
                in the config?"}
            Yes
            No
        end
    end

    subgraph Build Cluster
        Pod:::yellow
    end

    subgraph GCP Project
        subgraph Pub/Sub
            Topic
            Subscription
        end
    end
    
    subgraph Message
        Payload["{"data":
            {"name":"my-prow-job-name",
            "attributes":{"prow.k8s.io/pubsub.EventType": "..."},
            "data": ...,
            ..."]
    end

    Message --> Topic --> Subscription --> Sub --> |Pulls| Subscription
    staticconfig --> YesOrNo
    inrepoconfig -.-> YesOrNo
    YesOrNo --> Yes --> |Create| Prowjob --> PCM --> |Create| Pod
    YesOrNo --> No --> |Report failure| Topic

Sending a Pub/Sub Message

Pub/Sub has a generic PubsubMessage type that has the following JSON structure:

{
  "data": string,
  "attributes": {
    string: string,
    ...
  },
  "messageId": string,
  "publishTime": string,
  "orderingKey": string
}

The Prow-specific information is encoded as JSON as the string value of the data key.

Pull Server

All pull subscriptions need to be defined in Prow Configuration:

pubsub_subscriptions:
  "gcp-project-01":
  - "subscription-01"
  - "subscription-02"
  - "subscription-03"
  "gcp-project-02":
  - "subscription-01"
  - "subscription-02"
  - "subscription-03"

Sub must be running with GOOGLE_APPLICATION_CREDENTIALS environment variable pointing to the service account credentials JSON file. The service account used must have the right permission on the subscriptions (Pub/Sub Subscriber, and Pub/Sub Editor).

More information at https://cloud.google.com/pubsub/docs/access-control.

Periodic Prow Jobs

When creating your Pub/Sub message, for the attributes field, add a key prow.k8s.io/pubsub.EventType with value prow.k8s.io/pubsub.PeriodicProwJobEvent. Then for the data field, use the following JSON as the value:

{
  "name":"my-periodic-job",
  "envs":{
    "GIT_BRANCH":"v.1.2",
    "MY_ENV":"overwrite"
  },
  "labels":{
    "myLabel":"myValue",
  },
  "annotations":{
    # GCP project where Prow Job statuses are published by Prow. Must also
    # provide "prow.k8s.io/pubsub.topic" to take effect.
    #
    # It's highly recommended to configure this even if prowjobs are monitored
    # by other means, because this is also where errors are reported when the
    # jobs are failed to be triggered.
    "prow.k8s.io/pubsub.project":"myProject",

    # Unique run ID.
    "prow.k8s.io/pubsub.runID":"asdfasdfasdf",

    # GCP pubsub topic where Prow Job statuses are published by Prow. Must be a
    # different topic from where this payload is published to.
    "prow.k8s.io/pubsub.topic":"myTopic"
  }
}

Note: the # lines are comments for purposes of explanation in this doc; JSON does not permit comments so make sure to remove them in your actual payload.

The above payload will ask Prow to find and trigger the periodic job named my-periodic-job, and add/overwrite the annotations and environment variables on top of the job’s default annotations. The prow.k8s.io/pubsub.* annotations are used to publish job statuses.

Note: periodic jobs always clone source code from ref (a branch) instead of a specific SHA. If you need to trigger a job based on a specific SHA you can use a postsubmit job instead.

Postsubmit Prow Jobs

Triggering presubmit job is similar to periodic jobs. Two things to change:

• instead of an attributes with key prow.k8s.io/pubsub.EventType and value prow.k8s.io/pubsub.PeriodicProwJobEvent, replace the value with prow.k8s.io/pubsub.PostsubmitProwJobEvent
• requires setting refs instructing postsubmit jobs how to clone source code:

{
  # Common fields as above
  "name":"my-postsubmit-job",
  "envs":{...},
  "labels":{...},
  "annotations":{...},

  "refs":{
    "org": "org-a",
    "repo": "repo-b",
    "base_ref": "main",
    "base_sha": "abc123"
  }
}

This will start postsubmit job my-postsubmit-job, clones source code from base_ref at base_sha.

(There are more fields can be supplied, see full documentation)

Presubmit Prow Jobs

Triggering presubmit jobs is similar to postsubmit jobs. Two things to change:

• instead of an attributes with key prow.k8s.io/pubsub.EventType and value prow.k8s.io/pubsub.PostsubmitProwJobEvent, replace the value with prow.k8s.io/pubsub.PresubmitProwJobEvent
• for the refs field, additionally supply a pulls field, like this:

{
  # Common fields as above
  "name":"my-presubmit-job",
  "envs":{...},
  "labels":{...},
  "annotations":{...},

  "refs":{
    "org": "org-a",
    "repo": "repo-b",
    "base_ref": "main",
    "base_sha": "abc123",
    "pulls": [
      {
        "sha": "def456"
      }
    ]
  }
}

This will start presubmit job my-presubmit-job, clones source code like pull requests defined under pulls, which merges to base_ref at base_sha.

(There are more fields that can be supplied, see full documentation. For example, if you want the job to be reported on the PR, add number field right next to sha)

Gerrit Presubmits and Postsubmits

Gerrit presubmit and postsubmit jobs require some additional labels and annotations to be specified in the pubsub payload if you wish for them to report results back to the Gerrit change. Specifically the following annotations must be supplied (values are examples):

  annotations:
    prow.k8s.io/gerrit-id: my-repo~master~I79eee198f020c2ff23d49dbe4d2b2ef7cdc4091b
    prow.k8s.io/gerrit-instance: https://my-project-review.googlesource.com
  labels:
    prow.k8s.io/gerrit-patchset: "4"
    prow.k8s.io/gerrit-revision: 2b8cafaab9bd3a829a6bdaa819a18f908bc677ca