Re-executing batch specs multiple times

Idempotency as goal

One goal behind the design of Batch Changes and the src batch [apply|preview] commands in the Sourcegraph CLI is idempotency.

Re-applying the same batch spec produces the same batch change and changesets.

In practical terms: a user should be able to run src batch apply -f my.batch.yaml multiple times and, if my.batch.yaml didn't change, the batch change shouldn't change.

Exceptions

That can only work, though, if the inputs to the batch spec stay the same too.

Since a batch spec's on.repositoriesMatchingQuery uses Sourcegraph's search functionality to dynamically produce a list of repositories in which to execute the steps, the list of repositories might change between executions of src batch apply.

If that's the case, the steps need to be re-executed in any newly found repositories. Changesets in repositories that are not found anymore will be closed and archived in the batch change (see "Updating a batch change" for more details). In repositories that are unchanged Sourcegraph CLI tries to use cached results to avoid having to re-execute the steps.

Local caching

Whenever Sourcegraph CLI re-executes the same batch spec it checks a local cache to see if it already executed the same steps in a given repository.

Whether a cached result can be used is dependent on multiple things:

  1. the repository's default branch's revision didn't change (because if new commits have been pushed to the repository, re-executing the steps might lead to different results)
  2. the steps themselves didn't change, including and all their inputs, such as steps.env, and the steps.run field (which can change between executions if it uses templating and is dynamically built from search results)

That also means that Sourcegraph CLI can use cached results when re-executing a changed batch spec, as long as the changes didn't affect the steps and the results they produce. For example: if only the changesetTemplate.title field has been changed, cached results can be used, since that field doesn't have any influence on the steps and their results.