Continuous Integration & Deployment¶
Bedrock runs a series of automated tests as part of continuous integration workflow and Deployment Pipeline. You can learn more about each of the individual test suites by reading their respective pieces of documentation:
- Python unit tests (see Run the tests).
- Redirect tests (see Testing redirects).
- Functional tests (see Front-end testing).
Tests in the lifecycle of a change¶
Below is an overview of the tests during the lifecycle of a change to bedrock:
The change is developed locally, and all integration tests can be executed against a locally running instance of the application.
Push to master branch¶
Whenever a change is pushed to the master branch, a new image is built and deployed to the dev environment, and the full suite of headless and UI tests are then run against Firefox on Windows 10 using Sauce Labs. This is handled by the pipeline, and is subject to change according to the settings in the .gitlab-ci.yml file in the www-config repository.
Push to stage branch¶
Whenever a change is pushed to the stage branch, a production docker image is built, published to Docker Hub, and deployed to a public staging environment. Once the new image is deployed, the full suite of UI tests is run against it with Firefox, Chrome, and Internet Explorer on Windows 10, and the sanity suite is run with IE9.
Push to prod branch (tagged)¶
When a tagged commit is pushed to the prod branch, a production docker image is built and published to Docker Hub if needed (usually this will have already happened as a result of a push to the stage branch), and deployed to each production deployment. After each deployment is complete, the full suite of UI tests is run against Firefox, Chrome and Internet Explorer on Windows 10, and the sanity suite is run against IE9. As with untagged pushes, this is all handled by the pipeline, and is subject to change according to the settings in the .gitlab-ci.yml file in the www-config repository.
Push to prod cheat sheet
- Check out the
- Make sure the
masterbranch is up to date with
- Check that dev deployment is green:
- View deployment pipeline and look at
- View deployment pipeline and look at
- Tag and push the deployment by running
By default the
tag-release.sh script will push to the
origin git remote. If you’d
like for it to push to a different remote name you can either pass in a
--remote argument, or set the
MOZ_GIT_REMOTE environment variable. So the following
$ bin/tag-release.sh --push -r mozilla $ MOZ_GIT_REMOTE=mozilla bin/tag-release.sh --push
And if you’d like to just tag and not push the tag anywhere, you may omit the
Instance Configuration & Switches¶
We have a separate repo for configuring our primary instances (dev, stage, and prod).
The docs for updating configurations in that repo are on their own page,
but there is a way to tell what version of the configuration is in use on any particular instance of bedrock.
You can go to the
/healthz-cron/ URL on an instance (see prod for example) to see the current
commit of all of the external Git repos in use by the site and how long ago they were updated. The info on that page also includes the latest
version of the database in use, the git revision of the bedrock code, and how long ago the database was updated. If you recently made
a change to one of these repos and are curious if the changes have made it to production, this is the URL you should check.
There are two components for Selenium, which are independently versioned. The first is
the Python client, and this can be updated via the test dependencies. The other
component is the server, which in the pipeline is either provided by a Docker container
or Sauce Labs. The
SELENIUM_VERSION environment variable controls both of these, and
they should ideally use the same version, however it’s possible that availability of
versions may differ. You can check the Selenium Docker versions available. If needed, the global
default can be set and then can be overridden in the individual job configuration.
Adding test runs¶
Test runs can be added by creating a new job in the .gitlab-ci.yml file in the www-config repository with the desired variables. For example, if you wanted to run tests in Firefox on both Windows 10 and OS X, against our dev environment, you could create the following clauses:
dev-test-firefox-osx: extends: - .dev - .test variables: BROWSER_NAME: firefox BROWSER_VERSION: latest PLATFORM: OS X 10.11 dev-test-firefox-win10: extends: - .dev - .test variables: BROWSER_NAME: firefox BROWSER_VERSION: latest PLATFORM: Windows 10
You can use Sauce Labs platform configurator to help with the parameter values.
If you have commit rights to our Github repo (mozilla/bedrock) you can simply push
your branch to the branch named
run-integration-tests, and the
app will be deployed and the full suite of integration tests for that branch will be run.
Please announce in our Slack channel (#www on mozilla.slack.com) that you’ll be doing this so
that we don’t get conflicts.