Introduction
Automated testing is crucial for maintaining software quality. Playwright, a powerful end-to-end testing framework, helps test web applications across multiple browsers. Integrating Playwright with GitLab CI/CD enables automated test execution, ensuring continuous validation of application functionality.
What is Playwright?
Playwright is an open-source testing framework by Microsoft that supports browser automation across Chromium, Firefox, and WebKit. It allows developers to write reliable UI tests using JavaScript, TypeScript, Python, C#, or Java. Key features include:
- Cross-browser support (Chromium, Firefox, WebKit)
- Headless and headful modes
- Network interception
- Automatic waiting for elements
- Parallel and isolated test execution
What is GitLab CI/CD?
GitLab CI/CD is a built-in continuous integration and deployment system in GitLab. It automates code testing, building, and deployment using pipelines defined in a .gitlab-ci.yml file.
Key components:
- Runners: Execute jobs in CI/CD pipelines
- Jobs: Individual tasks in a pipeline
- Stages: Groups of related jobs (e.g., build, test, deploy)
- Artifacts: Files generated during job execution (e.g., test reports)
Why Integrate Playwright with GitLab CI?
Integrating Playwright with GitLab CI/CD ensures:
Automated test execution on every commit/merge request
Cross-browser testing in a CI environment
Test reports and logs for debugging failures
Consistent test execution across environments
Early bug detection before deployment
Prerequisites
Before setting up Playwright with GitLab CI, ensure the following:
GitLab Account and Repository Setup
- A GitLab repository with a configured runner (shared or self-hosted).
Basic Understanding of CI/CD Pipelines
- Knowledge of
.gitlab-ci.yml syntax and CI/CD workflows.
Existing Playwright Test Suite
- A working Playwright test setup with sample test cases.
Node.js and npm Installed
- Install Node.js (LTS version) and npm for package management.
Setting Up a Playwright Project
1. Initializing a Playwright Project
Run the following commands:
mkdir playwright-gitlab-ci
cd playwright-gitlab-ci
npm init -y
npx playwright@latest install
This sets up Playwright with Chromium, Firefox, and WebKit browsers.
2. Writing Sample Tests
Create a sample test file (tests/example.spec.js):
const { test, expect } = require('@playwright/test');
test('Check homepage title', async ({ page }) => {
await page.goto('https://example.com');
expect(await page.title()).toBe('Example Domain');
});
3. Running Playwright Tests Locally
Execute tests using:
This confirms Playwright is working correctly before integrating with GitLab CI.
Configuring GitLab CI for Playwright
1. Creating .gitlab-ci.yml File
In the root directory, create .gitlab-ci.yml:
stages:
- test
test_playwright:
stage: test
image: mcr.microsoft.com/playwright:v1.41.1-jammy
script:
- npm ci
- npx playwright install --with-deps
- npx playwright test
artifacts:
paths:
- test-results/
when: always
This configures GitLab CI to run Playwright tests in a containerized environment.
2. Choosing the Right Docker Image
Use the official Playwright Docker image (mcr.microsoft.com/playwright) to avoid installation issues.
3. Caching Dependencies (Node Modules)
To speed up pipeline runs, cache node_modules:
cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/
4. Defining Pipeline Stages
Define test as a separate stage to improve pipeline organization.
5. Installing Dependencies and Artifacts
- Dependencies: Installed via
npm ci - Artifacts: Stores test reports/logs for debugging
6. Installing Playwright Browsers in CI
Ensure browsers are installed:
script:
- npx playwright install --with-deps
7. Using Artifacts to Save Test Reports
Store Playwright test reports for later review:
artifacts:
paths:
- test-results/
when: always
8. Configuring Timeouts and Retries
Modify playwright.config.js to handle retries:
use: {
timeout: 30000, // 30 seconds
retries: 2,
}
This improves test stability in CI.
Running Playwright Tests in GitLab CI
1. Example Pipeline Configuration
A full .gitlab-ci.yml example with parallel execution and reporting:
stages:
- test
test_playwright:
stage: test
image: mcr.microsoft.com/playwright:v1.41.1-jammy
script:
- npm ci
- npx playwright install --with-deps
- npx playwright test --reporter=html,junit
artifacts:
paths:
- test-results/
- playwright-report/
when: always
parallel: 2
2. Parallel Test Execution
Improve test speed using parallel workers:
Or configure in playwright.config.js:
3. Handling Headless and Headful Modes
Playwright runs headless by default in CI. To debug:
npx playwright test --headed
Reporting Test Results
1. Generating JUnit/HTML Reports
Modify playwright.config.js:
reporter: [['html'], ['junit', { outputFile: 'junit.xml' }]]
2. Publishing Reports in GitLab CI
Upload reports as artifacts:
artifacts:
paths:
- playwright-report/
3. Interpreting Test Results in GitLab
View test results under CI/CD > Jobs > Browse Artifacts.
Advanced Configuration
1. Environment Variables for Secrets (e.g., Credentials)
Store sensitive data in GitLab CI/CD Variables (Settings > CI/CD > Variables).
Use in .gitlab-ci.yml:
script:
- echo "TOKEN=${TOKEN}" >> .env
2. Cross-Browser Testing (Chromium, Firefox, WebKit)
Modify playwright.config.js:
projects: [
{ name: 'chromium', use: { browserName: 'chromium' } },
{ name: 'firefox', use: { browserName: 'firefox' } },
{ name: 'webkit', use: { browserName: 'webkit' } }
]
3. Using Services (e.g., Databases, APIs) in CI
Run dependent services using GitLab’s services:
services:
- name: postgres:latest
alias: db
Troubleshooting Common Issues
Browser Launch Failures in CI
- Ensure
--with-deps is installed (npx playwright install --with-deps).
Dependency Version Conflicts
- Use
npm ci instead of npm install.
Debugging Pipeline Logs
- Check logs under CI/CD > Jobs > View Job Logs.
- Run tests locally with
DEBUG=pw:browser npx playwright test.
Best Practices
Optimizing Pipeline Speed with Caching
Securing Sensitive Data (Use CI/CD Variables)
Maintaining Test Suites for CI (Regular updates and cleanup)
Conclusion
Automating Playwright tests in GitLab CI ensures reliable test execution.
CI/CD integration helps detect bugs early, improving software quality.
Next Steps: Expand coverage, optimize performance, and explore Playwright’s advanced features.
Additional Resources
- Playwright Docs
- GitLab CI/CD Docs
