Engineering Medical Law Business Civil Engineering Computer Science Electrical Engineering Class 10 Class 11 Class 12 NEET JEE SSC CUET Mathematics Physics Chemistry Biology History

Playwright with GitLab CI: A Step-by-Step Guide

Last updated on: February 25, 2025

Notesly Team

School Student

Share :

Introduction

Automated testing is crucial for maintaining software quality. Playwright, a powerful end-to-end testing framework, helps test web applications across multiple browsers. Integrating Playwright with GitLab CI/CD enables automated test execution, ensuring continuous validation of application functionality.

What is Playwright?

Playwright is an open-source testing framework by Microsoft that supports browser automation across Chromium, Firefox, and WebKit. It allows developers to write reliable UI tests using JavaScript, TypeScript, Python, C#, or Java. Key features include:

  1. Cross-browser support (Chromium, Firefox, WebKit)
  2. Headless and headful modes
  3. Network interception
  4. Automatic waiting for elements
  5. Parallel and isolated test execution

What is GitLab CI/CD?

GitLab CI/CD is a built-in continuous integration and deployment system in GitLab. It automates code testing, building, and deployment using pipelines defined in a .gitlab-ci.yml file.

Key components:

  1. Runners: Execute jobs in CI/CD pipelines
  2. Jobs: Individual tasks in a pipeline
  3. Stages: Groups of related jobs (e.g., build, test, deploy)
  4. Artifacts: Files generated during job execution (e.g., test reports)

Why Integrate Playwright with GitLab CI?

Integrating Playwright with GitLab CI/CD ensures:

Automated test execution on every commit/merge request

Cross-browser testing in a CI environment

Test reports and logs for debugging failures

Consistent test execution across environments

Early bug detection before deployment

Prerequisites

Before setting up Playwright with GitLab CI, ensure the following:

GitLab Account and Repository Setup

  1. A GitLab repository with a configured runner (shared or self-hosted).

Basic Understanding of CI/CD Pipelines

  1. Knowledge of .gitlab-ci.yml syntax and CI/CD workflows.

Existing Playwright Test Suite

  1. A working Playwright test setup with sample test cases.

Node.js and npm Installed

  1. Install Node.js (LTS version) and npm for package management.

Setting Up a Playwright Project

1. Initializing a Playwright Project

Run the following commands:

mkdir playwright-gitlab-ci
cd playwright-gitlab-ci
npm init -y
npx playwright@latest install

This sets up Playwright with Chromium, Firefox, and WebKit browsers.

2. Writing Sample Tests

Create a sample test file (tests/example.spec.js):


const { test, expect } = require('@playwright/test');

test('Check homepage title', async ({ page }) => {
await page.goto('https://example.com');
expect(await page.title()).toBe('Example Domain');
});

3. Running Playwright Tests Locally

Execute tests using:


npx playwright test

This confirms Playwright is working correctly before integrating with GitLab CI.

Configuring GitLab CI for Playwright

1. Creating .gitlab-ci.yml File

In the root directory, create .gitlab-ci.yml:


stages:
- test

test_playwright:
stage: test
image: mcr.microsoft.com/playwright:v1.41.1-jammy
script:
- npm ci
- npx playwright install --with-deps
- npx playwright test
artifacts:
paths:
- test-results/
when: always

This configures GitLab CI to run Playwright tests in a containerized environment.

2. Choosing the Right Docker Image

Use the official Playwright Docker image (mcr.microsoft.com/playwright) to avoid installation issues.

3. Caching Dependencies (Node Modules)

To speed up pipeline runs, cache node_modules:


cache:
key: ${CI_COMMIT_REF_SLUG}
paths:
- node_modules/

4. Defining Pipeline Stages

Define test as a separate stage to improve pipeline organization.

5. Installing Dependencies and Artifacts

  1. Dependencies: Installed via npm ci
  2. Artifacts: Stores test reports/logs for debugging

6. Installing Playwright Browsers in CI

Ensure browsers are installed:


script:
- npx playwright install --with-deps

7. Using Artifacts to Save Test Reports

Store Playwright test reports for later review:


artifacts:
paths:
- test-results/
when: always

8. Configuring Timeouts and Retries

Modify playwright.config.js to handle retries:


use: {
timeout: 30000, // 30 seconds
retries: 2,
}

This improves test stability in CI.

Running Playwright Tests in GitLab CI

1. Example Pipeline Configuration

A full .gitlab-ci.yml example with parallel execution and reporting:


stages:
- test

test_playwright:
stage: test
image: mcr.microsoft.com/playwright:v1.41.1-jammy
script:
- npm ci
- npx playwright install --with-deps
- npx playwright test --reporter=html,junit
artifacts:
paths:
- test-results/
- playwright-report/
when: always
parallel: 2

2. Parallel Test Execution

Improve test speed using parallel workers:


parallel: 2

Or configure in playwright.config.js:


workers: 4

3. Handling Headless and Headful Modes

Playwright runs headless by default in CI. To debug:


npx playwright test --headed

Reporting Test Results

1. Generating JUnit/HTML Reports

Modify playwright.config.js:


reporter: [['html'], ['junit', { outputFile: 'junit.xml' }]]

2. Publishing Reports in GitLab CI

Upload reports as artifacts:


artifacts:
paths:
- playwright-report/

3. Interpreting Test Results in GitLab

View test results under CI/CD > Jobs > Browse Artifacts.

Advanced Configuration

1. Environment Variables for Secrets (e.g., Credentials)

Store sensitive data in GitLab CI/CD Variables (Settings > CI/CD > Variables).

Use in .gitlab-ci.yml:


script:
- echo "TOKEN=${TOKEN}" >> .env

2. Cross-Browser Testing (Chromium, Firefox, WebKit)

Modify playwright.config.js:


projects: [
{ name: 'chromium', use: { browserName: 'chromium' } },
{ name: 'firefox', use: { browserName: 'firefox' } },
{ name: 'webkit', use: { browserName: 'webkit' } }
]

3. Using Services (e.g., Databases, APIs) in CI

Run dependent services using GitLab’s services:


services:
- name: postgres:latest
alias: db

Troubleshooting Common Issues

Browser Launch Failures in CI

  1. Ensure --with-deps is installed (npx playwright install --with-deps).

Dependency Version Conflicts

  1. Use npm ci instead of npm install.

Debugging Pipeline Logs

  1. Check logs under CI/CD > Jobs > View Job Logs.
  2. Run tests locally with DEBUG=pw:browser npx playwright test.

Best Practices

Optimizing Pipeline Speed with Caching

Securing Sensitive Data (Use CI/CD Variables)

Maintaining Test Suites for CI (Regular updates and cleanup)

Conclusion

Automating Playwright tests in GitLab CI ensures reliable test execution.

CI/CD integration helps detect bugs early, improving software quality.

Next Steps: Expand coverage, optimize performance, and explore Playwright’s advanced features.

Additional Resources

  1. Playwright Docs
  2. GitLab CI/CD Docs