GitHub Actions for Dynamic Cross-Platform Testing

Automate cross-platform testing with GitHub Actions, using dynamic matrices for efficient, targeted tests.

Ensuring consistent application performance across multiple platforms is a cornerstone of quality software delivery. In this post, I'll share my approach for using GitHub Actions to automate testing across a variety of platforms, drawing from a real use case at my workplace.

Our solution utilizes GitHub Actions, a CI/CD service that enables automation of build, test, and deployment workflows directly within GitHub. The highlight of the approach is the dynamic matrix strategy, which handles both scheduled runs and on-demand tests across specified platforms.

Scheduled and On-demand Testing

In this example, the workflow is being triggered automatically at 22:00 every day for nightly tests. Additionally, we enabled manual triggers for specific platforms, allowing for targeted testing as needed. This flexibility is crucial for maintaining a fast-paced development cycle without compromising on quality.

on:
  schedule:
    - cron: '0 22 * * *'
  workflow_dispatch:
    inputs:
      platform:
        description: 'Choose platform to run the test on'
        required: true
        type: choice
        options:
          - webos
          - tizen
          - playstation4
          - playstation5
          - xbox

This configuration leverages the schedule and workflow_dispatch events in GitHub Actions. The cron syntax schedules automated test runs, while workflow_dispatch allows for manual execution with platform-specific parameters.

Dynamic Test Matrix Configuration

The core of our workflow dynamically adjusts the test matrix based on the context of the trigger—whether it's a scheduled job covering all platforms or a manual job focusing on a specific platform.

jobs:
  setup-test:
    runs-on: ubuntu-latest
    outputs:
      platform: ${{ steps.set-matrix.outputs.platform }}
    steps:
      - id: set-matrix
        run: |
          if [ "${{ github.event_name }}" = "schedule" ]; then
            echo 'platform=["webos", "tizen", "playstation4", "playstation5", "xbox"]' >> $GITHUB_ENV
          else
            echo 'platform=["${{ github.event.inputs.platform }}"]' >> $GITHUB_ENV

In this snippet, we employ conditional logic to populate the platform variable differently based on the trigger type. For scheduled runs, we test across all platforms, while manual triggers allow for focused testing on a selected platform. This strategy ensures that resources are allocated efficiently, focusing our testing efforts where they are most needed. Of course, you could always do that with an external script or with a composite action, but this is a simple and effective way to achieve the same result.

Executing the Tests with Dynamic Configuration

Our approach to executing tests in this workflow benefits significantly from the dynamic nature of GitHub Actions, especially when it comes to handling multiple platforms which can be quite different from each other. This flexibility is further extended into our application's package.json file, where we define specific scripts for our testing needs. The package.json plays a crucial role in integrating the GitHub environment variables directly into our test execution process.

Without getting into the tests code, you can imagine that each platform has its own specific tests and configurations, and that's why we need to run the tests separately for each platform. This is where the dynamic matrix comes into play.

- name: Run tests
  working-directory: ./src/lr-launcher-e2e
  env:
    npm_config_env: ${{ matrix.platform }}
  run: npm test

Here, the npm test command is executed within the context of each platform specified by the dynamic matrix. This ensures that our E2E tests are relevant and efficient, automatically adapting to the complexities of multi-platform support.

Here's a look at the relevant part of our package.json file:

{
  "name": "lr-launcher",
  "version": "1.0.0",
  "scripts": {
    "pre-test": "node ./utils/setupTest.js --",
    "test": "mocha \"tests/*.test.js\" \"tests/$npm_config_env/*.test.js\" --no-timeouts --retries 1 --exit"
  },
  "dependencies": {
    "axios": "^1.6.2",
    "mocha": "^10.2.0",
  }
}

This script utilizes the npm_config_env environment variable, which we set dynamically based on the selected platform in our GitHub Actions workflow. By incorporating $npm_config_env into the mocha command, we ensure that the test runner picks up not only the general tests from tests/.test.js but also platform-specific tests located in directories named after each platform (e.g., tests/webos/.test.js).

This setup allows us to maintain a clear and organized test structure where platform-agnostic tests are separated from those that are platform-specific. It ensures that when our GitHub Actions workflow runs, it only executes the relevant tests for the specified platform, making our testing process both efficient and comprehensive.



Tags:

TestingDevOpsJavaScriptNodeJS

Related Articles

S3 Bucket Redirect URL With Terraform

Read More

Integrating Azure AD Authentication in Your Next.js App Using MSAL

Read More

Scheduling Lambdas Using AWS EventBridge and Terraform

Read More

Why Makefile is One of My Essentials Tools in Every Software Project

Read More