TestRail Integration Plugin

by railflow

sourcereadmeimage

TestRail integration plugin by Railflow (https://railflow.io) allows users easily integrate their Harness pipelines with TestRail and automatically create Test Suites, Cases, Test Plans, Test Runs and Test Results, etc.. in TestRail.

Example

kind: pipeline
type: docker
name: default

steps:
  - name: railflow
    image: railflow/railflow:latest
    settings:
      license_key: XXXXX-XXXXX-XXXXX-XXXXX
      url: https://testrail
      username: user@company.com
      password: s3cret
      project: My Project
      test_path: Railflow/Harness
      format: JUnit
      report_files:
        - target/surefire-reports/*.xml
      search_mode: path
      upload_mode: 1

Properties

• license_key

  stringrequired

  Railflow license key. Either license key or license file. It can be obtained from https://railflow.io.

  Secret recommended

  Default: none

• license_file

  stringrequired

  License file path or URL. Either license key or license file. It can be obtained from https://railflow.io.

  Default: none

• url

  stringrequired

  TestRail instance URL.

  Default: none

• username

  stringrequired

  TestRail username.

  Secret recommended

  Default: none

• password

  stringrequired

  TestRail password or API Key.

  Secret recommended

  Default: none

• project

  stringrequired

  TestRail project name.

  Default: none

• test_path

  stringrequired

  TestRail test cases path.

  Default: none

• format

  stringrequired

  Report format: JUnit, JUnit-Steps, TestNg, TestNg-Steps, Cucumber, NUnit, NUnit-SpecFlow, Allure, Robot, TRX, xUnit, PyTest-Railflow, Playwright (case insensitive)

  Default: none

• report_files

  arrayrequired

  The file path(s) to the test report file(s) generated during the build. Ant-style patterns such as `**/surefire-reports/*.xml` can be used.

  Default: none

• search_mode

  stringrequired

  Specifies the test case lookup algorithm. Allowed values are `name` and `path`. `name` - search for test case matching the name within the entire test suite. `path` - search for test case matching the name within the specified `test_path`.

  Default: none

• proxy

  stringoptional

  HTTP or SOCKS proxy configuration.

  Default: none

• timeout

  numberoptional

  Upload timeout (seconds).

  Default: none

• test_run

  stringoptional

  TestRail test run name.

  Default: none

• test_plan

  stringoptional

  TestRail test plan name.

  Default: none

• milestone_path

  stringoptional

  TestRail milestone path.

  Default: none

• case_fields

  arrayoptional

  TestRail test case custom fields. The format is `[Field label]=[value]` pairs.

  Default: none

• result_fields

  arrayoptional

  TestRail test result custom fields. The format is `[Field label]=[value]` pairs.

  Default: none

• assign

  arrayoptional

  Smart Test Failure Assignment. Comma-separated list of TestRail users (email IDs). Railflow will assign failures based on a round robin algorithm.

  Default: none

• assign_file

  stringoptional

  Smart Test Failure Assignment. File path containing list of TestRail users (email IDs). Each user email ID should go with the new line.

  Default: none

• config_names

  arrayoptional

  TestRail test plan configuration options.<br/>Configuration format is: [config_group_name]/[config_name].

  Default: none

• close_run

  booleanoptional

  If Railflow should close the corresponding run after uploading test results.

  Default: false

• close_plan

  booleanoptional

  If Railflow should close the corresponding plan after uploading test results.

  Default: false

• disable_grouping

  booleanoptional

  If Railflow should ignore report structure and just upload all tests into a folder which is set by test-path parameter.

  Default: false

• template_name

  stringoptional

  The name of a template to use in TestRail. If it is not set, 'Test Case (Steps)' or the default one will be used.

  Default: none

• case_type

  stringoptional

  The name of a type for cases.

  Default: none

• case_priority

  stringoptional

  The name of a priority for cases.

  Default: none

• thread_number

  numberoptional

  The number of concurrent threads for exporting data. Default is 4.

  Default: 4

• upload_mode

  numberoptional

  Upload mode. Available values are: 0 - create new test cases and do not overwrite existing ones; 1 - create new cases and overwrite existing ones; 2 - do not create new cases and overwrite existing ones; 3 - do not create new cases and do not overwrite existing ones

  Default: 0

• case_search_field

  stringoptional

  The name of the case field in TestRail which will be using for searching for existing test cases instead of test case title.

  Default: none

• disable_stats

  booleanoptional

  If Railflow should disable collecting usage and error logs.

  Default: false

• fully_qualified_test_name

  booleanoptional

  If Railflow should allow exporting test cases using qualified name as the case name.

  Default: false

• untested_status

  stringoptional

  The name of the status to use in TestRail for untested/skipped tests.

  Default: none

• attachment_max_size

  stringoptional

  Maximum size of an attachment. Can be set in Gb, Mb, Kb or b. If the unit is not provided considered as Mb. E.g. 1Gb, 2, 20Kb, 3Mb.

  Default: none

• attachment_type_whitelist

  arrayoptional

  The list of file extensions which are allowed to upload.

  Default: none

• attachment_type_blacklist

  arrayoptional

  The list of file extensions which are not allowed to upload.

  Default: none

• no_run

  booleanoptional

  Do not create or update Test Run in TestRail.

  Default: false

• tags_field_name

  stringoptional

  The name of a test case field which will be holding tags from the report file if any. E.g. `Cucumber Tags`

  Default: none