Full Format

Jikken tests are defined in a YAML/JSON format. Below is an example of the full structure with all possible fields. This format is subject to change over time as we add more capabilities to Jikken.

You can find some basic example tests in our Quick Start Guide or Example Tests sections.

name:
id:
description:
project:
env:
tags:
requires:
disabled:
iterate:
setup:
  request:
    method:
    url:
    params:
    - param:
      value:
    headers:
    - header:
      value:
    body:
  response:
    status:
    headers:
    - header:
      value:
    body:
    ignore:
    -
    extract:
    - name:
      field:
request:
  method:
  url:
  params:
  - param:
    value:
  headers:
  - header:
    value:
  body:
compare:
  method:
  url:
  params:
  - param:
    value:
  addParams:
  - param:
    value:
  ignoreParams:
  -
  headers:
  - header:
    value:
  addHeaders:
  - header:
    value:
  ignoreHeaders:
  -
  body:
response:
  status:
  headers:
  - header:
    value:
  body:
  ignore:
  -
  extract:
  - name:
    field:
stages:
- name:
  request:
    method:
    url:
    params:
    - param:
      value:
    headers:
    - header:
      value:
    body:
  compare:
    method:
    url:
    params:
    - param:
      value:
    addParams:
    - param:
      value:
    ignoreParams:
    -
    headers:
    - header:
      value:
    addHeaders:
    - header:
      value:
    ignoreHeaders:
    -
    body:
  response:
    status:
    headers:
    - header:
      value:
    body:
    ignore:
    -
    extract:
    - name:
      field:
  variables:
  - name:
    type:
    value:
    modifier:
      operation:
      value:
      unit:
    format:
  delay:
cleanup:
  onsuccess:
    method:
    url:
    params:
    - param:
      value:
    headers:
    - header:
      value:
    body:
  onfailure:
    method:
    url:
    params:
    - param:
      value:
    headers:
    - header:
      value:
    body:
  always:
    method:
    url:
    params:
    - param:
      value:
    headers:
    - header:
      value:
    body:
variables:
- name:
  type:
  value:
  file:
  modifier:
    operation:
    value:
    unit:
  format:

Field Definitions

Test Definition

Field	Example	Description
name	`User Login`	(Optional) A name used for providing a user-friendly label in test execution output and the dashboard. If this field is not present, the test will be referred to by its number in the execution order, or its `id`.
id	`7431857f-7e00-40b7-ac2c-077d230dff1c`	(Optional) A unique identifier for tracking a test across many runs, and for referencing a test in dependent tests. This identifier can be any string, but we recommend using a UUID. When creating tests via the `jk new` command, an `id` will be generated automatically. If this field is not present, a hash of the file’s contents will serve as the unique identifier, so any change to the test will result in a new `id`. The `id` changing over time may impact tracking the test execution history in the Jikken webapp. It is also required to provide this field if you want to depend on the execution of this test in another test (as the `id` field is what links them).
description		(Optional) A description for the test. This can be visualized in the test dashboard but it is not leveraged by the CLI test executor.
project		(Optional) The project to be associated with the test. This is used for organizing, filtering, and monitoring test runs in the dashboard.
env	`prod`	(Optional) The environment to be associated with the test. This is used for organizing, filtering, and monitoring test runs in the dashboard.
tags	`regression` `smoke` `login`	(Optional) A list of tags that apply to this test. Tags are used for selecting tests to be run, and for filtering in the dashboard. Tags are provided as a white-space-delimited string.
requires	`7431857f-7e00-40b7-ac2c-077d230dff1c`	(Optional) The `id` value of another test that is required to run before this one. This will enforce an order of execution where the required tests are executed prior to their dependents. This is useful for variable extraction, where a value returned from one request is injected into a future request in a different test file. Currently this only supports a single value, but we plan to support more robust dependency graphing of test execution.
disabled	`true`	(Optional) A mechanism for specifying that a test should not be executed. By default, this is false. If `true` the test will not execute even if the filters match when running `jk run`
iterate	`5`	(Optional) The number of times this test should be repeated per run. When variables are defined for the test, based on the generative nature of those variables, each iteration can pass in different values. This is useful if you want to test the same API multiple times with different parameters - you can do so with a single test file.
setup		(Optional) The setup structure defines a special stage that occurs prior to all other requests/stages. If the setup stage fails, subsequent stages and request definitions will not execute. See the Setup section for details.
request		The request structure defines the API request to be made, including the URL and parameters. See the Request section for details.
compare		(Optional) The compare structure defines an additional API request to be made, for which the response will be compared against the result of `request`. This is useful for validating two different environments or versions of an API. You can define `request` to call an API in a QA/Staging environment and `compare` to call that same API in Production, for example, and then compare the results to see if there are any unintended changes in the response. See the Compare section for details.
response		(Optional) The expected response values to validate. Though the `compare` and `response` fields are both optional, if at least one is not defined, then no validation is performed and nothing is being tested. See the Response section for details.
stages		(Optional) Stages allow you to define multiple steps to be performed during a single test. Each stage can extract data from a response, which can then be injected into future requests. See the Stage section for details.
cleanup		(Optional) The cleanup structure defines a special stage that occurs after all other requests/stages. The cleanup stage can run only on success, only on failure, or always. See the Cleanup section for details.
variables		(Optional) A list of locally-defined variables. These variables allow you to generate and embed values into requests. Variable embedding is currently supported by the following fields: `Request.Url`, `Request.Headers`, `Request.Params`, `Compare.Url`, `Compare.Headers`, `Compare.AddHeaders`, `Compare.Params`, and `Compare.AddParams`. We plan to expand the scope of what variables can do, as well as where they can be injected. See the Variables section for details.

Request

Field	Example	Description
method	`Post`	(Optional) The HTTP method for the request. Supported values are: `Get`, `Post`, `Put`, `Patch`, and `Delete`. If this field is not present, it is defaulted to `Get`.
url	`http://api.myurl.com`	The URL to be called for the request. This field supports injection of both global and local variables.
params		(Optional) A list of query parameters to add to the URL request. See the Params section for details.
headers		(Optional) A list of HTTP headers to send in the request. See the Headers section for details.
body	`{ "test": "response" }`	(Optional) The JSON body to send in the request. This field supports literal JSON and variable embedding.

Compare

Field	Example	Description
method	`Post`	(Optional) The HTTP method for the request. Supported values are: `Get`, `Post`, `Put`, `Patch`, and `Delete`. If this field is not present, it is defaulted to `Get`.
url	`http://api.myurl.com`	The URL to be called for the request. This field supports injection of both global and local variables.
params		(Optional) A list of query parameters to add to the URL request. For the `compare` structure, if this field is not provided, it will be defaulted to the same value defined in the base `request.params` field. This simplifies the common case of testing different API versions by removing the need to define parameters twice. If this field is provided, the base `request` params will not be included. See the Params section for details.
addParams		(Optional) A list of additional query parameters to add to the URL request. This is used to add more query parameters in addition to those specified in the `request`. See the Params section for details.
ignoreParams	`- foo`	(Optional) A list of query parameters to be excluded from the `compare` request. This field allows you to specify parameters defined in the base `request.params` object that you do not wish to include in the `compare` request.
headers		(Optional) A list of HTTP headers to send in the request. For the `compare` structure, if this field is not provided, it will be defaulted to the same value defined in the base `request.headers` field. This simplifies the common case of testing different API versions by removing the need to define headers twice. If this field is provided, the base `request` headers will not be included. See the Headers section for details.
addHeaders		(Optional) A list of additional HTTP headers to send in the request. This is used to add more headers in addition to those specified in the `request`. See the Headers section for details.
ignoreHeaders	`- Authorization`	(Optional) A list of HTTP headers to be excluded from the `compare` request. This field allows you to specify headers defined in the base `request.headers` object that you do not wish to include in the `compare` request.
body	`{ "test": "response" }`	(Optional) The JSON body to send in the request. This field supports literal JSON and variable embedding.

Response

Field	Example	Description
status	`200`	(Optional) The expected HTTP status code. We recommend defining this in most cases, as it will be a great indicator of unforeseen problems such as authorization issues, gateway/proxy issues, etc.
headers		(Optional) A list of expected HTTP headers. Any header defined here must match the headers returned for the request and compare calls. The request may have additional headers which are not defined here, and not be considered a failure.
body	`{ "test": "response" }`	(Optional) The expected JSON body. This field supports literal JSON and variable embedding.
ignore	`- data.users.lastLogin`	(Optional) A list of JSON fields to ignore when performing `body` validation. This can be useful when there are runtime-dependent fields in the response JSON, for example, if an API has a timestamp indicating the time of the request, This allows you to prune out sections of JSON in the received response prior to comparing the data between the `request` and `compare` responses and/or the `response.body` definition. This notation supports traversing both arrays and objects, so if a field is an array, the ignored field will be pruned from all objects in the array.
extract		(Optional) A list of variable extraction terms. This field allows you to extract fields from a JSON response and store them into a variable, which can then be injected into subsequent tests.
extract.name	`authToken`	The name of the variable to store the extracted field into.
extract.field	`token`	The JSON field to be extracted. In this case, the expected response structure is an object with a first-level field called `token`. This path does support nested path traversal similar to the `response.ignore` definitions, so you could extract a field at `data.user.token` if the structure had nested objects that contained the field you wish to extract.

Variables

Field	Example	Description
name	`foo`	The name of the variable. This is used the reference the variable in the places where you want it to be injected. Variable names must be unique or they will overwrite each other. If a global variable is defined with a given name and a local variable has the same name, the local variable will overwrite the global one.
type	`Int`	The data type of the variable. Currently valid values are `Int`, `String`, `Date`, and `Datetime`. The data type is used when handling value generation and modifier operations. You can use `String` to cover mixed data types when using sequence values (arrays). The default value is `String`. (NOTE:* `Datetime` is a placeholder and is not currently supported, but will be added in the near future.)*
value	`[1, 5, 23143]`	(Optional) The value of the variable. This supports single values and JSON sequences (arrays), as defined by a comma-delimited list surrounded by `[]`. When a sequence is present, each iteration will grab the next value in the sequence. In this example, the first iteration would inject the value `1`, the second iteration would inject the value `5`, and the third iteration would inject `23143`. If you iterated a fourth time, it would start over and repeat the values. This allows you to have varying-length sequences for different variables and cycle combinations between them as you iterate. Either `value` or `file` must be provided.
file		(Optional) The absolute or relative path of a file whose contents should be stored in the variable. Either `value` or `file` must be provided.
modifier		(Optional) The modifier object allows you to define simple operations to be performed against the variable value. This is useful when leveraging extracted variables or globals. This structure is currently only supported for `Date` types, but we plan to expand this capability with a number of operations for other data types. This allows a test to start with the value for `${TODAY}` and then to add/subtract a provided number of days, weeks, or months from the starting date.
modifier.operation	`subtract`	The operation to perform. Currently this supports `add` and `subtract`.
modifier.value	`3`	The amount to modify the variable’s value. Currently this supports unsigned integers (positive whole numbers).
modifier.unit	`days`	The unit to be applied to `value`. This currently supports `days`, `weeks`, and `months`.
format	`%Y-%m-%d`	(Optional) The format field is used to define a string formatter pattern when generating the values. (NOTE:* This field is a placeholder and is not currently used. We are looking at options of different ways we want to support formating various data types.)*

Setup

Field	Description
request	The API request to be made prior to all other stages. See the Request section for details.
response	(Optional) The expected response values to validate. If the setup stage fails, no other stages will be executed. See the Response section for details.

Stage

Field	Description
name	(Optional) The name for the stage. This is used in console output and dashboard messaging.
request	The API request to be made. See the Request section for details.
compare	(Optional) The comparison request to be made. See the Compare section for details.
response	(Optional) The expected response values to validate. See the Response section for details.
variables	(Optional) A list of variable definitions. See the Variables section for details.
delay	(Optional) The millisecond duration to pause after executing the stage. If not provided, there is no delay.

Cleanup

Field	Description
onsuccess	(Optional) An API request to be made only when the test has succeeded. See the Request section for details.
onfailure	(Optional) An API request to be made only when the test has failed. See the Request section for details.
always	(Optional) An API request to be made every time this test executes. This runs whether the test passes or fails. If you define `onsuccess` and/or `onfailure` as well, this request will trigger after the `onsuccess`/`onfailure` requests execute. See the Request section for details.

Params

Field	Example	Description
param	`foo`	The query field name.
value	`bar`	The query parameter value.

With the provided example, the request URL would become <url>?foo=bar.

Headers

Field	Example	Description
header	`Authorization`	The HTTP header key.
value	`eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9`	The HTTP header value.