Advanced Validation

All Jikken tests include some form of response validation. At its simplest, this validation is just providing an expected status code or response body to be compared against the actual response. However, we may not always know the exact values that will be returned in our test.

This is where advanced validation comes in. This feature allows you to specify the expected response in a more flexible way, based on constraints instead of exact values. You may notice that the format is much the same as in data generation. We use the same principles of specifying a schema of data types and constraints here, but instead of generating a variable, we apply this structure to validating a response body.

Note

In the following examples, response body field names are surrounded by double quotes, as in "field1". This is done only for clarity reasons, as YAML does not require string values to be quoted unless they contain reserved characters.

Applying advanced validation

While we might normally define the response.body field to validate that the response matches a literal JSON value, here we are validating the schema of the response, so we define our specification in the response.bodySchema field instead.

response:
  status: 200
  bodySchema:
    type: Object
    schema:
      "field1": "value"
      "field2":
        type: String
        length: 5

In this example, we are specifying that the response should be an Object, and it should have “field1” with a value of “value”, and “field2” with any 5-character string value. This would correspond to the following JSON object:

{
  "field1": "value",
  "field2": "12345" // or any 5-character string
}

As you can see, we can easily combine exact values with constraints to meet our own specific validation needs.

Let’s look at a few examples of validating data of different types. Though it is technically possible to validate any data type, since we are looking at response bodies here, we’ll focus on objects and lists as the base structures.

Strict mode

By default, strict mode is enabled for response validation. This means if there is any additional data in the response body other than what is specified in your body or bodySchema, the response will be considered invalid and the test will fail.

This behavior can be disabled by setting the response.strict field to false.

Alternatively, you can also specify individual fields to be excluded from validation using the response.ignore field.

Validating objects

Object is naturally going to be the most common data type for a REST API response.

First, let’s expand on the previous example to show a more realistic case.

response:
  bodySchema:
    type: Object
    schema:
      "id":
        type: Integer
        min: 0
      "createdAt":
        type: DateTime
        min: ${NOW}

Here is something we might see if we were creating a new entity via an API.

We use type: Object to specify that the response is an object. Since the entity’s ID will be generated, we can’t know the exact value. Instead, we can validate that the “id” field contains a positive integer.

We’re also checking that the “createdAt” field holds a timestamp which occurs after the time our test session was started. NOW is a built-in global variable that can be leveraged almost anywhere in your test file, as a replacement for the current timestamp at the time of session execution.

What if our ID is a UUID instead of an integer?

response:
  bodySchema:
    type: Object
    schema:
      "id":
        type: String
        length: 36

This version would match any 36-character string. If you wanted to be more restrictive, type: String also supports validating by pattern:

response:
  bodySchema:
    type: Object
    schema:
      "id":
        type: String
        pattern: "(?i)^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$"

Using the pattern constraint will validate that the field matches the specified regular expression. In this case, a string in the form of a UUID, allowing only hexadecimal characters in the format {8}-{4}-{4}-{4}-{12}.

Just like in data generation, schemas can also be nested inside one another, as in the case of a nested object structure.

response:
  bodySchema:
    type: Object
    schema:
      "id":
        type: Integer
      "metrics":
        type: Object
        schema:
          "average":
            type: Float
            min: 0
            max: 100
          "median":
            type: Integer
            min: 0
            max: 100

Here, we have a field called “metrics” which contains a nested object. It has its own schema with its fields defined inside - in this case, a decimal “average” and an integer “median”.

This nesting can continue for as many levels as are needed, to support any data structure.

Validating lists

We can also validate lists in much the same way.

response:
  bodySchema:
    type: List
    length: 5
    schema:
      type: String
      minLength: 1

In this simple case, we’re specifying that the response body should be type: List, and contain exactly 5 non-empty string values.

Realistically, our list would probably contain objects instead:

response:
  bodySchema:
    type: List
    length: 5
    schema:
      type: Object
      schema:
        "id":
          type: Integer
          min: 0
        "createdAt":
          type: DateTime
          min: ${NOW}

This is the same object schema as in our earlier example, but in this case, we are expecting a list of 5 objects in our response. As with nested objects, we can also nest Object and List types together to create more complex structures.

Let’s say we have a set of IDs that we are expecting in the response:

response:
  bodySchema:
    type: List
    length: 3
    schema:
      type: Object
      schema:
        "id":
          type: String
          anyOf: ["this", "that", "other"]
        "createdAt":
          type: DateTime
          min: ${NOW}

This validates that each item’s “id” field is one of the values in the set specified by anyOf.

Note that this will not validate that each ID is different, or that all three values are present. We have plans to expand list validation to be more flexible in the future.

Type reference

This section contains examples with all valid fields for each supported data type, for easy reference. Examples for simple types are shown as fields nested inside an object schema, but these could also be used with lists, or on their own.

For details and descriptions of each field, see the Full Format page.

Constraint compatibility

The anyOf constraint is not compatible with other constraints, as it specifies the exact set of valid values.

The length constraint is not compatible with minLength or maxLength.

Boolean

Validates a boolean value. The alias Bool is also supported.

bodySchema:
  type: Object
  schema:
    "boolean":
      type: Boolean

Integer

Validates an integer value. The alias Int is also supported.

bodySchema:
  type: Object
  schema:
    "integer":
      type: Integer
      min: 0
      max: 100
      anyOf: [1, 2]
      noneOf: [3]

Float

Validates a float value.

bodySchema:
  type: Object
  schema:
    "float":
      type: Float
      min: 0.0
      max: 100.0
      anyOf: [1.2, 3.4]
      noneOf: [5.6]

String

Validates a string value.

bodySchema:
  type: Object
  schema:
    "string":
      type: String
      length: 5
      minLength: 5
      maxLength: 10
      anyOf: ["this", "that"]
      noneOf: ["notthis"]
      pattern: "^[0-9a-z]*$"

Email

Validates an email address.

In addition to the constraints specified, this type will also validate that the value is in a valid format for an email address.

bodySchema:
  type: Object
  schema:
    "email":
      type: Email
      length: 5
      minLength: 5
      maxLength: 10
      anyOf: ["email@example.com", "email@other.com"]
      noneOf: ["not@email.com"]
      pattern: ".*@example.com$"

Date

Validates a date value.

For details on format syntax, see the Full Format page.

bodySchema:
  type: Object
  schema:
    "date":
      type: Date
      min: "2024-01-01"
      max: "2024-12-31"
      anyOf: ["2024-01-01", "2024-02-01"]
      noneOf: ["2024-02-29"]
      format: "%Y-%m-%d"

DateTime

Validates a date-time (or timestamp) value.

For details on format syntax, see the Full Format page.

bodySchema:
  type: Object
  schema:
    "datetime":
      type: DateTime
      min: "2024-01-01 00:00:00"
      max: "2024-12-31 23:59:59"
      anyOf: ["2024-01-01 00:00:00", "2024-02-01 00:00:00"]
      noneOf: ["2024-02-29 00:00:00"]
      format: "%Y-%m-%d %H:%M:%S"

List

Validates a list. The type and constraints for items inside the list are specified in the schema block.

bodySchema:
  type: List
  length: 5
  minLength: 5
  maxLength: 10
  schema:
    type: String
    ...

Object

Validates an object. The fields inside the object and their types and constraints are specified in the schema block.

bodySchema:
  type: Object
  schema:
    "field1": "value"
    "field2":
      type: String
      ...