This document introduces a declarative RDF data model to represent test cases, allowing users to represent some input, execution instructions and then some expected output. While the framework is designed to be of general use, it has built-in test case types for SHACL validation [[!shacl]], SPARQL queries [[!sparql11-query]] and SPARQL functions.

Note that this document covers both the general design of the test case framework and also illustrates specific tool support as part of TopBraid platform including the TopBraid Composer Free Edition. The TopBraid binding should only be understood as one possible implementation and is in no way limiting the general applicability of the framework. Parts of the framework have also been published as part of the open-source TopBraid SHACL API. The document has been updated for TopBraid release 5.2.

The button below can be used to show or hide the TopBraid-specific paragraphs:

General Concepts

The DASH Test Cases vocabulary provides a declarative data model for representing and sharing information about test cases. Test Cases represented using this vocabulary can be executed by tools to verify that a given test scenario still applies as originally designed. The vocabulary it intentionally minimalistic and kept simple.

Test Case Graphs and Files

Test cases are represented in RDF graphs. When published, these RDF graphs SHOULD be stored in a commonly used RDF serialization format such as [[!turtle]]. Files containing test cases should contain the partial name .test. so that tools can automatically recognize them when walking a folder structure. A typical file name for a test case is

	ExampleTests.test.ttl

In order to produce a new test case file with TopBraid Composer, use File > New > RDF Test Cases File. In order to execute a single test case or all test cases, use the following window, accessible from Window > Show View > Test Cases:

The Base Class dash:TestCase

The DASH namespace (prefix: dash, namespace: http://datashapes.org/dash#) includes a class dash:TestCase that serves as the (abstract) base class of all types of test cases supported by the framework. Extensions may subclass this base class. The DASH namespace includes a collection of built-in subclasses of dash:TestCase for frequently needed kinds of tests, as described in the following sections.

The Query Graph

Test cases typically need to be evaluated against a default query graph. In the current design of the framework, queries simply run against the graph that also contains the definition of the test case itself, plus its (transitive) owl:imports. While this design may in principle lead to situations where the test case itself may interfere with the evaluation results, this case is considered too rare to be of practical relevance. So in the current design, we went for the simplest possible approach. If this becomes an issue, then future versions may introduce additional properties to link to specific query graphs.

SPARQL Functions Test Cases

The class dash:FunctionTestCase can be used to represent test cases to repeatedly evaluate a given SPARQL function (or more general: SPARQL expression). Instances of this type of test cases need to specify a SPARQL expression and an RDF node representing the expected result. For the test case to succeed, the provided SPARQL expression is evaluated against the query graph and needs to produce the expected result.

The following example illustrates a test case that verifies that a given SPARQL function strlen returns 5 for the parameter "Hello".

ex:strlenTest
    rdf:type dash:FunctionTestCase ;
    dash:expectedResult 5 ;
    dash:expression "strlen(\"Hello\")" .

TopBraid includes a convenience feature to quickly produce such test cases from the SPARQL view. Select the expression that you want to turn into a test case, right-click on it and select Turn into dash:FunctionTestCase...:

SPARQL Query Test Cases

The class dash:QueryTestCase can be used to represent test cases to repeatedly evaluate a given SPARQL SELECT query. Instances of this type of test cases need to specify a SPARQL query and a string-serialization of a SPARQL Result in JSON [[!sparql11-results-json]]. For the test case to succeed, the provided SPARQL query is evaluated against the query graph and needs to produce exactly the same result set. Note that in order to produce reliable results, the query MUST include an ORDER BY clause.

The following example illustrates a test case that verifies that a given SPARQL query still produces the given result set.

ex:queryTest
  rdf:type dash:QueryTestCase ;
  dash:expectedResult """{
  \"head\": {
    \"vars\": [ \"label\" ]
  } ,
  \"results\": {
    \"bindings\": [
      {
        \"label\": { \"type\": \"literal\" , \"value\": \"Nothing\" }
      } ,
      {
        \"label\": { \"type\": \"literal\" , \"value\": \"Thing\" }
      }
    ]
  }
}
""" ;
  sh:select """
      SELECT ?label
      WHERE {
          ?class a owl:Class .
          ?class rdfs:label ?label .
      }
      ORDER BY ?label
      """ .

TopBraid includes a convenience feature to quickly produce such test cases from the SPARQL view. Enter and execute the query until you are happy with the result, then select Turn into Test Case... from the context menu of the SPARQL view:

SHACL Validation Test Cases

The class dash:GraphValidationTestCase can be used to represent test cases to repeatedly validate the query graph against a SHACL [[!shacl]] shapes graph. Instances of this type of test cases need to specify zero or more instances of the SHACL validation results classes such as sh:ValidationResult. If the set of provided validation results is empty, the validation is expected to return no violations. For the test case to succeed, the query graph is used as the data graph and the shapes graph of a SHACL validation, and the actual validation results are compared with those stored in the test case.

The following example illustrates a graph validation test case that is expected to return two violations. The example graph also includes the shapes and instances that are part of the validation.

ex:GraphValidationTestCase
  rdf:type dash:GraphValidationTestCase ;
  dash:expectedResult [
      rdf:type sh:ValidationResult ;
      sh:focusNode ex:InvalidResource1 ;
      sh:path ex:dateProperty ;
      sh:severity sh:Violation ;
      sh:sourceConstraintComponent sh:DatatypeConstraintComponent ;
      sh:value "2011-01-01"^^xsd:dateTime ;
    ] ;
  dash:expectedResult [
      rdf:type sh:ValidationResult ;
      sh:focusNode ex:InvalidResource1 ;
      sh:path ex:integerProperty ;
      sh:severity sh:Violation ;
      sh:sourceConstraintComponent sh:DatatypeConstraintComponent ;
      sh:value 11.1 ;
    ] ;
.
ex:InvalidResource1
  rdf:type ex:TestShape ;
  ex:dateProperty "2011-01-01"^^xsd:dateTime ;
  ex:integerProperty 11.1 ;
.
ex:TestShape
  rdf:type rdfs:Class ;
  rdf:type sh:NodeShape ;
  rdfs:label "Test shape" ;
  sh:property [
      sh:path ex:dateProperty ;
      rdfs:label "date property" ;
      sh:datatype xsd:date ;
    ] ;
  sh:property [
      sh:path ex:integerProperty ;
      rdfs:label "integer property" ;
      sh:datatype xsd:integer ;
    ] ;
.
ex:ValidResource
  rdf:type ex:TestShape ;
  ex:dateProperty "2014-09-01"^^xsd:date ;
  ex:integerProperty 0 ;
  ex:integerProperty 1234 ;
  rdfs:label "Valid resource" ;
.

TopBraid includes a convenience feature to quickly produce such test cases from the SHACL validation view. Simply run the validation (with the green button) and, when satisfied with the results, use the Create a test case... button from the tool bar: