JFCUnit with XML

This feature allows the developer to write XML code to contain the bulk of the test suite definition. Much of the test generation process is automated via recording to the XML. Not all java coding requirements have been eliminated via this implementation. It was not the goal to completely replace the java coding, but provide a efficient means of defining and manipulating the test cases to drive GUI components testing. A iterative pattern can be used when recording via the XML, so that new code can be inserted easily. Modified code remains unchanged.

The Foundation

In order to develop the XML interfaces implemented in the JFCUnit classes, a base XML JUnit framework was developed to provide building blocks which could be re-used by other projects.

The framework provides for definitions of properties via XML. These properties are held in a scoping fashion to allow for properties to be used between test cases. Each test case defined in the XML has access to the properties defined in the parent test case and grandparent test cases. Parent level can be implicitly accessed via specifying the name of the property only or explicitly accessed by prefixing the name with the appropriate number of "../" parent references. Properties can be substituted in attribute values by using the following convention "${name}" where "name" is the name of the property to be substituted. Property substitutions can be combined to create composite attribute values.

Procedures can also be defined at the suite and test case levels. These procedures follow the same scoping rules as the properties. The parent reference can also be used in the definition of a procedure to place the procedure at a higher level in the scope. This allows for the generation of procedure libraries.

Two procedure names are reserved for integration of "setUp" for JUnit's TestCase.setUp() method and "tearDown" for TestCase.tearDown() method.

The XML interface is easily extendable by the test developers. A custom tag handler can be developed to integrate Java code back into the XML. These custom tag handlers can be registered/unregistered via the XML script definition. Permanent tag handlers defined by the JFC XML framework can be overridden and restored.

Example foundation concepts in XML:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="My Test Cases">

    <!-- Define a doc tag handler to allow the embed doc into the test case. -->
    <taghandlers action="add" tagname="doc" classname="junit.extensions.xml.elements.NoOpTagHandler"/>

    <!-- A Suite level procedure -->
    <procedure name="setUp">
        <!-- Show popup with "Starting Test" -->
        <echo message="Starting Test" mode="dialog/>
    </procedure>
    <procedure name="tearDown">
        <!-- Show popup with "Test Complete" -->
        <echo message="Test Complete" mode="dialog/>
    </procedure>

    <!-- Load a procedure library -->
    <file name="library.xml"/>
    <suite name="My Internal test suite">

        <!-- a procedure defined in a internal suite -->
        <procedure name="Another procedure">
           <!-- return a value to the calling test case -->
           <!-- to the value of the result property     -->
           <property name="../suitetest" value="${result}"/>
           ...
        </procedure>

        <test name="Test Procedure">
           <procedure name="setUp">
              <!-- one ../ takes us to the test case scope  -->
              <!-- two ../ takes us to the test suite scope -->
              <procedure call="../../setUp"/>
           </procedure>
           <procedure name="TEST">
               ...
           </procedure>

           <doc>
               This is a custom tag handler and should be
               ignored since the doc is mapped to the NoOp
               (No Operation) tag handler.
           </doc>
           <!-- expample of calling and passing a property -->
           <procedure call="TEST" result="true"/>
           <procedure call="Another procedure"/>
        </test>

    </suite>
</suite>

Example of a procedure library:

<?xml version="1.0" encoding="UTF-8"?>
<suite name="Library">

    <!-- Define a custom tag handler -->
    <!-- Tag handlers are global in nature. Thus they are not scoped. -->
    <taghandlers action="add" tagname="mytagHandler" classname="customPackage.CustomTagHandler"/>

    <!-- A Library procedure -->
    <!-- NOTE: the subtlety of using the ../ to place the Login procedure -->
    <!--       into the parent test suite scope. Without this the         -->
    <!--       procedure would only be visible to tests and suites        -->
    <!--       contained in this XML.                                     -->
    <procedure name="../Login">
      <mytagHandler .../>
       ...
    </procedure>

    <!-- A Library procedure -->
    <procedure name="../Logout">
       ...
    </procedure>

</suite>

Generic XML language elements

Several language elements have been created to help support the generation of robust test scripts in XML.

Choose/When/Otherwise - A choose when otherwise construct has been adopted from the XLS transforms to allow for branching logic within the XML test cases.

... Example...

Echo - Echo allows for text to be sent to stdout/stderr or a dialog. The echo can also post a confirm dialog which will assert if No is selected. The dialog mode can be used to synchronize the execution of the test case.

Noop - The noop tag handler can be used to turn off other tag handlers for testing.

Assert/Fail - Assert certain properties.

Evaluate - Evaluate a null parameter method and store the value into the a property..

Save - Save the XML to a file.

Foreach - for each list item or table cell.

indexof - locate a cell or item in a table or list.

Dump - Dump the swing containment hierachy.

AssertHasFocus - Assert that the component referenced has focus.

AssertEnabled - Asserts that the componenet referenced is enabled.

AssertTableCellContains - Assert the contents of a table cell

AssertTextFieldContains - Assert the contents of a text field.

JFCUnit Specific Language Elements

Click - Enter click(s)
Find - Find a component/frame/dialog
Drag - create a drag operation.
Record - Record AWT Events into the XML.
Please contribute tag handlers ..

Starting Testing Quickly with XMLRoot

A generic tool has also been generated which allows for the running of a application, however this tool may not cover all circumstances. The junit.extensions.jfcunit.tools.XMLRoot can be used to start a application and fire off a test case defined in XML as the application above does. It will generate a stubbed out version of a XML file to get you quickly started into test case generation. It can take arguments either from the command line, system properties or via a API. Below is a example of how to get started with the SwingSet

# to start recording to a new testcases.xml
java -Djfcunit.xmlroot.record=true \
     -Djfcunit.xmlroot.classname=demo.SwingSet\
     -Djfcunit.xmlroot.testsuite=testcases.xml\
     -classpath jfcunit.jar;SwingSet.jar;jarkarta-regexp-1.2.jar;junit.jar\
     junit.swingui.TestRunner junit.extensions.jfcunit.tools.XMLRoot

# to re-run recordings

java -Djfcunit.xmlroot.classname=demo.SwingSet\
     -Djfcunit.xmlroot.testsuite=testcases.xml\
     -classpath jfcunit.jar;SwingSet.jar;jarkarta-regexp-1.2.jar;junit.jar\
     junit.swingui.TestRunner junit.extensions.jfcunit.tools.XMLRoot

Note: A limitation to the current XML is that the individual test cases cannot be executed standalone.

Java Coding Requirements

What remains to code in Java, is how to bridge the JUnit test into the test cases defined by the XML. Below we have illustrated code on how to derive a XMLTestCase, to load and run testSwingSet.xml. This Test case can be used to allow for the embedding of the XMLTestSuite into a much larger automated test which may have both XML defined tests and Java defined tests.

public class TestXMLSwingSet extends XMLTestSuite {

    public static final String DOCUMENT_FACTORY =
         "javax.xml.parsers.DocumentBuilderFactory";

    public TestXMLSwingSet() throws Exception {
        super("testSwingSet.xml",
              XMLUtil.readFileFromClassContext(
                 TestXMLSwingSet.class, "testSwingSet.xml"));
        SwingSet.main(new String[] {});
    }

    public static Test suite() throws Exception {
        return new TestXMLSwingSet();
    }

    public static void main(String[] args) throws Exception {
        if (System.getProperty(DOCUMENT_FACTORY) == null) {
            System.setProperty(DOCUMENT_FACTORY,
            "org.apache.xerces.jaxp.DocumentBuilderFactoryImpl");
        }
        TestRunner.run((Test) TestXMLSwingSet.suite());
    }
}

Creating Custom Tag Handler Example

For the example, lets consider a stopwatch TagHandler that measures the performance of a set of calls. Step one is to define the XML, here we will take something quite simple.

The following code illustrates the construction of the Tag Handler. The AbstractTagHandler gives us a toolkit for processing the tags. This tool kit has methods for retrieving the values form the attributes, and validating that specific attributes exist.

The API must have a constructor with the arguments Element and IXMLTestCase to enable the attribute to work with the TagHandler.properties file discussed later. The handler is based around two methods validateElement() and processElement(). The validateElement() method should validate that all of the necessary information is available within the message. The processElement actually takes the action defined. In this example, our processElement either adds a marker with the ID to the IXMLTestCase or it retrieves the mark referenced and performs a comparison against the current time.

package junit.extensions.xml.elements;

import org.w3c.dom.Element;
import junit.extensions.xml.XMLConstants;
import junit.extensions.xml.IXMLTestCase;
import junit.extensions.xml.XMLException;

/**
* This is tag handler can measure the milliseconds within a
* test case.
*
* <h3>Description</h3>
* <p>
* Mark the start of the process to be timed with a mark action.
* <stopwatch action="mark"/>
*
* Then afterward assert that the duration in milliseconds.
* <stopwatch action="lessthan" value="8000"/>
*
* @author Kevin Wilson
*/
public class StopWatchTagHandler
extends AbstractTagHandler
implements XMLConstants {

/**
* constructor
* @param element Element to be processed.
* @param testCase containing test case.
*/
public StopWatchTagHandler(Element element, IXMLTestCase testCase) {
super(element, testCase);
}

/**
* Validate that the element is properly configured.
* @throws XMLException Exception may be thrown if there
* are missing elements.
*/
public void validateElement() throws XMLException {
// check the element tag name
checkElementTagName(STOPWATCH);

// reqd attribute: at least one of refid or id is present
checkAtLeastOneRequiredAttribute(getElement(), new String[] {REFID, ID});

// action is a required attribute
checkRequiredAttribute(ACTION);

String action = getString(ACTION);

if (!action.equals(MARK)) {
    // if action is not a mark, then a value is required.
    checkRequiredAttribute(getElement(), VALUE);
}
}

/**
* Process the element
*/
public void processElement() {
String action = getString(ACTION);
if (action.equals(MARK)) {
    String id = getString(ID);
    Long mark = new Long(System.currentTimeMillis());
    getXMLTestCase().addFoundObject(id, mark);
} else {

    //Get the stored objects
    long mark = ((Long) getXMLTestCase().getFoundObject(getString("refid"))).
        longValue();
    long value = getLong(VALUE, 0);

    long newMark = System.currentTimeMillis();

    if (action.equals(LESSTHAN)) {
      getTestCase().assertTrue("Duration exceeded.", newMark - mark >= value);
    }

}
}
}

Registering Tag Handlers

Now that we have a tag handler created. We need to register the handler in the XML file. This enables the test case to locate the handler and execute the code. All tag handlers are global in scope and are carried between test cases and suites.

When the XML is processed, the element name will be used to identify and run the tag handler.

jfcUnit XML Sample

In the case of jfcUnit, we provide many tag handlers for performing all of the functions that jfcUnit can perform. Below is a sample test case defined for jfcUnit. This example illustrates a common pattern of finding a component, followed by the set of event on the component.

<suite name="JTextComponent">
    <test name="Add my own text" robot="true">
        <manager debug="true" recording="true"/>
        <find finder="ComponentFinder" id="TabPane" class="javax.swing.JTabbedPane" index="0"/>
        <click type="JTabbedPaneMouseEventData" refid="TabPane" title="Plain Text" clicks="1"/>
        <find finder="ComponentFinder" id="Text" class="javax.swing.text.JTextComponent" index="5"/>
        <click type="JTextComponentMouseEventData" refid="Text" index="0"/>
        <click type="JTextComponentMouseEventData" refid="Text" index="1"/>
        <click type="JTextComponentMouseEventData" refid="Text" index="2"/>
        <click type="JTextComponentMouseEventData" refid="Text" index="3"/>
        <click type="JTextComponentMouseEventData" refid="Text" index="4"/>
        <key refid="Text" code="VK_A" modifiers="control"/>
        <key refid="Text" string="This is the text I wanted to enter." />
    </test>
</suite>

jfcUnit XML Sample Recording

To start recording a new JFCUnit test case, the following can be used as a template.

<suite name="Suite">
    <test name="Login" robot="true">
        <record/>
    </test>

    <test>
        <save file="new.xml"/>
    </test>
</suite>

Start your test case (See java coding requirement above). When the record element is hit in the test case new elements will be added above the record tag. When the save tag is processed, the current XML file will be written to the file specified. Afterward, the new.xml may contain the following:

Loop through tables and lists with:
for-each - junit.extensions.xml.elements.ForeachTagHandler
noop - junit.extensions.xml.elements.NoOpTagHandler
indexof - junit.extensions.xml.elements.IndexOfTagHandler
manager - junit.extensions.jfcunit.eventdata.JFCEventManagerTagHandler
path - junit.extensions.jfcunit.xml.eventdata.PathTagHandler
procedure - junit.extensions.xml.elements.ProcedureTagHandler
parentof - junit.extensions.xml.elements.ParentInstanceTagHandler
getparent - junit.extensions.xml.elements.PropertyTagHandler

Record events to the XML file.
record - junit.extensions.jfcunit.xml.XMLRecorder

Write out the current XML file:
save - junit.extensions.xml.elements.SaveTagHandler
sleep - junit.extensions.jfcunit.xml.SleepTagHandler
stopwatch - junit.extensions.xml.elements.StopWatchTagHandler
suite - junit.extensions.jfcunit.xml.SuiteTagHandler
taghandlers - junit.extensions.xml.elements.TagHandlersTagHandler
test - junit.extensions.jfcunit.xml.TestTagHandler

The following TagHandlers are used to submit events into the GUI.
click - junit.extensions.jfcunit.eventdata.ClickTagHandler
drag - junit.extensions.jfcunit.eventdata.DragTagHandler
wheel - junit.extensions.jfcunit.eventdata.MouseWheelEventDataTagHandler
key - junit.extensions.jfcunit.eventdata.KeyTagHandler

The find TagHandler help to locate objects in your GUI application.
find - junit.extensions.jfcunit.finder.FindTagHandler