Invitation to EXPE
Last updated: Thu Feb 28 01:51:40 2008
Why EXPE?
EXPE is an extensible environment for designing and running psychology experiments, formal usability tests and any other activities that ask people to perform tasks in order to collect data. When data collection is not required, it can also be used to present these tasks, useful for education, demonstration, and other purposes.
The benefits of EXPE to its user can be many. Two major advantages stand out: openness and extensibility.
Open Source - Open Science
EXPE is released under GNU General Public License published by Free Software Foundation. The benefits of open source software for scientific research has been proved in many projects. One prominent example is R. Since you are on sourceforge.net, I am not going to lecture you on this point.
Flexible, Extensible and Easy to Use
Instead of requiring users to write computer programs, designing experiments in EXPE is mostly done with graphic user interface (GUI). Using EXPE should not be much more difficulty than using GUI presentation software. Both top-down (starting from overall design) and bottom-up (starting from individual experimental trials) approaches of experiment design are supported. An experiment can be run in whole or in parts. Sessions, blocks, and trials, all these three levels of granularities are supported. This flexibility is useful for testing, demonstration and educational purposes, in addition to running experiments.
Under the hood, EXPE is mostly controlled by human editable XML files. Instead of using arcane technical jargons, EXPE's declarative experiment configuration language adopts as much as possible the standard experimental design terminology familiar to psychology researchers.
It is possible for an experienced researcher to fine-tune experiment setup by editing XML files. Users can also extend EXPE's XML schema if they are so desired. However, we would like to build the GUI of EXPE to be powerful enough so that direct editing on XML files can be avoided.
For users who are willing to write code, and for those experiments that require writing code, scripting is supported in EXPE. However, we do not intend to invent yet another scripting language of our own. Instead, EXPE provides a Java language interface to the underlying functionalities, and let scripting language interpretors talk with this interface. Any existing scripting languages that run in a Java Virtual Machine can be supported. This includes a long list of languages. Users should be able to find a language that they are comfortable with instead of being forced to learn a new language to use EXPE.
The flexibility of EXPE is mostly due to its architecture, which is built entirely on plugins. Using the plugin framework in Java Plugin Framework (JPF), EXPE is infinitely extensible by third-parties. For example, instead of running on normal computer screens, a plugin can be written to support experiments on unusual platform such as mobile devices. Plugins can also be written to support running experiments on the Web, which would be very useful for Web usability testing. Plugins can be written to support instrumentation of third-party software instead of running experiments inside EXPE. People can even use EXPE as a data collection platform for observations and interviews that are not conducted on a computer. With different plugins, data files in EXPE can be transformed into different formats. Adding a script language support is a matter of writing a plugin.
We hope the openness, extensibility, and usefulness of EXPE would gain itself some friends among researchers and usability practitioners alike. Right now, we are just beginning to work on EXPE, your contributions in any form are welcomed. As an early contributor, you can shape the direction of the project and be recognized as core contributors. If you want to join the project, please contact us.
The Components of EXPE
Although EXPE can potentially be composed of untold number of plugins, there are several major plugins that serves as scaffoldings for other plugins.
EXPE Designer
This component is one of the main graphic interfaces that an experiment designer will work with. It offloads some routine and error-prone aspects of experiment implementation from the users, such as dealing with within/between subject design, randomization, counter-balancing, Latin-squares, and so on. Users only need to declare these constraints on independent variables, and EXPE will work out the details of experiment arrangement. This way, users can focus on the logic of their own experiments.
Cognitive Experiment Description Language (CEDL)
CEDL is the XML document format that embodies the declarative description of experiment design. An experiment's independent variables, dependent variables and other experiment design aspects are defined in a CEDL file.
EXPE Trial Builder
This is the core user interface component of EXPE, where the actual experimental trials are designed: instructions, stimulus, data records, timing, and so on. Most engineering and extensions in EXPE are expected to happen here.
The default Trial Builder uses Scalable Vector Graphics (SVG) as the visual stimulus rendering layer. Instead of building everything from scratch, we start our work from the code base of GLIPS Graffiti, a SVG editor based on Apache Batik SVG library.
Cognitive Experiment Trial Template (CETT)
CETT is the XML document format that defines a template for an experiment trial. It can then be instantiated with proper values during experiment runs. CETT refers to independent and dependent variables defined in CEDL.
EXPE Session Composer
This user interface component allows experimental trails to be assembled into experiment sessions. Session Composer aims to support maximal flexibility in running an experimental session. For instance, experimental blocks from multiple experiments can be assembled into one session; another session can consists of one trial only (for demonstration purpose), and so on.
Session Composer also ensures that experiment sessions can actually be realistically ran on subjects. For example, a session taking more than 3 hours is often not realistic. Session Composer can estimate how long an experiment session would take and warn the designer.
Cognitive Experiment Running Plan (CERP)
An experiment session is described by a CERP file. It contains instantiated CETTelements. One experiment (describe by a CEDL file) will often be compiled into many CERP files.
EXPE Player
This component takes a CERP file, runs the experimental session and produces data. It can be launched from within the main EXPE user interface, or can run as a standalone application.
EXPE Data Transformer
This component is used to transform EXPE native XML based data format to whatever format users would like to have.
Implementation Details
Cross-platform
EXPE is written in pure Java.
Modular Design
As mentioned before, EXPE consists entirely of plugins. The interrelations among plugins are well defined extension-point/extention contract. Other than that, plugins are relatively independent.
XML Handling
We use XML schema to define CEDL, CETT and CERP file formats. JAXB 2 generated code is used to read and write XML instance files and provide mapping between XML and Java objects.
Timing Accuracy
For Reaction Time (RT) experiments, the system's time measurement accuracy is important. Using "long time=System.nanoTime()" call, tests found that Java can achieve one millisecond time resolution on most operating systems. In the future, real-time clock hardware may be supported, I have not investigate this possibility yet.
Build Enviornment
We use standard Java build toolchain: ant build files, log4j for logging, JUnit for testing, and svn for version control.