Join us on:
Facebook
LinkedIn
Plaxo

T15. Parameterized Unit Testing: Principles, Techniques, and Applications in Practice

Nikolai Tillmann, Microsoft Research

Nikolai Tillmann is a Research Software Design Engineer in the Foundations of Software Engineering (FSE) group at Microsoft Research in Redmond, USA, where he has been since January 2001. Previously, he received a M.S. ("Diplom") in Computer Science from the Technical University of Berlin in 2000. After working on modeling techniques and tools in the AsmL and Spec Explorer projects, his current research area is runtime verification and techniques for automatic test case generation. He gave a tutorial on "Parameterized Unit Testing with Pex" at TAP 2008, and will give a previous version of the tutorial at ICSE 2009. http://research.microsoft.com/users/nikolait/

Jonathan De Halleux, Microsoft Research

Peli joined the Foundations for Software Engineering in October 2006. From 2004 to 2006, he worked in the Common Language Runtime (CLR) as a Software Design Engineer in Test (SDE/T) in charge of the Just In Time compiler. Before joining Microsoft, he earned a PhD in Applied Mathematics from the Catholic University of Louvain. Earlier, he developed the unit testing framework MbUnit. He gave tutorials internally at Microsoft and a tutorial on "Parameterized Unit Testing with Pex" at TAP 2008 and will give a previous version of the tutorial at ICSE 2009.

Tao Xie, North Carolina State University

Tao Xie received his Ph.D. in Computer Science from the University of Washington at Seattle in 2005. He is currently an Assistant Professor in the Department of Computer Science at North Carolina State University since 2005. His research interests are in automated software testing and mining software engineering data. He received an IBM Faculty Award and an IBM Jazz Innovation Award in 2008. He is Program Committee Co-Chair of ICSM 2009, Student Papers Track Program Co-Chair of ICST 2008, and Tutorial Co-Chair of ASE 2009. He gave or will give various tutorials at ICSE 2007, 2008, and 2009.

Wolfram Schulte, Microsoft Research

Developer testing has been widely recognized as an important and valuable means of improving software reliability. Recently parameterized unit testing has emerged as a very promising and effective methodology to allow the separation of two developer testing concerns or tasks: the specification of external, black-box behavior (i.e., assertions or specifications) by developers and the generation and selection of internal, white-box test inputs (i.e., high-code-covering test inputs) by tools. A parameterized unit test (PUT) is simply a test method that takes parameters, calls the code under test, and states assertions.

PUTs have been supported by JUnit 4 and various .NET test frameworks. Various industrial testing tools also exist to generate test inputs for PUTs. This tutorial presents the latest research and practice on principles, techniques, and applications of parameterized unit testing in practice, highlighting success stories, research and education achievements, and future research directions in developer testing.

The tutorial will help improve developer skills and knowledge for writing PUTs and give overview of tool automation in supporting PUTs. Attendees will acquire the skills and knowledge needed to perform research or conduct practice in the field of developer testing and to integrate developer testing techniques in their own research, practice, and education.

Objectives:

Unit tests are a key component of software engineering, and the Extreme Programming discipline, for instance, leverages them to permit easy code changes. Being of such importance, many companies now provide tools, frameworks, and services around unit tests. Tools range from specialized test frameworks, as for example integrated in Visual Studio Team System (VSUnit), to automatic unit-test generation, e.g. as provided by Parasoft Jtest and Agitar AgitarOne. However, these tools do not provide any guidance for:

  • which tests should be written (for internal and for external behavior),
  • how to come up with a minimal number of test cases and
  • what guarantees the test cases provide.

Parameterized unit testing is a new methodology extending the current industry practice of closed unit tests (i.e., test methods without input parameters). Test methods are generalized by allowing parameters. This generalization serves two purposes.

  • First, parameterized test methods are specifications of the behavior of the methods-under-test: they do not only provide exemplary arguments to the methods-under-test, but ranges of such arguments.
  • Second, parameterized unit tests describe a set of traditional unit tests which can be obtained by instantiating the parameterized test methods with given argument sets. Instantiations should be chosen so that they exercise different code paths of the methods under test.

Admittedly, writing open, parameterized unit tests is more challenging than writing closed, traditional unit tests. This tutorial serves as a starting point for familiarizing the attendees with principles, techniques, and applications of parameterized unit testing. This tutorial will greatly benefit from the increasingly importance and popularity of the developer testing field and will help increase awareness of this important and promising field. The attendees will gain the skills and knowledge needed to conduct research and/or practice on developer testing, especially in the form of parameterized unit testing.

In particular, the following skills and knowledge of conducting developer testing will be gained by the attendees and teaching these skills and knowledge will be gained by attendees with educational interests.

  • Write parameterized unit tests.
  • Write mock objects.
  • Analyze results.
  • Interact with tools to give guidance to tools.
  • Understand how dynamic symbolic execution can be used to test software.
  • Understand the limitations of the approach.
  • Understand and use test generation tools.

In addition, the attendees will also gain skills and knowledge of developing new research techniques and tools in the area of developer testing.

Format:

The tutorial will be given using an interactive style with slide-based lecture, hands-on exercise/demo, and discussions.

Audience: Researchers, Practitioners, Educators
Please email any questions to . This e-mail address is being protected from spambots. You need JavaScript enabled to view it