stiefels.net

Unit testing is some kind of a controversial topic. There are many opinions by software developers whether to make unit tests and how frequently they should be used.
I don’t want to start a preach here or discuss the pros and cons of unit tests in every little detail. It’s just a little tutorial how Unit Testing works with Objective-C and Xcode.

Software needs to be tested, right? So, when is the right time to test your application?
Well, before I heard something about Unit Tests I did tests every time after I changed a function or added some functionality to my software and I think this is what most people do.
The problem is that mostly after these changes only a few parts of the application are tested. The application is/can never tested completely by the developer by hand and errors might occur when a end user uses it.
So the ideal test would be to go all through your software and test every little feature and functionality. I think that is not really possible. But this process can be automated. And at this point Unit Tests come into the play.

Technorati Tags: objective-c, unit testing

Unit Tests are used to validate components of your software (e.g. classes) during the development cycle. If written in parallel to the actual application code these tests can be used to validate your code immediately after it is compiled.
This ensures that your code is working as expected, if your tests succeed. Also, you’ll notice very soon if something is broken and this can save you a lot of time to locate the error(s).
The more tests, the more functionality can be tested, but also the time effort to write these tests increases. Time which could have been used to write actual code. But on the other hand it could be possible, that one small additional test could save you hours of debugging in the future.
It is important to find the right tradeoff between the time which is invested in writing tests and the usefulness of this test.

How much tests are created is the decision of each individual developer but there are a few reasonable cases which should be considered.
For example, if a method does some calculations it would be nice to know how this method reacts to particular values such as extreme values or the null value.
In Apples documentation I found a nice rule of thumb: One test is better than none.

In this arcticle I want to show how Unit Tests can be included in your software development cycle.
I want to do this in a tutorial like style, so you can follow the steps easily. Although this tutorial sticks to Objective-C and the SenTestingKit Unit Tests are also available to a wide range of programming languages since it is a general concept of software development.

So far so good. Let’s get started.

The fraction class

Our tutorial project is a small class which represents a mathematical fraction.
One thing we want to do is that we will create the Unit Tests before we start creating the actual project code. This approach is called Test-Driven-Development as your tests describe what the class have to do.

In a UML-like class diagramm it looks like this:

Creating a Xcode project

At this point we should create our Xcode project, but we won’t write any fraction-class code yet.
Start Xcode, select New project… from the File menu and as we just have a simple class it is sufficient to create a command line tool.
Select Foundation Tool within the Command Line Utility section and name it UTFraction.
This will create one Objective-C file containing the main function.

What to test?

Writing tests for the getters and setters is boring and does not make very much sense.
But we have two interesting methods in our class which we should test: cancelDown and greatestCommonDivisorOf:and:.
And what exaclty do we want to test?

cancelDown:

1. Successful canceling with numerator = 10, denominator = 5; expected result = 2/1
2. Unsuccessful canceling with numerator = 9, denominator = 5; expected result = 9/5
3. Canceling down with numerator = 0, denominator = 5; expected result: 0/5

greatestCommonDivisorOf:and:

1. GCD not equal 1: numerator = 119, denominator = 14; expected result = 7
2. Numerator zero: numerator = 0, denominator = 11; expected result = 1

Above I mentioned that testing the getters and setters is useless. Well, except that setting the denominator to zero should be prohibited by throwing an exception because a fraction with a denominator value of zero is not defined.

setDenominator:

1. throws exception if parameter is zero.

As we now have defined what tests we want to implement we can start writing them.

Adding the Unit Test target

Adding Unit Tests to a project is a very simple procedure as you just have to add another target to your Xcode project.
To do this click on the gearwheel and select New Target.

An assistant window appears. Select Unit Test Bundle in the Cocoa section and name it Unit Tests.

Close the upcoming Target Info window. You’ll notice that the new target appears in the targets section within the Groups & Files column.

Writing the tests

Unit tests are organized in classes, so called test case classes.
Within these classes test cases are defined. Each method of a test case class represents a test case. Usually, a test case method implements one specific test case but it is also possible to create complex test cases within one method.
To add a test case class to your Unit Test target, open the File menu and select New File…. Select the Objective-C test case class entry within the Cocoa section.

Name it FractionTests. Please note that the corresponding target is Unit Tests so be sure the right checkbox is set.

A test case class is organized in three parts: setup, teardown and tests.

In the setup, preparing work is done, such as object instantiation or variable definitions. Setup is automatically called before each test case is executed.
In our setup method we will instantiate one Fraction object and set some default values.

In the test class header file (FractionTests.h), we will add a attribute named fraction of the class Fraction (which does not exist yet).

#import <SenTestingKit/SenTestingKit.h>
#import "Fraction.h"

@interface FractionTests : SenTestCase {
    Fraction *fraction;
}

@end

In the implemetation file (FractionTests.m) we start writing the setUp method:

- (void)setUp
{
    fraction = [[Fraction alloc] initWithNumerator:1
                                       denominator:1];
}

The teardown method is the complement to the setup method and is used to clean up the memory i.e. by releasing objects.
As you already might think this method is automatically called after each test case.

- (void)tearDown
{
    [fraction release];
}

Now we come to the part where we implement the tests we considered above. Let’s start with the first one.

- (void)testSuccessfulCanceling
{
    [fraction setNumerator:10];
    [fraction setDenominator:5];
    [fraction cancelDown];
    STAssertTrue([fraction numerator] == 2,
      @”Numerator != 2 (is %d)”, [fraction numerator]);

    STAssertTrue([fraction denominator] == 1,
      @”Denominator != 1 (is %d)”, [fraction denominator]);
}

All test case methods have to begin with test in the method name. This ensures that this method is called automatically by the test case class.
You can also add some other methods but they are not called automatically. The first step is to set our fraction object attributes.
After that we call our cancelDown: method which will (if implemented correctly) cancel down our fraction. To check whether this worked we use one of the SenTestingKit macros.
The STAssertTrue-macro checks whether the expression (first parameter) is true. If so nothing happens, but if the expression is false, an error message appears (second parameters).

These macros have to be used to tell the test case class if this test case worked or not. There are many macros you can use but there is only a small number of commonly used ones:

STAssertNotNil(a1, description, ...)
STAssertTrue(expression, description, ...)
STAssertFalse(expression, description, ...)
STAssertEqualObjects(a1, a2, description, ...)
STAssertEquals(a1, a2, description, ...)
STAssertThrows(expression, description, ...)
STAssertNoThrow(expression, description, ...)
STFail(description, ...)

The description parameter is a formatable NSString which is shown as error message if the test fails. It is adviseable to display the current value of the object tested beside the actual error message (as in the tests we write).

I think most of them are self-explanatory. The STFail(…) macro can be called to create an error message without any expression to be checked in the parameter list.

The next test so not very different to the first.

- (void)testUnsuccessfulCanceling
{
    [fraction setNumerator:9];
    [fraction setDenominator:5];
    [fraction cancelDown];
    STAssertTrue([fraction numerator] == 9,
      @”Numerator != 9 (is %d)”, [fraction numerator]);
    STAssertTrue([fraction denominator] == 5,
      @”Denominator != 5 (is %d)”, [fraction denominator]);
}

I think you got the principle.

- (void)testCancelingWithZeroNumerator
{
    [fraction setNumerator:0];
    [fraction setDenominator:5];
    [fraction cancelDown];
    STAssertTrue([fraction numerator] == 0,
      @”Numerator != 0 (is %d)”, [fraction numerator]);

    STAssertTrue([fraction denominator] == 5,
      @”Denominator != 5 (is %d)”, [fraction denominator]);
}

- (void)testGreatestCommonDivisor
{
    int gcd = [Fraction greatestCommonDivisorOf:119 and:14];
    STAssertTrue(gcd == 7, @”GCD != 7 (is %d)”, gcd);
}

- (void)testGreatestCommonDivisorWithZeroNumerator
{
    int gcd = [Fraction greatestCommonDivisorOf:0 and:11];
    STAssertTrue(gcd == 1, @”GCD != 1 (is %d)”, gcd);
}

The sixth test is different as we use the STAssertThrows(…) macro which catches thrown exceptions and fails if no exception is thrown.

- (void)testSetDenominatorThrowingException
{
    STAssertThrows([fraction setDenominator:0],
       @”setDenominator should throw an exception”);
}

As you should have noticed, the setUp:, tearDown: and all of the test methods return void and don’t have any parameters.
Don’t forget to put the test method prototypes in your header file (FractionTests.h).

- (void)testSuccessfulCanceling;
- (void)testUnsuccessfulCanceling;
- (void)testCancelingWithZeroNumerator;
- (void)testGreatestCommonDivisor;
- (void)testGreatestCommonDivisorWithZeroNumerator;
- (void)testSetDenominatorThrowingException;

That’s it. Let’s jump over to implement our Fraction class.

Implementing the Fraction class

To implement our Fraction class we have to add another class to our project. Click New File… in the File menu and select Objective-C class.

Name the class Fraction and ensure the class is added to the UTFraction target by setting the checkbox.

Our header (Fraction.h) file will look something like this:

#import <Cocoa/Cocoa.h>

@interface Fraction : NSObject {
    int _numerator;
    int _denominator;
}

+ (int)greatestCommonDivisorOf:(int)a and:(int)b;

- (id)initWithNumerator:(int)numerator denominator:(int)denominator;
- (void)setNumerator:(int)numerator;
- (void)setDenominator:(int)denominator;
- (int)numerator;
- (int)denominator;
- (void)cancelDown;

@end

I won’t discuss the implementation of the Fraction class in detail since it is not the topic of this article but I think it should be easy to understand what happens here.

#import "Fraction.h"

@implementation Fraction

+ (int)greatestCommonDivisorOf:(int)a and:(int)b
{
    if (a == 0 || b == 0) return 1;
    if (a < b) return [Fraction greatestCommonDivisorOf:a and:(b-a)];
    if (a > b) return [Fraction greatestCommonDivisorOf:(a-b) and:b];

    return a;
}

- (id)init {
    [self initWithNumerator:0 denominator:1];
    return self;
}

- (id)initWithNumerator:(int)numerator denominator:(int)denominator
{
    self = [super init];
    if (self != nil) {
        [self setNumerator:numerator];
        [self setDenominator:denominator];
    }
    return self;
}

- (void)setNumerator:(int)numerator
{
    _numerator = numerator;
}

- (void)setDenominator:(int)denominator
{
    if (denominator == 0) {
        NSException *divisionByZeroException =
          [NSException exceptionWithName:@"DivisionByZeroException" 
                                  reason:@"Denominator cannot be zero!" 
                                userInfo:nil];
        @throw divisionByZeroException;
    }
    _denominator = denominator;
}

- (int)numerator
{
    return _numerator;
}

- (int)denominator
{
    return _denominator;
}

- (void)cancelDown
{
    int gcd = [Fraction greatestCommonDivisorOf:[self numerator]
                                            and:[self denominator]];
    [self setNumerator:[self numerator]/gcd];
    [self setDenominator:[self denominator]/gcd];
}

@end

As we are going to use Objective-C exceptions we have to turn on the corresponding compiler option.
Right click on the UTFraction target, select Get Info, switch to the Build tab and enable the Objective-C exception flag.

Also, it is recommended to deactivate ZeroLink.

So, now we have our tests and our class, but how can we run the tests?

Dependent and independent mode

There are two different modes you can choose from: Independent or dependent.

In independent mode you have to run your tests manually every time you want to see if your code is validated by your tests.
The disadvantage is that this approach can lead to infrequent use of your tests and errors may not be recognized immediately.

In dependent mode the UTFraction target is made dependent to the Unit Tests target. Every time you compile your code all tests are run automatically.
The advantage is that errors are detected by your tests very soon.

In our tutorial we’ll use the dependent mode.

Setting up the dependent mode

Setting up the dependent mode is not a big thing. All you have to do is adding the UTFraction target as a direct dependency of the Unit Tests target and set some of the build settings.
Open the Target Info window by right clicking on the Unit Tests target and select Get Info.

Switch to the General tab and add the target Unit Tests by clicking on the + icon at the bottom of the window.

Switch to the Build tab and set the Bundle Loader value to $(BUILT_PRODUCTS_DIR)/UTFraction.

Setting this values tells the linker to use the executable as a additional “framework” so that our class can be used in our tests.

As the Test Host setting needs to be set to exact same value you can use $(BUNDLE_LOADER) in this field.

Setting the Test Host value tells the Unit Test framework to “inject” your tests into the executable.

If you are building an application you have to set the value of the Bundle Loader and Test Host setting to the path of your executable (e.g. $(BUILT_PRODUCTS_DIR)/MyApp.app/Contents/MacOS/MyApp).
If you are developing a framework or a shared library the Test Host value should not be set.

Close the window. Be sure to select the Unit Tests target as your Active Target.

Now, every time you start compiling your code the tests are run automatically after the compilation process ended successfully. If you don’t want that the tests are run, simply select the UTFraction target as your Active Target.

Evaluating the tests

Whether you use dependent or independent mode the tests are run as a shell script. If any test fails you’ll get an error in Xcode. In this example all test should run without errors.

To see how it would look like if tests fail, we change the implementation of the greatestCommonDivisor: method by setting the first line to:

if (a == 0 || b == 0) return 2;

Compile your code and you’ll get two errors. Two tests failed!

Change the line back to the original one, compile it and you’ll see no more errors.

The End

Well, that’s it. This tutorial should have given you a small insight in the unit testing framework included in Xcode.

You can download the full tutorial project here.

If you are interested in further information, have a look at Apple’s documentation about Unit Tests.

If you have any suggenstions or comments, please don’t hesitate to leave a line (or more) in the comments.

Thank you and happy testing!