In this article I’m going to talk about Unit/Solitary/Mockist/London Style tests and compare them to Integrated/Social/Classicist/Detroit Style tests. This won’t be a thorough review. I’m actually going to focus on one specific argument: If you choose one over the other, will you write more or less tests? There are many other important aspects worth considering (the speed of your tests, the confidence they give you, the way they impact your code design, etc.), but I’m just going to focus on the number of tests you have to write.
If you’re unfamiliar with these concepts, the terminology in Martin Fowler’s Unit Test article can be a good primer. Here’s an image for reference: 
The reason I’m writing this article is because I saw a video a few years ago called “Integrated Tests Are A Scam”.
J.B. Rainsberger – Integrated Tests Are A Scam from devtraining on Vimeo.
The main theme of this video is that integrated (AKA Sociable) tests are generally inferior to mockist/unit/Solitary tests and one of J.B. Rainsberger’s arguments is that it doesn’t make sense to use integrated tests when you think about things mathematically. Here’s some code to explain the argument:
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
public class A { private B b; public void execute(int maybeEven) { if (isEven(maybeEven)) { b.execute(maybeEven); } else { b.execute(maybeEven + 1); } } } class B { private C c; void execute(int maybeFavoriteNumber) { if (isFavoriteNumber(maybeFavoriteNumber)) { c.execute(true); } else { c.execute(false); } } } class C { void execute(int maybeFactorial) { if (isFactorial(maybeFactorial) { System.out.println("Is factorial"); } else { System.out.println("Is not factorial"); } } } |
If we wanted to completely test all the paths, how many tests would we have to write? Well if we’re writing mockist style unit tests, we’d have to write 6:
- When testing
A, we’d mock outBand write a test for whenmaybeEvenis even and another test for whenmaybeEvenis odd. - When testing
B, we’d mock outCand write a test for whenmaybeFavoriteNumberis your favorite number and another test for whenmaybeFavoriteNumberisn’t. - When testing
C, we’d write a test for whenmaybeFactorialis a factorial and another test for whenmaybeFactorialisn’t.
But, if we were writing integrated tests, we’d have to write 8 tests. Each of these would test A‘s execute method and the input would have to satisfy all these conditions:
even? == true,favoriteNumber? == true,factorial? == trueeven? == true,favoriteNumber? == false,factorial? == falseeven? == true,favoriteNumber? == false,factorial? == trueeven? == true,favoriteNumber? == true,factorial? == falseeven? == false,favoriteNumber? == false,factorial? == trueeven? == false,favoriteNumber? == false,factorial? == falseeven? == false,favoriteNumber? == true,factorial? == trueeven? == false,favoriteNumber? == true,factorial? == false
The more classes and paths you have, the more integrated tests you have to write relative to the unit tests. This argument never felt right to me but it took over a year for me to figure out why. The scenario assumes that both branch paths lead into the same class and method. That’s not always true. What if your code looks like this?
|
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 |
public class X { private Y y; private Z z; public void execute(int maybeEven) { if (isEven(maybeEven)) { y.execute(maybeEven); } else { z.execute(maybeEven); } } } class Y { void execute(int maybeFavoriteNumber) { if (isFavoriteNumber(maybeFavoriteNumber)) { System.out.println("Favorite number"); } else { System.out.println("Not Favorite number"); } } } class Z { void execute(int maybeFactorial) { if (isFactorial(maybeFactorial) { System.out.println("Is factorial"); } else{ System.out.println("Is not factorial"); } } } |
In this scenario, each branch leads to a different class instead of the same class. How many unit tests will we have to write to test all paths? Just like before, you’d have to write 6 tests:
- When testing
X, we’d mock outY/Zand write a test for whenmaybeEvenis even and another test for whenmaybeEvenis odd. - When testing
Y, we’d write a test for whenmaybeFavoriteNumberis your favorite number and another test for whenmaybeFavoriteNumberisn’t. - When testing
Z, we’d write a test for whenmaybeFactorialis a factorial and another test for whenmaybeFactorialisn’t.
But, how many integrated tests would we have to write?
even? == true,favoriteNumber? == trueeven? == true,favoriteNumber? == falseeven? == false,factorial? == trueeven? == false,factorial? == false
Now we only have to write 4 integrated tests. If you look at the first two, factorial? is irrelevant because execution doesn’t go into Z when the number is even . If you look at the last two, favoriteNumber? is irrelevant because execution doesn’t go into Y if the number is odd.
Back to the original question: Will you write less tests with Integrated or Unit tests? It depends on the shape of your code. If your conditional branches converge into the same class and methods, you’ll write less tests with unit tests. If your conditional branches diverge into different classes and methods, you’ll write less tests with integrated tests.