There are a lot of people out there who will say, “I hate unit tests” or “unit tests are usually not worth the hassle” but when you dig down into what people really mean, they’re actually saying, “I hate this specific kind of unit test.” The problem is most people think there is only one kind and that everyone agrees on their definition. This actually isn’t true and I’m going to explain all the varieties in this article.
A lot of people have a very specific idea of what a unit test is and think any variation on that is not a “true” unit test. They’ll say things like, “What you’re calling a unit test isn’t really a unit test because you’re doing x, y and z” or, “This isn’t a unit test, it’s an integration test”. Sometimes that’s true, but a lot of times it’s not. As you’ll see later, the boundary between unit test and integration test is not so clear cut.
You may feel like you dislike unit tests, but you shouldn’t be so hasty about your opinion if you don’t know about all the different types. There may be a variety out there that you really enjoy or you are already doing without considering them to be unit tests.
There are many goals of this article:
- To teach people about the different kinds of unit tests
- To make people realize that they may have been wrong in the past when they said some tests were not unit tests
- To make people see that the term “unit test” is so overloaded, it’s not a very useful term to use when you communicate. You’ll usually need to give clarification of the kind of unit test you’re talking about.
A good reference of these different kinds of unit tests is in Martin Fowler’s article about the topic. Basically, there are two independent axes for unit tests: Collaborator Isolation and Speed.
Collaborator Isolation
I think when most people think of unit tests, they think of one test class per production class (OTCPPC). That means every Foo class has a FooTest . But in production code, a class will often call another class. You can think of the callers to be the “system under test” and the callees to be “collaborators”. In the OTCPPC way of writing unit tests, all collaborators are mocked out so their logic does not factor in to the tests that you write. In this way, the “unit” becomes that single class that you’re testing.
There are a lot of people who feel that this is the only way to write unit tests, but this is actually not true. In fact, when unit testing originally started, everyone would write one test class for many production classes.
…when xunit testing began in the 90’s we made no attempt to isolate unless communicating with the collaborators was awkward (such as a remote credit card verification system).
Notice that the “unit” in this case is practically the opposite of what it is in the OTCPPC way of testing. A “unit” is the group of classes you’re running that is not mocked out. Some people who write code this style will pick and choose where to start the testing. They may initialize a class in the middle of the architecture. Or, they may write tests as far outside as possible (e.g., call the code that gets executed when a user clicks a button, call the method that gets called if a particular Rest API resource is used, etc.).
Speed
Unit tests are expected to have a certain amount of speed to run, but this is yet another axis that varies depending on the kind of unit tests you’re writing. On one extreme, you’ve got people who expect all of their unit tests to run super fast. These people start to cringe if all their tests take more than 3 seconds to run. This can be 10s of thousands of tests that they feel should run in less than a second. You can get these speeds regardless of which Collaborator Isolation technique you use, you just have to isolate the slow parts of your system (e.g., the deploy process, the database, remote API calls, etc.). On the other extreme, there are people who don’t mind waiting a few minutes for all their unit tests to run as long as the relevant tests for the code their changing run fast.
The benefit of the super fast technique is you can run your tests more frequently. In fact, you can configure your setup so that it runs all your tests every time you change a line of code. The downside is the effort you have to put into isolating the slow parts of your system. Also, you may not feel that you get much benefit to running all tests all the time and feel that you just need to run the relevant tests for the parts of the code you’re changing. If this is the case you’re not going to get a huge return on your investment by making everything fast: You’re not going to run all of them anyway (except on your Continuous Integration server, of course).
I should explicitly say that once your unit tests get to the point where you skip running them because they’d take too long, they aren’t unit tests anymore.
Conclusion
Regardless of the Collaborator Isolation you use or the Speed at which your unit tests run, they can still be unit tests. There’s no “one true way”. There are two axes, each one can vary and anything that falls under both can be considered unit tests. This is why the phrase “unit test” causes communication problems. There is so much variation between the different kinds that when someone uses the phrase without elaboration, you can’t really be sure you’re on the same page.