Unit Tests vs Integration Tests

Testing software is a vital part of any development process, we want to guarantee that our software complies against the business requirements and rules. Of course, doing this validation manually is pretty complex as a software product could have a lot of features.

How can we automate the testing process? Unit testing and integration testing are two ways to guarantee that the software product complies with the business rules, without human intervention. Those two ways are usually misunderstood, so, in this post, we are going to compare those two testing strategies focusing in the following aspects:

  • Abstraction: How many implementation details we must know to test
  • Scope: How many components we are testing (methods, classes, modules, systems)
  • Speed: How fast the tests run

NOTE: There are more testing strategies, like functional testing, API testing, end to end testing and so on. For now, we will just focus on unit and integration tests.

TRY IT YOURSELF: You can find the source code of this post here.

The Example

For the purpose of this post, we are going to use the Resume Generator Module we reviewed in the Open Close Principle by Example post. There, we define the requirements as follows:

As a business, I want to generate another kind of resume, a long person’s resume in PDF with the person’s name, a description and the date until the resume is validwhich is the current date plus 7 days.

The final system design for that requirement looks like this:

Templates design

In the following sections, we will implement unit tests and integration tests for this design. Let’s see them regarding the three aspects: Abstraction, Scope and Speed.

Unit Tests

Unit tests are validations for our business rules. For the case of the example, we created the following unit tests:

Unit Tests

Now, let’s use them to discuss the Abstraction, Scope and Speed aspects.

Abstraction

Unit tests are pretty specific, detailed. They validate the business rules at low level, for instance:

  • Valid inputs: Guarantee that any input into the system has the right type, value, syntax, etc.
  • Decisions branches: Multiple flows could be done to accomplish a use case. Unit tests should validate each of those.
  • Exceptions flows: Error flows can happen, and they are part of the business logic we should validate.

For example, let’s see the BasicResumeDataAggregator class. This class is responsible of aggregating data to the basic resume. There, we required the personName value, otherwise, an exception will be thrown.

public class BasicResumeDataAggregator implements
        DataAggregator {
  @Override
  public Map<String, Object> aggregate(
          Map<String, Object> data) {
    if (!isValidResume(data)) {
      throw new IllegalArgumentException(
              "Not valid data for basic resume");
    }

    Map<String, Object> newData = new HashMapMap<>(data);

    newData.put("validUntil", getNow());

    return newData;
  }

  LocalDateTime getNow() {
    return LocalDateTime.now();
  }

  private boolean isValidResume(
          Map<String, Object> data) {
    return data != null && !data.isEmpty() && data
            .containsKey("personName");
  }
}

Now, let’s define unit tests for this class, regarding the valid inputs, decisions branches and exceptions flows.

@ExtendWith(MockitoExtension.class)
public class BasicResumeDataAggregatorTest {
  @InjectMocks
  @Spy
  private BasicResumeDataAggregator basicResumeDataAggregator;

  @Test
  public void aggregate_nullData_exception() {
    IllegalArgumentException thrown = assertThrows(
            IllegalArgumentException.class,
            () ->
                    basicResumeDataAggregator
                            .aggregate(null));
    assertThat(thrown.getMessage()).isEqualTo(
            "Not valid data for basic resume");
  }
  .......

  @Test
  public void aggregate_personName_newData() {
    doReturn(LocalDateTime.of(2020, 1, 1, 1, 1, 1))
            .when(basicResumeDataAggregator).getNow();
    Map<String, Object> data = basicResumeDataAggregator
            .aggregate(Map
                    .of("personName", "daniel",
                            "key2", "value2"));
    assertThat(data).isEqualTo(Map
            .of("personName", "daniel",
                    "key2", "value2",
                    "validUntil",
                    LocalDateTime
                            .of(2020, 1, 1, 1, 1, 1)));
  }
  ......
}

In the first test case, we validate that the data map exists. We guarantee that if the data map doesn’t exist, an exception is thrown.

The second test case, validate that if the data is complete, a new map is created adding the validUntil date.

The test cases are pretty detailed, we know the implementation details, like exceptions, branches and validations.

Scope

How many components/classes/methods should you test in an unit test? well, a few.

For instance, let’s see TemplateProcess class. This class processes a template with its name and parameters, choosing the right DataAggregator to aggregate data to the initial information.

public class TemplateProcess {
  private final TemplateRender templateRender;
  private final Map<String, DataAggregator> dataAggregators;

  public TemplateProcess(
          TemplateRender templateRender,
          Map<String, DataAggregator> dataAggregators) {
    this.templateRender = templateRender;
    this.dataAggregators = dataAggregators;
  }

  public String processTemplate(String templateName,
                                Map<String, Object> data) {
    if (dataAggregators.containsKey(templateName)) {
      DataAggregator dataAggregator = dataAggregators
              .get(templateName);

      data = dataAggregator.aggregate(data);

      return templateRender
              .render(templateName + ".template",
                      data);
    } else {
      throw new IllegalArgumentException(
              "Template not supported: " + templateName);
    }
  }
}

Now, let’s see some unit tests:

@ExtendWith(MockitoExtension.class)
public class TemplateProcessTest {
  @Mock
  private TemplateRender templateRender;
  @Mock
  private DataAggregator dataAggregator1;
  @Mock
  private DataAggregator dataAggregator2;

  @Test
  public void processTemplate_templateNotFound_exception() {
    TemplateProcess templateProcess = new TemplateProcess(
            templateRender,
            Map.of("template0", dataAggregator1,
                    "template2", dataAggregator2));

    IllegalArgumentException thrown = assertThrows(
            IllegalArgumentException.class,
            () ->
                    templateProcess
                            .processTemplate("template1", Map.of()));

    assertThat(thrown.getMessage()).isEqualTo(
            "Template not supported: template1");
  }

  @Test
  public void processTemplate_templateFound_render() {
    when(dataAggregator2
            .aggregate(Map.of("ke1", "value2",
                    "key2", "value2")))
            .thenReturn(Map.of("ke1", "value2",
                    "key2", "value2",
                    "keyAggregated", "value3"));
    when(templateRender.render("template2.template",
            Map.of("ke1", "value2",
                    "key2", "value2",
                    "keyAggregated", "value3"))).thenReturn("rendered");

    TemplateProcess templateProcess = new TemplateProcess(
            templateRender,
            Map.of("template0", dataAggregator1,
                    "template2", dataAggregator2));

    String result = templateProcess
            .processTemplate("template2", Map.of("ke1", "value2",
                    "key2", "value2"));

    assertThat(result).isEqualTo("rendered");
  }
}

There, we are testing just one method processTemplate in the TemplateProcess class. We mocked everything else, the DataAggregators and the TemplateRender classes. That means, the focus of this tests is just the logic into processTemplate method, nothing more.

Could you test more components in one unit tests? yes, you could, sometimes makes sense to tests a little flow of two or three classes. However, we should think in that carefully, if the logic is huge, in some of those classes, it might be wise to isolate that logic into its own tests.

On the other hand, testing multiple classes at the same time helps us to avoid mocking, you don’t need a test class by class, and any refactor in your business logic shouldn’t affect your tests classes structure, that means, you are decoupling the structure of your business logic and your tests.

Now, let’s see another interesting example, the TemplateController class. This class is a controller, a RESTFul service who exposes an API, it is just a pass through to the TemplateProcess class.

@RestController
public class TemplateController {
  private final TemplateProcess templateProcess;

  public TemplateController(
          TemplateProcess templateProcess) {
    this.templateProcess = templateProcess;
  }

  @PutMapping("/templates/{templateName}")
  public String renderResume(
          @PathVariable String templateName,
          @RequestBody Map data) {
    return templateProcess
            .processTemplate(templateName, data);
  }
}

How do you unit test that class? well, as any other class:

@ExtendWith(MockitoExtension.class)
public class TemplateControllerTest {
  @Mock
  private TemplateProcess templateProcess;
  @InjectMocks
  private TemplateController templateController;

  @Test
  public void renderResume_userTemplateProcess_render() {
    when(templateProcess.processTemplate("tempplate1",
            Map
                    .of("ke1", "value2",
                            "key2", "value2")))
            .thenReturn("rendered template");

    String render = templateController
            .renderResume("tempplate1",
                    Map
                            .of("ke1", "value2",
                                    "key2", "value2"));

    assertThat(render).isEqualTo("rendered template");
  }
}

As we can see, we just mocked TemplateProcess class and validate that the pass through is done well.

You might think “but you are not testing the controller itself, what if some developer change the path from /templates/{templateName} to /othertemplates/{templateName}? your unit tests won’t catch that change “, you are right, we are testing the business rules in the controller, not the controller itself.

Testing the controller itself is beyond the scope of the unit tests, why? well, to test the controller itself you will test more components like:

  • The JSON to Object mapping framework.
  • The Mvc (RESTFul) framework that maps URLs to Java methods.
  • The HTTP Server whose receives HTTP requests.

We will increase the complexity of the unit tests adding those components in the testing scope, as we will need to be aware of more things outside of your business rules.

Finally, in the example, the following image shows which components we tests in isolation:

Which components unit tests cover

Speed

Unit tests should run fast. The idea is to get fast feedback, if you add a new requirement to the code and run the whole tests suite and everything passes, it means, the business rules are stable.

For instance, if we execute the tests suite, we get the following.

Running the unit tests

As we can see, we run 13 unit tests in 599 ms. That means, in average, one unit test run in 46 ms.

NOTE: There are other factors that affects the speed of unit tests, like the JVM warm period and the resources you machine has.

Now, 13 tests are pretty low in compare with real huge projects, so, let’s see a real one.

Spring Initializr is a an Open Source project to generate Spring Boot applications (https://start.spring.io/). You can find the source code in GitHub:

Spring Initialzir source code

Let’s run the unit tests for the project initializr-generator, the results are below:

Spring Initializr unit tests

As we can see, we run 548 unit tests in 3953 ms. That means, in average, one unit test run in 7 ms.

That is a pretty good feedback from 548 business rules in just 4 seconds. As they are fast, you are willing to run them frequently, so, you guarantee that any change fits the current business rules.

Integration Tests

Integration tests are validations for our business rules, but, at high level, with less detail control. For the case of the example, we created the following integration tests:

Integration tests

By each high level flow in the application, we created one test case. Those test cases are bounded by the following items:

Abstraction

Integration tests are pretty generic, not detailed. They validate the use cases at high level, for instance:

  • Components flow: Check if two or more components work fine together, building a user flow in the application. For instance, an API REST is called, and it calls a service layer, who calls a database layer.
  • Frameworks: Validate how the frameworks work with the business rules to get the job done. Frameworks like Spring, JEE, Jackson, Servers, and so on are part of this item. Those are hidden components, they are needed to get the application working, but, they are not part of your business rules.

NOTE: Take into account that a component could be outside of the application control, for instance, an external service. If that external service changes its logic/API without prior notice, it’s wise to have tests against it to catch those changes. Contract tests in REST is a good strategy to validate those scenarios.

Integration tests usually don’t care about low level details, because those level details must be handled by unit tests. Now, depending on your requirements, you might need to know some details, like database connections, or external services.

For example, let’s see the WholeAppTest class. This class will test the whole application, from the RESTFul API to the template generation.

@SpringBootTest
@AutoConfigureMockMvc
public class WholeAppTest {
  @Autowired
  private MockMvc mvc;

....

  @Test
  public void renderResume_basicResume_render()
          throws Exception {
    MvcResult result = mvc
            .perform(MockMvcRequestBuilders
                    .put("/templates/basicResume")
                    .content(asJsonString(Map
                            .of("personName",
                                    "Daniel")))
                    .contentType(
                            MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andReturn();

    assertThat(
            result.getResponse().getContentAsString())
            .contains(
                    "Template to be replaced: basicResume.template, with data: {personName=Daniel, validUntil=");
  }

...
}

There, we have one test case to test the basic resume template. Let’s see each part in detail:

  • @SpringBootTest and @AutoConfigureMockMvc help us to start the whole Spring Boot application, with the Mvc (RESTFul API) ready to receive requests.
  • MockMvc will allow us to do RESTFul requests to the running Spring Boot application.
  • In the test method, we perform a RESTFul request, with the method PUT, to the PATH /templates/basicResume, with the personName data.
  • And finally, we get the response and check if it is what we expected.

As we can see, we don’t know the details about how the basic resume template is render, we don’t know which classes they use, we just test a high level flow.

Scope

How many components should you test in an integration test? well, several, perhaps, all.

For instance, let’s see WholeAppTest class again, regarding the long resume template flow.

@SpringBootTest
@AutoConfigureMockMvc
public class WholeAppTest {
  @Autowired
  private MockMvc mvc;

...

  @Test
  public void renderResume_longResume_render()
          throws Exception {
    MvcResult result = mvc
            .perform(MockMvcRequestBuilders
                    .put("/templates/longResume")
                    .content(asJsonString(Map
                            .of("personName", "Daniel",
                                    "description",
                                    "My cv")))
                    .contentType(
                            MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andReturn();

    assertThat(
            result.getResponse().getContentAsString())
            .contains(
                    "Template to be replaced: longResume.template, with data: {personName=Daniel, description=My cv, validUntil=");
  }

...
}

Which components are we testing there? the following image shows them:

Components involved in the integration test

As we can see, we test a lot of components, but, there are some hidden components also, so, let’s detail which they are:

  • Obvious components: TemplateController, TemplateProcess, LongResumeDataAggregator, BasicTemplateRender.
  • Hidden components: MVC (RESTFul API), Tomcat server, HTTP processing, JSON transformations, Spring IoC.

You might think “that is plenty of components, but, what if my application is pretty complex and I cannot test so much components at the same time?” well, you can mock part of the system, as we did in the unit tests. For instance, a typical use case for that is when your application needs to connect to a database, save and query data from there, how do you test that? you can have a real testing database, or you can use a in-memory database like H2 or HSQL, this in-memory database will behave as a mocked database.

Let’s see an example of mocking in an integration tests:

@WebMvcTest(TemplateController.class)
public class TemplateControllerMVCTest {
  @Autowired
  private MockMvc mvc;

  @MockBean
  private TemplateProcess templateProcess;

  @Test
  public void renderResume_userTemplateProcess_render()
          throws Exception {
    given(templateProcess.processTemplate("template1",
            Map
                    .of("ke1", "value2",
                            "key2", "value2")))
            .willReturn("rendered template");

    mvc.perform(MockMvcRequestBuilders
            .put("/templates/template1")
            .content(asJsonString(Map
                    .of("ke1", "value2",
                            "key2", "value2")))
            .contentType(MediaType.APPLICATION_JSON))
            .andExpect(status().isOk())
            .andExpect(content()
                    .string("rendered template"));
  }

...
}

There, we just want to test the TemplateController class, nothing more downstream, so, we mocked the TemplateProcess class in line 12. Which components we are testing? let’s see:

Isolated integration test

We test just one obvious component, but, again, there are some hidden components, so, let’s detail them:

  • Obvious components: TemplateController.
  • Hidden components: MVC (RESTFul API), Tomcat server, HTTP processing, JSON transformations, Spring IoC.

As we can see, the hidden components don’t change at all, just the obvious ones, so, do you think it is worth to test just the TemplateController class? well, I think, it is not. It is better to test a whole flow as we did before, using multiple components. Besides, the integration tests are slow to run, that means, if you create one integration tests by each class/layer in the application, running all is going to take time, as we will discuss as follows.

Speed

Integration tests usually run slowly. The idea is test a lot of components at the same time, so, it is normal that the test is slow, besides, the hidden components add an overhead to the process.

For instance, if we execute the integration tests suite, we get the following.

Running the integration tests

As we can see, we run 4 integration tests in 575 ms. That means, in average, one integration test run in 144 ms. This means, we got slow feedback from the integration tests. We don’t want to run a lot of times the integration tests when we work in a use case, as they are too slow, we get mad.

Of course, integrations tests are vital to validate the correct behavior of an application. The business rules are pretty important, but, the hidden components are too, as they are who puts in motion those rules.

A common practice is separate the integration tests from the unit tests, so, as a developer, I can choose which one I want to run. If you have everything together, you will be mad when the feedback for the tests are pretty slow. For instance, we create a new folder to save the integration tests, named intTests:

intTest folder for integration tests

Besides, if you are using Gradle or Maven, a common practice is to chain both, the unit tests and the integration tests, so, after the unit tests run, integration tests run also. However, if you are using any kind of continuous integration platform like Jenkins, Github actions or Gitlab pipelines, it is better to have a different stage/action to run the integration tests.

Final Thought

Unit and integration tests must be part of any application, but, we should take into account when and how we use one or another. The following table summarize how those two kind of tests behaves:

Unit TestsIntegration Tests
AbstractionKnows each detailHigh level, black box
ScopeFew classes/methodsSeveral components,
hidden components
SpeedFastSlow
Summary of unit tests and integration tests

Do both, but think twice which one to use depending on your needs.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s