Scaling testcontainers with Bazel - part 0 - introducing testcontainers

What is integration testing?

Sometimes controversy is all you need to start and what could be more controversial in the software engineering world than clearly defining a testing methodology? How about we all agree on defining integration testing as "All testing involving more than a single unit (be it a class, a function or whatever) working together to create a complex system, but not complex enough that you need a diagram to understand it"?

If it's just a class or a method or a function you won't call it an integration test but instead a unit test. On the other hand, if looking at the underlying code does not explain the test, then you might be looking at an end-to-end or system test. In a way, those last two categories are still testing integrations, but to such a high level that your codebase is barely relevant.

So, why care about integration testing at all you might ask? Well, with the rise of microservices, it has become more and more common to have barely any business logic in an application and rather have most of the code just serve as "glue" between components and so testing the glue barely serves any purpose without the elements it's glueing together.

What is testcontainers?

This paradigm shift affected our view on tests and now the famous testing pyramid has sort of turned upside down. Naturally, developers won't re-implement the whole stack for each test and so we see the rise of libraries like testcontainers.

The goal of the library is to provide its users with a simple way to run and manage ephemeral containers for integration tests. Let's say that service-a needs an OurSql database, you can just use testcontainers, provide it with an image and plug it in a test, exposing ports and binding volumes as needed to simulate a production-like environment. In some cases, if you need to use a well-known container, testcontainers may even provide simpler abstraction and you won't even have to think about providing an image or setting it up.

But does this approach scale well? Can we build something that leverages Bazel's functionalities to do as little work as possible and do that fast? Well, this is pretty much the goal of this series, finding out the problems with testcontainers, if it's possible to make it faster and how to build a better integration with the bazel ecosystem, hopefully producing a working ruleset in the process.