Bazel models tests as a special case of running programs, where the exit code matters.
So you can think of bazel test my_test as syntax sugar for bazel build my_test && ./bazel-bin/my_test
The "program" run is usually a "Test Runner" from your language ecosystem, such as JUnit, pytest, mocha, etc. This is unlike most build systems, where the developer interacts directly with the test runner CLI.
However, it can also be a shell script or any other program you write. With Bazel, it's often useful to write your own Test Runner rather than build your test in some existing test/assertion framework since Bazel handles all the mechanics of including your test in the build process.
The encyclopedia
When Bazel’s interaction with the test runner doesn’t do what you expected, you may need to consult the Test Encyclopedia, described as "an exhaustive specification of the test execution environment" (and they really mean that).
This contract between Bazel and your test runner process will often resolve a dispute over why your test isn't working the way you expect under Bazel.
External services
Tests often want to connect to services/datasources as part of the "system under test". With some CI tools or custom scripts, you might do this outside the build tool, like so:
- Start up some services or populate a database
- Run the entry point for the testing tool
- Clean up
Bazel does not support this model. Bazel tests are just programs that exit 0 or not, and Bazel has no "lifecycle" hooks to run some setup or teardown for specific test targets.
You could script around Bazel, the same as in the scenario above, by starting some services before running bazel test and then shutting them down at the end.
However, this doesn't work well with remote execution. It also assumes that concurrent tests will be isolated from each other when accessing the shared resource. It means you startup the services even if Bazel doesn't execute any tests because they are cache hits.
Ideally, tests are hermetic. That means they depend only on declared inputs, which are files. If a test needs to connect to a service, you could invert the above model - the testing tool runs the test, which sets up the environment and tears it down. Testcontainers is a great library for using Docker containers as a part of the system under test. https://github.com/dzbarsky/rules_itest is also a good option to explore.
