This post demonstrates an easy way to test the effect of multiple concurrent threads running your software, using your existing unit tests.

Have you ever wondered whether your software is really thread safe? Whether it will stand up to the punishment of thousands of concurrent users when your site (or app) goes ballistic? Multi-threaded programming is notoriously difficult, and running software in a environment (like a web application server) that spawns multiple threads can often expose architectural problems that don’t arise in typical test scenarios. Running a set of tests in parallel is a good way to gain confidence that your code is thread safe.

Another benefit of running multiple tests at once is: they run faster! All modern computers have multiple cores, and many have multiple CPUs: when we run our tests single-threaded, we aren’t making use of all of that latent power we have just lying around.

Let’s say you have a test class called MyTestClass, and it defines a number of tests. Using the test runner we provide, you can run all of its tests in parallel by adding a single annotation (the standard org.junit.runner.RunWith annotation that comes with JUnit) to your class:

@RunWith (MultiThreadedRunner.class)

This runner plugs in to the JUnit framework by subclassing BlockJUnit4ClassRunner; this is the class that usually runs all tests from a test class. The runChild() method is called for each test that is run; we take that over and arrange for each test to run in its own thread. We also want to ensure that not too many threads run at once: each thread consumes memory, and, depending on the size of the test class, we may end up running hundreds of threads at once if we’re not careful, and run out of memory. Here is the code for runChild, which simply waits until there are fewer than maxThreads tests running, and then creates a Runnable called Test which actually runs the test:

    protected void runChild(final FrameworkMethod method, final RunNotifier notifier) {
        while (numThreads.get() > maxThreads) {
            try {
                Thread.sleep(25);
            } catch (InterruptedException e) {
                System.err.println ("Interrupted: " + method.getName());
                e.printStackTrace();
                return; // The user may have interrupted us; this won't happen normally
            }
        }
        new Thread (new Test(method, notifier)).start();
    }
 

Note that we keep track of the number of threads in a variable called numThreads. That is an AtomicInteger, which is a thread-safe primitive built into the standard JRE. We use it to ensure that the thread count isn’t updated simultaneously in two threads. Here is the core of the code for the Test class:

        public void run () {
            numThreads.incrementAndGet();
            MultiThreadedRunner.super.runChild(method, notifier);
            numThreads.decrementAndGet();
        }

All this does is keep track of the number of running threads. I haven’t shown the constructor and members used to track the test method and notifier, but as you can imagine, that is just straightforward copying of variables.

The only slight complication with using the code as shown so far is that JUnit will finish before all the tests do. It’s necessary to make the “outer loop” in the test runner wait until the last test has completed before it exits. Usually this happens implicitly, because tests are all run in the same thread, but now, when the runner starts a test, it returns immediately, while the test is still running. To solve this problem, we need to override the childrenInvoker method.

protected Statement childrenInvoker(final RunNotifier notifier) {
        return new Statement() {
            public void evaluate() throws Throwable {
                MultiThreadedRunner.super.childrenInvoker(notifier).evaluate();
                // wait for all child threads (tests) to complete
                while (numThreads.get() > 0) {
                    Thread.sleep(25);
                }
            }
        };
    }

Couldn’t be simpler: just call the super method to do all the real work, and then wait until there are no more test threads running before returning. Does anybody see the potential race condition here? It’s possible that when the last test is run, childrenInvoker will return and test numThreads before that last test has a chance to increment it. In practice this doesn’t seem to happen since there will generally be several threads running already when the last test is started, but just to be safe, it is better to increment the threadCount in the main runner thread, just before calling Thread.start(), and then to decrement it in the child thread, just before exiting. The attached file has that change.

Download the source code here:
MultiThreadedRunner

Note that this code is distributed under the Mozilla Public License (2.0), which basically says you can use this code freely, embed it in your software, and even redistribute it, as long as it retains its license and attribution (includes the comments it has now saying who wrote it), and as long as any changes you make to the software are distributed under the same terms: also I encourage you to post any changes you make here so I can incorporate them.