Thursday, February 07, 2008

Pod Racing: How to synchronize threads across multiple JVMs

Sometimes, you want to start the job in multiple JVMs only when all the VM has started. This scenario often comes up with multiplayer games. Let's take a pod racing game for example. The players need to start at the same time and each game server handles 1 player (for simplicity's sake)

Starting from Java 5, there's a CyclicBarrier to synchronize threads and is a perfect tool for this. However, it only handles threads in 1 VM. D'oh! Here's where Terracotta comes in. If you make that CyclicBarrier a shared object, Terracotta will handle the clustering for you, transparently (no API). All of a sudden, your CyclicBarrier object will be seen across all the VMs participated in the game.

Here's an example of how to do it. First, let's take a look at the PodRacer class
package demo;

import java.util.concurrent.CyclicBarrier;

public class PodRacer {
public final static int COUNT = 2;
private final CyclicBarrier barrier = new CyclicBarrier(COUNT);
private String name;

public PodRacer(String name) { = name;

public void ready() {
System.out.println(name + ": ready");

public void set() throws Exception {
/* in Terracotta world, all threads in different VMs will block here */

public void go() throws Exception {
System.out.println(name + ": go");
Thread.sleep((int) (Math.random() * 5000) + 10);
System.out.println(name + ": arrived at " + System.currentTimeMillis());

public static void main(String[] args) throws Exception {
PodRacer racer = new PodRacer(System.getProperty("", "unknown"));

Notice I hardcoded number of racers here to 2 but it can be made dynamic. If you run this class just like a normal Java program, it will block at racer.set() call because there's only 1 thread arriving at the barrier whereas it requires 2 for the barrier to be lifted. And if you don't use Terracotta underneath, it doesn't matter how many VMs you start, it will just block there.

Next, I'll add Terracotta into the mix and share the barrier object. That can be achieved by marking it in Terracotta configuration file tc-config.xml

<?xml version="1.0" encoding="UTF-8"?>
<con:tc-config xmlns:con="">
<server host="%i" name="localhost">

And that's it. When I start 2 JVMs running PodRacer class with Terracotta, I will efficiently have 2 threads calling racer.set() on the shared barrier

To make it even easier to try out with Terracotta, there's a Maven 2 plugin that will handle starting up your project with Terracotta enabled. Here's the pom.xml of my project:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="" xmlns:xsi="" xsi:schemaLocation="">

<process nodeName="racer1"
<process nodeName="racer2"

<releases />
<snapshots />

See how I defined 2 racers through the process element in the plugin configuration. That will start 2 distinct VMs with Terracotta enabled. The output looks something like this:

[INFO] Starting DSO nodes
[INFO] Starting node racer1: c:\jdk\jdk1.6.0_02\jre/bin/java.exe -Dtc.nodeName=racer1 -Dtc.numberOfNodes=2 -Dtc.config=d:\work\workspace\projects\PodRacing\tc-config.xml -Dtc.classpath=file:/c:/Users/hhuynh/AppData/Local/Temp/tc-classpath37736.tmp -Dtc.session.classpath=/C:/Users/hhuynh/.m2/repository/org/terracotta/tc-session/2.5.0/tc-session-2.5.0.jar -Xbootclasspath/p:d:\work\workspace\projects\PodRacing\target\dso-boot.jar -cp d:\work\workspace\projects\PodRacing\target\classes; demo.PodRacer
[INFO] Starting node racer2: c:\jdk\jdk1.6.0_02\jre/bin/java.exe -Dtc.nodeName=racer2 -Dtc.numberOfNodes=2 -Dtc.config=d:\work\workspace\projects\PodRacing\tc-config.xml -Dtc.classpath=file:/c:/Users/hhuynh/AppData/Local/Temp/tc-classpath37737.tmp -Dtc.session.classpath=/C:/Users/hhuynh/.m2/repository/org/terracotta/tc-session/2.5.0/tc-session-2.5.0.jar -Xbootclasspath/p:d:\work\workspace\projects\PodRacing\target\dso-boot.jar -cp d:\work\workspace\projects\PodRacing\target\classes; demo.PodRacer
[INFO] ------------------------------------------------------------------------

[INFO] [racer2] Sebulba: ready
[INFO] [racer1] Anakin: ready
[INFO] [racer1] Anakin: go
[INFO] [racer2] Sebulba: go
[INFO] [racer2] Sebulba: arrived at 1202406914274
[INFO] Finished node racer2
[INFO] [racer1] Anakin: arrived at 1202406915125
[INFO] Finished node racer1

Ouch, Anakin sucked!

Download the Maven project and try it out. Have fun.

P.S. More about sharing memory between JVMs here or visit Terracotta


ngonidan said...

Excellent!!!! You hammered right to the marrow!!!

Just a few additions, to save a few hairs on the heads of other programmers

***I could not insert a full version of my additions due to restrictions on this server, which disallowed any special characters****

To compile the above java file, just go to your working directory , the file is in a package so you have to compile and run carefully.

mkdir classes
javac -d classes

To run as an ordinary java file, simply use the java command

java -cp classes demo.PodRacer

** the –cp switch means class search path of directories and zip/jar files*****


unknown: ready

***Note it stops at ready, even if you open another command prompt to run another thread, still they all stop there ****

So how do we synchronize the threads, we use Terracotta.

2) The tc-config.xml file has a few errors due to those multiple back slashes,remove them or use a basic tc-config.xml file found at, it worked fine

And that's it. Now to run the file, but this time NOT as an ordinary java file, but using java environment in Terracotta, open 2 command prompts, and Run the application using the Terracotta dso-java script found again in the Terracotta bin folder:


unknown: ready
unknown: go
unknown: arrived at 1204458149687

***Note use the complete path to dso-java.bat given above, it’s required****
**Also take note of the temporary pause of the first thread waiting for the second one***

**any further questions, get me at *****

Unknown said...

Hi, Nice post. I am new to blogger, I was wondering how you have formatted your source code in blogger?

Unknown said...

I use this tool