Monday, August 24, 2009

Maven gotcha: "post-integration-test" phase

In Maven, "post-integration-test" phase can be used to do test clean up, for example, to shut down Cargo after the tests finished.

If you run "maven install" then the phases would be in this order:


pre-integration-test: start Cargo (web server)
integration-test: start tests
post-integratino-test: stop Cargo


However, if you just run "maven integration-test" to run the tests directly, "post-integration-test" phase is never called, which might lead to your cleanup process not being run.

So the workaround is to bind your clean up tasks to "integration-test" itself which will be run right after the tests run during that phase.

Saturday, August 08, 2009

Maven Javadoc workaround: Unable to find package java.lang in classpath or bootclasspath

Seen this before?


Embedded error: Error rendering Maven report: Exit code: 1 - com.sun.tools.javac.util.FatalError: Fatal Error: Unable to find package java.lang in classpath or bootclasspath
at com.sun.tools.javac.comp.MemberEnter.importAll(MemberEnter.java:123)
at com.sun.tools.javac.comp.MemberEnter.visitTopLevel(MemberEnter.java:509)
at com.sun.tools.javac.tree.JCTree$JCCompilationUnit.accept(JCTree.java:446)
at com.sun.tools.javac.comp.MemberEnter.memberEnter(MemberEnter.java:387)
at com.sun.tools.javac.comp.MemberEnter.complete(MemberEnter.java:819)
at com.sun.tools.javac.code.Symbol.complete(Symbol.java:386)
at com.sun.tools.javac.code.Symbol$ClassSymbol.complete(Symbol.java:758)
at com.sun.tools.javac.comp.Enter.complete(Enter.java:451)
at com.sun.tools.javac.comp.Enter.main(Enter.java:429)
at com.sun.tools.javadoc.JavadocEnter.main(JavadocEnter.java:53)
at com.sun.tools.javadoc.JavadocTool.getRootDocImpl(JavadocTool.java:152)
at com.sun.tools.javadoc.Start.parseAndExecute(Start.java:330)
at com.sun.tools.javadoc.Start.begin(Start.java:128)
at com.sun.tools.javadoc.Main.execute(Main.java:41)
at com.sun.tools.javadoc.Main.main(Main.java:31)
javadoc: error - fatal error


This error has baffled me for the last time. Though I don't understand why the plugin or javadoc would trigger this error, I found a workaround: adding rt.jar to the "bootclasspath":

<bootclasspath>${java.home}/lib/rt.jar</bootclasspath>


<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-javadoc-plugin</artifactId>
<version>2.6</version>
<configuration>
<minmemory>128m</minmemory>
<maxmemory>512m</maxmemory>
<!-- fix error can't find java.lang package -->
<bootclasspath>${java.home}/lib/rt.jar</bootclasspath>
</configuration>
<reportSets>
<reportSet>
<reports>
<report>aggregate</report>
</reports>
</reportSet>
</reportSets>
</plugin>

Tuesday, June 23, 2009

Starting Terracotta Server as a Windows service

There's an easy way to set up Terracotta server as Windows service using the open source Java Service Wrapper.

I've used their Integration Method 1, which uses a wrapper WrapperSimpleApp to run Terracotta server main class com.tc.server.TCServerMain.

First, just download and install the latest version wrapper-windows-x86-32-3.3.5. Then make copy of conf/wrapper.conf and name it TerracottaServer.conf.

Fill out these needed properties:

# for logging
wrapper.java.command.loglevel=INFO

# Method 1 main class
wrapper.java.mainclass=org.tanukisoftware.wrapper.WrapperSimpleApp

# classpath
# (1) is for WrapperSimpleApp
# (2) is for Terracotta Server - You'll need to fix up the path for your own use
wrapper.java.classpath.1=../lib/wrapper.jar
wrapper.java.classpath.2=d:/work/builds/terracotta-3.0.1/lib/tc.jar

# Java Additional Parameters
## NOTE: -server option only works with a JDK, not with JRE
wrapper.java.additional.1=-server
wrapper.java.additional.2=-XX:+HeapDumpOnOutOfMemoryError
wrapper.java.additional.3=-Dcom.sun.management.jmxremote
wrapper.java.additional.4=-Dtc.install-root=d:/work/builds/terracotta-3.0.1

# Initial Java Heap Size (in MB)
wrapper.java.initmemory=512

# Maximum Java Heap Size (in MB)
wrapper.java.maxmemory=512

# Application parameters. This is where you specify TC server main class
wrapper.app.parameter.1=com.tc.server.TCServerMain



That's pretty much all you need. There are Batch scripts in the "bin" folder of the wrapper installation where you can install/uninstall your service. Just make sure you modify those scripts to point to TerracottaServer.conf file you made earlier.

Once you have the service install, you can start/stop the service by using Windows services manager or by using the scripts. There will be a log of the run under "/logs"

I've made a copy of the scripts and the TerracottaServer.conf for easy tryout.

Wednesday, June 03, 2009

Executing a Maven plugin from a Maven plugin

I've searched for a way to call a Maven plugin from inside another plugin but couldn't find any. So I've put on my hacker hat and made it happen. It might not be the best way to do it or even orthodox but here goes.

For example, Terracotta has a Maven plugin, namely "tc" and I want to add a 'help' goal to it which works exactly like the help plugin

With the help plugin, you can do this:
mvn help:describe org.terracotta.maven.plugins:tc-maven-plugin -Ddetail


But it's a little verbose. I want to do
 
mvn tc:help


which would accomplish the same thing. To be able to do that, I have my "tc" plugin depended on "help" by declaring it as a dependency in the pom.

    <dependency>
     <groupId>org.apache.maven.plugins</groupId>
     <artifactId>maven-help-plugin</artifactId>
     <version>2.1</version>
    </dependency> 


Then I looked in the source of the "help" plugin, specifically DescribedMojo.java which handles to goal "help:describe" as shown above. So basically, if I can construct a DescribeMojo instance and fill in the needed info, you'll be able to execute it just like your own mojo. The trick is, DescribeMojo has all of it fields "private" and no setters.
So I hacked the hell out of it and dug into its privates by using reflection. This is a no-no in so many books but I feel I'm working on a known version (2.1) of the "help" plugin so the chance of its API change is slim.

/**
 * Print help for all known goals
 * 
 * @author hhuynh
 * 
 * @goal help
 */
public class HelpMojo extends AbstractMojo {
  .....

  /**
   * @parameter expression="${project.remoteArtifactRepositories}"
   * @required
   * @readonly
   */
  private List remoteRepositories;

  /**
   * The goal you want to see help. By default help prints for all goals
   * 
   * @parameter expression="${goal}"
   */
  private String goal;

  public void execute() throws MojoExecutionException, MojoFailureException {
    DescribeMojo describeMojo = new DescribeMojo();
    setValue(describeMojo, "artifactFactory", artifactFactory);
    setValue(describeMojo, "pluginManager", pluginManager);
    setValue(describeMojo, "projectBuilder", projectBuilder);
    setValue(describeMojo, "project", project);
    setValue(describeMojo, "settings", settings);
    setValue(describeMojo, "session", session);
    setValue(describeMojo, "localRepository", localRepository);
    setValue(describeMojo, "remoteRepositories", remoteRepositories);

    setValue(describeMojo, "plugin", "org.terracotta.maven.plugins:tc-maven-plugin");
    setValue(describeMojo, "detail", true);
    setValue(describeMojo, "goal", goal);
    
    describeMojo.execute();
  }

  private void setValue(Object o, String field, Object value) throws MojoFailureException {
    Class c = o.getClass();
    Field _field;
    try {
      _field = c.getDeclaredField(field);
      _field.setAccessible(true);
      _field.set(o, value);
    } catch (Exception e) {
      throw new MojoFailureException(e.getMessage());
    }
  }
}


The 3 important fields I had to fill out are:

  1. plugin: full name of the plugin you want to print help

  2. detail: I want full description by default

  3. goal: If specified, it only prints help for that goal


There are other component fields that Maven will normally inject them into your own mojo automatically. For example, the field "remoteRepositories" is needed by DescribeMojo and since we construct it by hand, we have to inject the value of this field ourselves. But you can get the value of it for free by Maven so all I need to do is just declaring it and passing it along.

Voila, you got a fully functional DescribeMojo instance and all that left is to call "execute()" on it.

So that's how the hack is done.

Monday, April 27, 2009

Sharing data of Spring beans with Terracotta

This is a quick demo showing how you would share data of Spring beans between different JVMs using Terracotta.

The example is based on the Voting Booth of Spring By Example website (a great website to learn Spring by the way)

In this example, I have 2 beans defined in application-context.xml


<bean id="recorder" class="org.terracotta.demo.LocalVoteRecorder" />

<bean id="votingBooth" class="org.terracotta.demo.VotingBooth">
<property name="voteRecorder" ref="recorder" />
</bean>


So simply, the 'votingBooth' bean has a reference to the 'recorder' bean which has a map of candidates and their votes.


public class VotingBooth {
VoteRecorder recorder = null;

public void setVoteRecorder(VoteRecorder recorder) {
this.recorder = recorder;
}

public void vote(Candidate candidate) {
recorder.record(candidate);
}

public void printResult() {
recorder.tallyVotes();
}
}

/* ************* */
public class LocalVoteRecorder implements VoteRecorder {
final static Log logger = LogFactory
.getLog(LocalVoteRecorder.class);
private final Map<Candidate, Integer> votesRecord = new ConcurrentHashMap<Candidate, Integer>();

public void record(Candidate candidate) {
int count = 0;

if (!votesRecord.containsKey(candidate)) {
votesRecord.put(candidate, count);
} else {
count = votesRecord.get(candidate);
}
count++;
votesRecord.put(candidate, count);
}

public void tallyVotes() {
for (Candidate candidate : votesRecord.keySet()) {
logger.info("Candidate '" + candidate + "' has "
+ votesRecord.get(candidate) + " votes.");
}
}
}


Spring will do its magic and inject the right data to into main program at runtime. To demo, I have the main function vote 5 times between 2 known candidates then print out the result in the end.


public class Main {

final static Log logger = LogFactory.getLog(Main.class);
final static Random rand = new Random();
final static Candidate[] candidates = new Candidate[] {
new Candidate("Hung"), new Candidate("Nick") };

public static void main(String[] args) {
ApplicationContext applicationContext = new ClassPathXmlApplicationContext(
"/application-context.xml");

VotingBooth votingBooth = (VotingBooth) applicationContext
.getBean("votingBooth");

// vote 5 times
for (int i = 0; i < 5; i++) {
Candidate luckyCandidate = candidates[rand.nextInt(2)];
votingBooth.vote(luckyCandidate);
logger.info("voted for candidate " + luckyCandidate);
}

votingBooth.printResult();
}
}


So if you run this Spring app as is, it will print the result of that run and exit. Quite simple.

Now let say we want to maintain a voting record that would last beyond the life cycle of that run. And also, any subsequent runs should add to the vote tally. In other words, the data is shared between the nodes (JVMs) and persisted.

This is where Terracotta comes in. The data we're interested in sharing will be marked as "ROOT". In this case, it's the "votesRecord" field of the 'recorder' bean. This field will be transparently clustered by Terracotta, its value will be saved in Terracotta server and it will be faulted back into a new node as needed. There's no thirdparty function calls nor networking code, just plain Java. Here is what needed in Terracotta configuration file tc-config.xml:

<application>
<dso>
<instrumented-classes>
<include>
<class-expression>org.terracotta.demo.*</class-expression>
</include>
</instrumented-classes>
<roots>
<root>
<field-name>org.terracotta.demo.LocalVoteRecorder.votesRecord</field-name>
</root>
</roots>
</dso>
</application>


That piece of config tells Terracotta to instrument all classes under the org.terracotta.demo package and make "votesRecord" a root.

Since Terracotta has Maven plugin, I just use its plugin and run the example. The pom is here if you're interested. First, you need to start Terracotta server:

% mvn tc:start

Then you can run the example with Terracotta enable simply by:

% mvn tc:run

On first run, I got this result: Nick got voted twice


[INFO] [vote] INFO [main] voted for candidate Nick
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] voted for candidate Nick
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] Candidate 'Hung' has 3 votes.
[INFO] [vote] INFO [main] Candidate 'Nick' has 2 votes.
[INFO] Finished node vote
[INFO] DSO processes finished
[INFO] Skipping stopping DSO Server
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESSFUL
[INFO] ------------------------------------------------------------------------


On second run: Nick got voted one more time brings his total votes to 3


[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] voted for candidate Nick
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] voted for candidate Hung
[INFO] [vote] INFO [main] Candidate 'Hung' has 7 votes.
[INFO] [vote] INFO [main] Candidate 'Nick' has 3 votes.


This shows that the votes are indeed shared between 2 separate runs. To see the shared data live, you can use Terracotta Developer Console to look at the roots:

Run this Maven command to start the console:

% mvn tc:dev-console



You can check out the whole project from Subversion: http://svn.terracotta.org/svn/forge/projects/TerracottaSpringDemo

Tuesday, April 14, 2009

Create unique temp filename with Batch script

For Windows batch scripts, you could use %RANDOM% to generate a random number and use that as part of the temp file name but that doesn't guarantee uniqueness since random number can be repeated. By adding another variable to the mix, seconds and fraction of seconds, the file name would be unique:



call :GETTEMPNAME
echo "Temp file name is %TMPFILE%"
goto :EOF

:GETTEMPNAME
set TMPFILE=%TMP%\mytempfile-%RANDOM%-%TIME:~6,5%.tmp
if exist "%TMPFILE%" GOTO :GETTEMPNAME

:EOF

Monday, January 19, 2009

Using Ant as a library

Ant has a lot of predefined tasks that could be a great help. You can use them directly from your Java code. Here are some examples using Ant to download a file, unzip a package and exec some shell command.

You would need to have
+ ant-1.7.1.jar
+ ant-launcher-1.7.1.jar
+ commons-io-1.3.2.jar (for exec example, found at http://commons.apache.org/io)

in your classpath.


public class AntDemo {

/**
* Download a file at sourceUrl to dest
*
*/
public static void download(URL sourceUrl, File dest) {
Project dummyProject = new Project();
dummyProject.init();

Get antGet = new Get();
antGet.setProject(dummyProject);
antGet.setVerbose(true);
antGet.setSrc(sourceUrl);
antGet.setDest(dest);
antGet.execute();
}

/**
* Unzip a zip file
*
*/
public static void unzip(File src, File dest) {
Project dummyProject = new Project();
dummyProject.init();

Expand antUnzip = new Expand();
antUnzip.setProject(dummyProject);
antUnzip.setSrc(src);
antUnzip.setDest(dest);
antUnzip.execute();

/* ant doesn't preserve permission bits
need to restore them manually */

Chmod chmod = new Chmod();
chmod.setProject(dummyProject);
chmod.setDir(new File(src.getAbsolutePath().replaceAll(".zip", "")));
chmod.setPerm("a+rx");
chmod.setIncludes("**/**");
chmod.execute();
}


/**
* Run a shell command and return the output as String
*
*/
public static String exec(String command, List params, File workDir) {
File outputFile;
try {
outputFile = File.createTempFile("exec", ".out");
} catch (IOException e) {
throw new RuntimeException(e);
}

Project dummyProject = new Project();
dummyProject.init();

ExecTask execTask = new ExecTask();
execTask.setProject(dummyProject);
execTask.setOutput(outputFile);
execTask.setDir(workDir != null ? workDir : new File(System
.getProperty("user.dir")));
execTask.setExecutable(command);
if (params != null) {
for (String param : params) {
execTask.createArg().setValue(param);
}
}

execTask.execute();

FileReader reader = null;
try {
reader = new FileReader(outputFile);
return IOUtils.toString(reader);
} catch (Exception e) {
throw new RuntimeException(e);
} finally {
IOUtils.closeQuietly(reader);
outputFile.delete();
}
}

public static void main(String[] args) {
List params = Arrays.asList(new String[] { "Hello", "World" });
System.out.println(exec("echo", params, null));
}
}


Just note that you'll need a dummy project for the Ant task. Otherwise you'll get an NullPointerException.

There are a lot of tasks that you could use. The list is here http://ant.apache.org/manual/tasksoverview.html

Sunday, January 04, 2009

Java Webstart Demos with Terracotta enabled

For easy deployment, Terracotta enabled Java applications can also be launched via web start. There are certain issues to look out for but it's certainly possible. I wrote a small demo to demonstrate it.

Issue 1: Terracotta installation required on user's computer before running.

This is solved easily by downloading a Terracotta zip file and extract it to a known location on user computer. In my demo, I install it to $HOME/terracotta_ws.

Issue 2: Java Webstart applications don't allow modification of boot classpath.

Since you can't append or pre-append to boot classpath of the webstart VM, which Terracotta requires, you are forced to spawn an external JVM for your app. This is solvable by signing your jars with a keystore, that will allow your webstart app to spawn external process. This is a good blog showing how to sign your jars.

Note that only jars that are part of your main webstart GUI frame and its libraries are required to be signed. Jars that are part of your Terracotta applications are not required.

When I use "webstart app", I mean the control app that is downloaded and run by Java webstart.

Wehn I use "main app" or "TC enabled app", I mean the Terracotta enabled applications that are spawned in external processes by the "webstart app"

Issue 3: You can't declare your main application jars as part of resources to download in JNLP file.

The main reason is that Java Webstart obfuscates those jars and stores them in a cache with variable folder name. So your webstart app has to handle the downloading and installing the main app's binaries (zip file) to a known location. This is also doable. Take a look at the JNLP file I have:

<jnlp spec="1.0+" codebase="http://localhost:8080/webstartdemo"
href="demo.jnlp">
<information>
<title>TC webstart demo</title>
<vendor>Terracotta</vendor>
<description>Demo of how to start Terracotta from webstart
</description>
<description kind="short">TC Webstart example</description>
<offline-allowed />
</information>
<resources>
<j2se version="1.5+" />
<jar href="TCWebStartDemo.jar" download="eager" />
<jar href="ant-1.7.1.jar" />
<jar href="ant-launcher-1.7.1.jar" />
<jar href="miglayout-3.6.2-swing.jar" />

<property name="tcws.codebase" value="http://localhost:8080/webstartdemo" />
<property name="tcws.terracotta.zip" value="terracotta-trunk-nightly-rev11190.zip" />
<property name="tcws.demo.list" value="sharededitor,mandelbrot" />
<property name="tcws.sharededitor.mainClass" value="demo.sharededitor.Main" />
<property name="tcws.mandelbrot.mainClass" value="mandelbrot.Main" />

</resources>
<application-desc main-class="demo.MainFrame" />
<security>
<all-permissions />
</security>
</jnlp>

From line 13 to 16, those jars are for the webstart app which will be downloaded automatically by java webstart. And those are the ones that need to be signed.
Line 25 indicates the main class of the webstart app.

From line 18 to 22, that is how I communicate to my webstart app (via system properies) about where to download Terracotta and the demos (Terracotta enabled apps). Each demo is a zip file which contains a tc-config.xml, a "lib" folder with all the jars it requires.

You can play with the demo here.

The webstart app Eclipse project can be checked out at SVN repo http://svn.terracotta.org/svn/forge/projects/labs/TCWebStartDemo

Your main interests should be the 2 classes:


Enjoy the demos :)

Friday, January 02, 2009

Redirecting System.out and System.err to JTextPane or JTextArea

In Swing, if you want to redirect System.err and System.out to a JTextPane or a JTextArea, you only need to override the write() methods of OutputStream to append the text to the text pane instead.

The example below shown how to do it with JTextPane. For JTextArea, it's quite simpler, just call JTextArea.append()

This is useful when you spawn external process from a Swing application and want to see its output in your own text component.

Jan 04 2009: Revised regarding Eric Burke's comment below.
JTextPane version:
  private void updateTextPane(final String text) {
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        Document doc = textPane.getDocument();
        try {
          doc.insertString(doc.getLength(), text, null);
        } catch (BadLocationException e) {
          throw new RuntimeException(e);
        }
        textPane.setCaretPosition(doc.getLength() - 1);
      }
    });
  }

  private void redirectSystemStreams() {
    OutputStream out = new OutputStream() {
      @Override
      public void write(final int b) throws IOException {
        updateTextPane(String.valueOf((char) b));
      }

      @Override
      public void write(byte[] b, int off, int len) throws IOException {
        updateTextPane(new String(b, off, len));
      }

      @Override
      public void write(byte[] b) throws IOException {
        write(b, 0, b.length);
      }
    };

    System.setOut(new PrintStream(out, true));
    System.setErr(new PrintStream(out, true));
  }
JTextArea version:
  private void updateTextArea(final String text) {
    SwingUtilities.invokeLater(new Runnable() {
      public void run() {
        textArea.append(text);
      }
    });
  }

  private void redirectSystemStreams() {
    OutputStream out = new OutputStream() {
      @Override
      public void write(int b) throws IOException {
        updateTextArea(String.valueOf((char) b));
      }

      @Override
      public void write(byte[] b, int off, int len) throws IOException {
        updateTextArea(new String(b, off, len));
      }

      @Override
      public void write(byte[] b) throws IOException {
        write(b, 0, b.length);
      }
    };

    System.setOut(new PrintStream(out, true));
    System.setErr(new PrintStream(out, true));
  }

[+] Old versions