Friday, November 06, 2009

Unique maven surefire plugin configurations for individual tests




Here's what I want to do:

I have a maven project, which has got multiple unit-tests, db-tests (e.g. DAO tests) and integration/system tests. One of the unit tests require high memory (e.g. 512 MB), but other unit tests runs fine with the default heap size (and doesn't need large heap).
I want to run all unit tests by default in the test phase of the build cycle, run the db-tests only when I know a db is present and run the system tests in the integration phase of the build cycle.

Lets assume some names for the tests:

1) Names of all unit tests end with Test.java, e.g. ClientTest.java, SomeLogicTest.java etc
2) The name of the unit test that require high-memory is TestTheWorldWithBigMemoryTest.java. Note that this is not a system tests, its a unit test and just that it requires large heap to run.
3) All db-tests end with DaoTest.java, e.g. UserDaoTest.java, CatalogDaoTest.java etc
4) All system/integration tests end with SystemTest.java, e.g. AddToShoppingCartSystemTest.java, PurchaseSystemTest.java etc

OK, so here's how to configure your pom to do it:



<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>example</groupId>
<artifactId>MySampleProject</artifactId>
<packaging>jar</packaging>
<version>0.0.1-SNAPSHOT</version>
<name>MySampleProject</name>
<url>http://www.abhisanoujam.blogspot.com/</url>
<dependencies>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>3.8.1</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
<plugins>

<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<inherited>true</inherited>
<configuration>
<!--
Skip tests by default. Will be using different executions
setup to run different set of tests with different
configurations
-->
<skip>true</skip>
<!-- other default configuration for all the tests, just an example -->
<reportFormat>plain</reportFormat>
<systemProperties>
<property>
<name>some.property.used.by.tests</name>
<value>what.you.want.value</value>
</property>
</systemProperties>
</configuration>
<executions>
<execution>
<!-- run all tests except for system tests and the big memory test -->
<id>test-phase-execution</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/*Test.java</include>
</includes>
<excludes>
<!-- exclude inner classes -->
<exclude>**/*$*</exclude>
<!-- exclude the test that need large heap -->
<exclude>**/TestTheWorldWithBigMemoryTest.java</exclude>
<!-- exclude the system-tests -->
<exclude>**/*SystemTest.java</exclude>
<!-- exclude the db-tests -->
<exclude>**/*DaoTest.java</exclude>
</excludes>
</configuration>
</execution>
<execution>
<!-- Run tests with 512 MB heap -->
<id>large-heap-test-execution</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<!--
You can even follow some pattern for these kind of
tests and use the pattern here
-->
<!-- For example, **/*BigMemoryTest.java -->
<include>**/TestTheWorldWithBigMemoryTest.java</include>
</includes>
<excludes>
<exclude>**/*$*</exclude>
</excludes>
<argLine>-Xms512m -Xmx512m</argLine>
</configuration>
</execution>
<execution>
<!-- Run the system tests in the integration-phase -->
<id>system-tests-execution</id>
<phase>integration-test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<include>**/*SystemTest.java</include>
</includes>
<excludes>
<exclude>**/*$*</exclude>
</excludes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>

<profiles>

<!-- Profile for running database tests -->
<profile>
<id>test-db</id>
<build>
<plugins>
<plugin>
<artifactId>maven-surefire-plugin</artifactId>
<groupId>org.apache.maven.plugins</groupId>
<configuration>
<!--
Skip tests as the other tests have been already executed
in the "test" and "integration-test" phases
-->
<skip>true</skip>
</configuration>
<executions>
<execution>
<id>db-test-execution</id>
<phase>test</phase>
<goals>
<goal>test</goal>
</goals>
<configuration>
<skip>false</skip>
<includes>
<!-- We only need to include the db tests here -->
<include>**/*DaoTest.java</include>
</includes>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</profile>

</profiles>

</project>





I'll try to explain in words (not xml ;-) ) what I did above:
By default, maven runs the "test" goal of maven surefire plugin. First we tell it to skip tests by adding the plugin in the <build> section of the pom and specifying <skip>true</skip> in the main configuration of the plugin. Here, you can specify other configurations that you want for your tests as a whole.
Then we specify other executions for the plugin and play around with the include/exclude pattern and the configurations for each setup. We can setup multiple executions and also have different set of configurations for each execution setup. Just as a note, adding multiple <plugin> sections for the same plugin in the build section does not work, in fact this is the sole reason of this blog post, and to figure out how to have different configurations depending on your requirements for the same plugin.
In the "test-phase-execution", we include all tests with the **/*Test and exclude the big memory test, dao tests and the system tests.
For the unit-test that requires large heap, its just a matter of adding <argLine>-Xms512m -Xmx512m</argLine> in the configuration for that execution setup. Also we include only that test to run in the include tag.
We bind another execution in the "integration-test" phase to run the system tests by matching the include pattern to only system tests.

We add another profile "test-db", which we can use when we know we have a database is up and running (you don't want your dao tests to fail all the time during development when you don't have a DB running in your environment). In the profile, we again set up the maven-surefire plugin to execute only the dao tests by playing around with the include/exclude pattern. You can activate this profile whenever you want to run the dao tests.

You can check-out a very simple project from here and see the above pom in action.

Here's how you would run them:

mvn clean package
-- This will run all the unit-tests (including the big memory test) but not the system tests and the dao tests

mvn -Ptest-db clean package
-- This will the all the unit tests (like above) and also run the dao tests. This still excludes the system test.

mvn clean install
-- This will run all the unit tests + the system tests. This won't run the dao tests.

mvn -Ptest-db clean install
-- This will run all the tests -- unit tests, big memory test, dao tests and the system tests

Regarding the system/integration tests, instead of doing like above, you can separate all your system-tests in a separate sub-module too, which I guess is more appropriate when your project is kind of large.

Enjoy...

Tuesday, August 25, 2009

... and you say you don't know what's going on in Second-level cache!!



Here it is..

Terracotta 3.1 just got released!! The developer console that comes with this release is ... just awesome!! If you use hibernate and second-level cache... and you had been wondering what's actually going on in your second-level cache.. how many hits, misses, puts... and all those statistical information that you need for your app... those days are over.. well at least if you are using Terracotta :) !! The new console gives you total visibility of your second-level cache...

When you open up the dev-console, a new "node" with the name "Hibernate" comes up in the left side tree view of the developer console (when you use second-level cache backed by Terracotta, of course). There you have two views/tabs.. "Hibernate" and "Second level Cache" (the two buttons on top right hand side, as of this release). Hibernate tab gives you stats like how many loads/updates/inserts/deletes etc of your domain entities and collections have happened etc. If you go over to the "Second-level Cache" view, there are tons of other information there. There are 3 sub-views here.. Overview, Statistics and Configuration.

Overview shows you how much hits/misses/puts are happening in your cache .. globally (all your cache regions) and per region-wise too. The cool part is that it is real-time.. the picture below might not show it, but its a real-time histogram...
In the Statistics panel, you've got 4 charts showing the cache hit ratio, # of cache hit/miss per sec, the # of sql queries happening (this is different from the actual number of queries you get from hibernate statistics, which gives only count of HQL), and the rate at which puts are happening. So if you had ever been wondering if your second-level cache is actually working or not, you can now *prove* its working -- you should see 100% cache hit ratio, no cache misses happening, no queries going to the DB etc. And of course, if these values are not what you expect you should go back and look what's actually going on in your app... the point is you get all those info that you had been wishing for!! isnt' tht sweet... :)

Also you can view # hits, # misses, hit ratio etc region wise... which is like the best visibility you can get for your second level cache!! :)
You'll have to try out the new dev-console to see what's actually in store for you...

Check out some of these screenshots...









Tuesday, July 07, 2009

Having fun with jquery -- Numbers to words

Having some fun with jquery, its cool btw...
Type in some numbers in the input box, (it should be in focus when u load this page, if not well u know who to blame - jquery of course, not me :-P)
And if you (un)knowingly type in something other than a number, it should tell you whats wrong.
Then click "Convert to words" and you'll get your number in words... one in Indian number system and another one in international standard. And, if you are really trying this out, you can use keyboard enter, escape.. i bound them to calculate and hide the results etc.

Just some plain short fun :)

[you can use enter and escape]
Enter a number:


And yeah, of course, here's the code (use this link to convert ur javascript into blogger friendly code):


<script type="text/javascript" src="http://abhi-sanoujam-blogspot-posts.googlecode.com/svn/trunk/js/jquery-1.3.2.min.js"></script>
<script type="text/javascript">
function NumberToWords() {

var units = [ "Zero", "One", "Two", "Three", "Four", "Five", "Six",
"Seven", "Eight", "Nine", "Ten" ];
var teens = [ "Eleven", "Twelve", "Thirteen", "Fourteen", "Fifteen",
"Sixteen", "Seventeen", "Eighteen", "Nineteen", "Twenty" ];
var tens = [ "", "Ten", "Twenty", "Thirty", "Forty", "Fifty", "Sixty",
"Seventy", "Eighty", "Ninety" ];

var othersIndian = [ "Thousand", "Lakh", "Crore" ];

var othersIntl = [ "Thousand", "Million", "Billion", "Trillion" ];

var INDIAN_MODE = "indian";
var INTERNATIONAL_MODE = "international";
var currentMode = INDIAN_MODE;

var getBelowHundred = function(n) {
if (n >= 100) {
return "greater than or equal to 100";
};
if (n <= 10) {
return units[n];
};
if (n <= 20) {
return teens[n - 10 - 1];
};
var unit = Math.floor(n % 10);
n /= 10;
var ten = Math.floor(n % 10);
var tenWord = (ten > 0 ? (tens[ten] + " ") : '');
var unitWord = (unit > 0 ? units[unit] : '');
return tenWord + unitWord;
};

var getBelowThousand = function(n) {
if (n >= 1000) {
return "greater than or equal to 1000";
};
var word = getBelowHundred(Math.floor(n % 100));

n = Math.floor(n / 100);
var hun = Math.floor(n % 10);
word = (hun > 0 ? (units[hun] + " Hundred ") : '') + word;

return word;
};

return {
numberToWords : function(n) {
if (isNaN(n)) {
return "Not a number";
};

var word = '';
var val;

val = Math.floor(n % 1000);
n = Math.floor(n / 1000);

word = getBelowThousand(val);

if (this.currentMode == INDIAN_MODE) {
othersArr = othersIndian;
divisor = 100;
func = getBelowHundred;
} else if (this.currentMode == INTERNATIONAL_MODE) {
othersArr = othersIntl;
divisor = 1000;
func = getBelowThousand;
} else {
throw "Invalid mode - '" + this.currentMode
+ "'. Supported modes: " + INDIAN_MODE + "|"
+ INTERNATIONAL_MODE;
};

var i = 0;
while (n > 0) {
if (i == othersArr.length - 1) {
word = this.numberToWords(n) + " " + othersArr[i] + " "
+ word;
break;
};
val = Math.floor(n % divisor);
n = Math.floor(n / divisor);
if (val != 0) {
word = func(val) + " " + othersArr[i] + " " + word;
};
i++;
};
return word;
},
setMode : function(mode) {
if (mode != INDIAN_MODE && mode != INTERNATIONAL_MODE) {
throw "Invalid mode specified - '" + mode
+ "'. Supported modes: " + INDIAN_MODE + "|"
+ INTERNATIONAL_MODE;
};
this.currentMode = mode;
}
}
}

function clear() {
$("#errSpan").hide();
$("#resultDiv").hide();
}

var num2words = new NumberToWords();

function translate() {
clear();
var input = $("#input").val();
if (isNaN(input)) {
$("#errSpan").html("This is not a number - " + input);
$("#errSpan").show();
$("#input").focus();
return;
};

num2words.setMode("indian");
var indian = num2words.numberToWords(input);

num2words.setMode("international");
var intl = num2words.numberToWords(input);

$("#resultDiv").html(
"<table bgcolor='#CCFFFF'><tr><td>In India</td><td>" + indian
+ "</td></tr><tr><td>Internationally</td><td>" + intl
+ "</td></tr></table>");
$("#resultDiv").show("slow");

}

$(document).ready( function() {
$("#resultDiv").hide();
$("#input").focus();
$(document).keypress( function(e) {
if (e.keyCode == 27) {
clear();
};
if (e.keyCode == 13) {
translate();
};
});
});
</script>

<div id="content" align="center">[you can use enter and escape]<br />
<span id="errSpan" style="color: #FF0000;"></span>
<div>Enter a number: <input id="input" type="text" size="15" /><input
type="button" onclick="translate()" value="Convert to words" /></div>

<div id="resultDiv" style="border: solid black 1px;"></div>
</div>

Friday, June 26, 2009

Meet Mac's "sandbox-exec"

I was working on something to block outgoing connections for one particular app *only*. And I use a mac, leopard... the first place I went to look was System Preferences -> Security -> Firewall pane. And I was disappointed to find no "Block Outgoing Connections"
It can only block incoming connections. Anyway...

Then I met sandbox-exec :) It runs apps in a sandbox, which you can define to restrict access to different resources, network, file-systems etc Here's a simple sandbox file which denies network access...



(version 1)
(allow default)
(deny network*)



Running sandbox-exec with the above file will deny all network access to your app, both outgoing and incoming. Here's a screenshot example showing running bash with and without it:



$ ping -t3 google.com
PING google.com (74.125.67.100): 56 data bytes
64 bytes from 74.125.67.100: icmp_seq=0 ttl=49 time=368.312 ms
64 bytes from 74.125.67.100: icmp_seq=1 ttl=49 time=318.240 ms
64 bytes from 74.125.67.100: icmp_seq=2 ttl=49 time=319.793 ms

--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 318.240/335.448/368.312/23.247 ms
$
$
$ cat block-network.sb
(version 1)
(allow default)
(deny network*)
$
$ sandbox-exec -f block-network.sb /bin/bash
$ ping -t3 google.com
bash: /sbin/ping: Operation not permitted
$ ping -t3 yahoo.com
bash: /sbin/ping: Operation not permitted
$ ping -t3 apple.com
bash: /sbin/ping: Operation not permitted
$ exit
exit
$ ping -t3 google.com
PING google.com (74.125.67.100): 56 data bytes
64 bytes from 74.125.67.100: icmp_seq=0 ttl=49 time=321.997 ms
64 bytes from 74.125.67.100: icmp_seq=1 ttl=49 time=321.350 ms
64 bytes from 74.125.67.100: icmp_seq=2 ttl=49 time=321.676 ms

--- google.com ping statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max/stddev = 321.350/321.674/321.997/0.264 ms
$




If you want to write more sophisticated sandbox files, you should probably check out the files present in /usr/share/sandbox/


Enjoy...

Wednesday, June 24, 2009

bash command line editing + custom inputrc -> efficient editing



Command line editing in bash is sometimes tiring when you have long commands accepting multiple arguments etc. Its hard to edit the command line when you want to change some parameter here and there... adding some parameters in the middle or even modifying existing parameters. I've seen people mostly using the arrow keys to go back and forth and deleting everything and typing in again, or sometimes using copy-paste from previous lines for editing those long long commands. I used to do it myself...


But then you can set command line editing modes in bash. This is where it helps a lot in editing those long long commands that you sometimes run.

The default mode is set to emacs, and you can choose between using the default or using vi mode. If you want to switch to vi mode, use the following:


$ set -o vi



In vi mode, you can edit the command line as you would normally edit using vi. Press ESC in the command line to switch to vi command mode. And you can go forward editing with all vi habits you got... moving forward-word, backward-word, delete, cut, paste...etc. Typing "i" in command mode will bring you back in vi insert mode.

If you are not into vi, you can switch back to emacs mode using:



$ set -o emacs




I have never used emacs myself and being a fan of vi, I used to have vi mode in bash. But I was never happy with that mode.... seemed like too many keystrokes needed to switch between command/insert mode. Also some of the default shortcuts didn't work in vi mode like control-a to move to beginning of line etc.

Using emacs mode, you don't need to switch between command/insert mode like in vi. One disadvantage is I don't know emacs and don't wanna learn :)

So here's I came up with - default emacs mode and a custom key mapping for most of the common editing functions. Yes, you can specify custom key mappings using a .inputrc in your home directory. You should probably go here if you want to find out about how to edit/manage your inputrc file....

I guess the most common editing functions that ppl would mostly care about would be (well its true for me at least)

  • Moving forward and backwards by character: Use left/right arrow keys. I guess am too used to using the arrow keys

  • Deleting backward and forward characters: Use backspace and delete keys of course, again too used to it

  • Moving forward and backwards by word: I didn't like the default mapping of using ESC+b and ESC+f. Felt kind of lazy reaching out for that far-off ESC key :) .... so I changed it to use control-b and control-f instead

  • Moving to beginning and end of line: Use the default mappings, control+a to move to beginning and control+e to move to end

  • Deleting everything from cursor to beginning: Use default mapping of control+u

    When you feel like deleting whole of line, use control-e controle-u, i.e. goto end of line and delete everything upto beginning of command

  • Deleting one word backward from cursor: Use default control+w

    If you want to delete the forward word, use control+f control+w, i.e. go forward one word and delete back one word

  • Undo: People might not need this normally, but it's handy sometimes. Use default control-x control-u



Here's the contents of my inputrc file:



# This file controls the behaviour of line input editing for
# programs that use the Gnu Readline library. Existing programs
# include FTP, Bash, and Gdb.
#
# You can re-read the inputrc file with C-x C-r.
# Lines beginning with '#' are comments.
#

set editing-mode emacs

$if mode=emacs

"\C-f": forward-word
"\C-b": backward-word
"\C-a": beginning-of-line
"\C-e": end-of-line
"\C-u": unix-line-discard
"\C-w": unix-word-rubout
"\C-x\C-u": undo

$endif




As in the comments, if you are editing you inputrc file its helpful to know you can reload your mappings with control-x control-r

I'm pretty sure if you have a handy .inputrc file, you won't be frustrated next time when you have one of those long command line and need to edit them, you'll be no longer using the arrow keys :)

Enjoy your bash command editing sessions...

Friday, April 17, 2009

Browser cached pages + override + refresh

This is one of those "just a quick note..."

There are pages in the web which are enabled to be "cached" on the browser and most browsers do cache these pages. But there are times when you want to refresh and get the "latest" copy on the server, at which times what I do normally is open up tools menu for the browser and do something like "clear browser cache". (One place where I normally do this is on the development svn server where the pages are cached, and after creating a branch/adding a directory etc, it doesn't show up.)

Here's a quick workaround to refresh the page and get the latest copy from the server overriding the local cached copy in the browser.
Just add some random parameters at the end of the page.
e.g. If I'm looking at http://www.abhisanoujam.blogspot.com/, what you type is http://www.abhisanoujam.blogspot.com/?a=b in the browser address bar.
This will make sure that your browser actually hits the server bringing back the latest. Don't use the same random parameters as it may as well end up being cached :)

For the non-geeks: Don't forget the ? mark before the random parameters...

Tuesday, March 24, 2009

Yahoo and Google Suggestions



I always loved the suggestions from the google searchbox in firefox. Most of the time the term that I wanted to search almost always came up in the suggestions. I don't remember when was the last time i really went to www.google.com to do some search, have been always using the search box in the toolbar.
Had been thinking of having some fun with it since quite some time now...

I don't think there's any official API from google for Google Suggest, but looking at iGoogle (it also gives you suggestions real time), there do seemed to be some kind of API. The urls used by Google suggest (as of now) do support callbacks, which gives some opportunity to have fun with it :)
Apparently yahoo has got some API for Related Suggestions (yahoo suggest?). I was playing around with the API's from both google and yahoo... it was fun. Seems to me like google is giving much better results at least compared to yahoo's Related Suggestions. Nonetheless, its just fun looking at the results :)

Have fun here... do some typing in the input box below :)





Oh, by the way, you can click on the results too (will open in a new tab/window)

And here's the code...
The html used for the above is as simple as this:

<div id="suggestions">
<script type="text/javascript">
init("suggestions");
</script>
</div>


And here's the actual javascript code thats doing the bulk of the work:

var gurl = "http://www.google.com/complete/search?output=json&callback=gcallback&q=";
var yurl = "http://search.yahooapis.com/WebSearchService/V1/relatedSuggestion?appid=abhisanoujam&output=json&callback=ycallback&query=";
var gsurl = "http://www.google.com/search?q=";
var ysurl = "http://search.yahoo.com/search?p=";

var gel, yel, lgel, lyel, inp, gres, yres, gh, yh;

function init(ename) {
var el = document.getElementById(ename);
if (el) {
inp = document.createElement("input");
inp.id = "inputText";
inp.type = "text";
inp.setAttribute("autocomplete", "off");
inp.onkeyup = keyup;
inp.size = 55;
inp.value = "Try your text here";
el.appendChild(inp);
}
var cont = document.createElement("div");
cont
.setAttribute("style",
"border-style:solid;border-width: 1px;float: left;padding:2px;margin:2px;");

var gcont = document.createElement("div");
gcont
.setAttribute(
"style",
"float: left;padding:2px;margin:2px;width: 300px;height: 300px;border-style:solid;border-width: 1px;");
gh = document.createElement("div");
gh.setAttribute("style", "font-size: 16px;color:#33CC33;");
gh.innerHTML = "Google Suggests";
gcont.appendChild(gh);
gres = document.createElement("div");
gcont.appendChild(gres);
cont.appendChild(gcont);

var ycont = document.createElement("div");
ycont
.setAttribute(
"style",
"float: left;padding:2px;margin:2px;width: 300px;height: 300px;border-style:solid;border-width: 1px;");
yh = document.createElement("div");
yh.setAttribute("style", "font-size: 16px;color:#33CC33;");
yh.innerHTML = "Yahoo Suggests";
ycont.appendChild(yh);
yres = document.createElement("div");
ycont.appendChild(yres);
cont.appendChild(ycont);

el.appendChild(cont);

gel = document.createElement("div");
el.appendChild(gel);
yel = document.createElement("div");
el.appendChild(yel);
keyup(null);
}
function keyup(evt) {
if (emptyCheck()) {
return;
}
var gsel = document.createElement("script");
gsel.setAttribute("type", "text/javascript");
gsel.setAttribute("src", gurl + inp.value);
gel.appendChild(gsel);
lgel = {
"node" :gsel,
"text" :inp.value
};

var ysel = document.createElement("script");
ysel.setAttribute("type", "text/javascript");
ysel.setAttribute("src", yurl + inp.value);
yel.appendChild(ysel);
lyel = {
"node" :ysel,
"text" :inp.value
};
}

function emptyCheck() {
if (trim(inp.value) == "") {
var e = new Array();
displayRes(gres, e, null, gh, "Google");
displayRes(yres, e, null, yh, "Yahoo");
return true;
}
return false;
}

function gcallback(resp) {
if (emptyCheck()) {
return;
}
var res = new Array();
var t = resp[1];
if (t) {
for ( var i = 0; i < t.length; i++) {
res.push( {
"text" :t[i][0],
"results" :t[i][1]
});
}
}
displayRes(gres, res, gsurl, gh, "Google");
removeAllChildrenExcept(gel, lgel.node);
}

function ycallback(resp) {
if (emptyCheck()) {
return;
}
var res = new Array();
var t = resp.ResultSet;
if (t) {
var tt = t.Result;
if (tt) {
for ( var i = 0; i < tt.length; i++) {
res.push( {
"text" :tt[i]
});
}
}
}
displayRes(yres, res, ysurl, yh, "Yahoo");
removeAllChildrenExcept(yel, lyel.node);
}

function displayRes(el, r, surl, hh, t) {
var h = '';
var l = r.length;
for ( var i = 0; i < l; i++) {
var fsurl = surl + escape(r[i].text);
h += '<span style="font-size: 12px; font-weight: bold;"><a target="_blank" '
+ 'style="text-decoration: none;" href="'
+ fsurl
+ '"> '
+ r[i].text + '</a>';
if (r[i].results) {
h += '</span> <span style="font-size: 10px;">' + r[i].results + '</span><br />';
} else
h += '<span><br />';
}
if (l <= 0) {
h = '---';
}
el.innerHTML = h;
hh.innerHTML = t + ' Suggests (' + l + ' results) ' + getSmiley(l);
}
function getSmiley(l) {
if (l <= 0) {
return '<img alt="sad" src="http://l.yimg.com/us.yimg.com/i/mesg/emoticons7/20.gif">';
} else if (l >= 1 && l <= 3) {
return '<img alt="smile" src="http://l.yimg.com/us.yimg.com/i/mesg/emoticons7/15.gif">';
} else if (l >= 4 && l <= 7) {
return '<img alt="happy" src="http://l.yimg.com/us.yimg.com/i/mesg/emoticons7/1.gif">';
} else
return '<img alt="elated" src="http://l.yimg.com/us.yimg.com/i/mesg/emoticons7/69.gif">';
}

function removeAllChildrenExcept(node, keepchild) {
for ( var i = node.childNodes.length - 1; i >= 0; i--) {
if (node.childNodes[i] != keepchild) {
node.removeChild(node.childNodes[i]);
}
}
}
function trim(str) {
return str.replace(/ [ ]*/g, '');
}



Have fun :)

Monday, March 23, 2009

Posting javascript in blogger

I once tried to post some javascript inside the post itself and found out that it was not working.
Well seems like if you want to embed <script> tags inside the post, and you have "Convert line breaks" set to true (by default), you cannot add script tags, as Blogger would add <br /> instead of your line breaks.
So you need to write down your script in one single line.

Which is what this post actually does. Use it to your heart's content :)
Post your javascript here
>




Remove New Lines Remove Multiple spaces Replace <, > to &lt; and &gt; resp.






Enjoy :)

BTW, the javascript for the above is embedded in this post itself. And here's the code, use it on the above textarea ;-):


<script type="text/javascript">

function toggleCheckbox(docid) {
var rs = document.getElementById(docid);
rs.checked = !rs.checked;
}
function removeNewLines() {
var rnl = document.getElementById("removeNewLines");
var rs = document.getElementById("removeSpace");
var rlt = document.getElementById("replaceLt");
var c = document.getElementById('content').value;
var r = rl(c, rnl.checked, rs.checked, rlt.checked);
document.getElementById('result').value = r;
}

function rl(str, rnl, rs, rlt) {
if (rnl) {
str = str.replace(/[\n\r\t]/g,' ');
}
if (rs) {
str = str.replace(/ [ ]*/g,' ');
}
if (rlt) {
str = str.replace(/</g,'&lt;').replace(/>/g, '&gt;');
}
return str;
}
</script>

Friday, January 30, 2009

Odd Even printing threads... in multiple jvms -- Terracotta wow factor!

Recently I got a comment asking about a small use-case demostrating Terracotta

I hacked up a small app in which there are 2 threads (can be more), and each thread is printing numbers sequentially, each thread printing one after another.
With 2 threads, one of them will print odd numbers, and the other will print even numbers... when we have multiple number of threads, we want each thread to print one after another, the threads executing serially.

So lets take a look at the classes.
First, lets have a Counter class that will maintain the current value ...

/**
* Jan 28, 2009
* @author abhi.sanoujam
*/
package sample;

public class Counter {
private final int numParties;
private int value = 0;
private final int maxValue;

public Counter(int numParties, int maxValue) {
this.numParties = numParties;
this.maxValue = maxValue;
}

public synchronized boolean isMyTurn(int partyNum) {
return value % numParties == partyNum;
}

public synchronized void setValue(int val) {
this.value = val;
}

public int getMaxValue() {
return maxValue;
}

public synchronized boolean isMaxValueReached() {
return value >= maxValue;
}

public synchronized int increment() {
this.value++;
return value;
}

public synchronized int getValue() {
return value;
}
}


Here's the Runnable class that each thread will be using:


/**
* Jan 28, 2009
* @author abhi.sanoujam
*/
package sample;

import java.util.concurrent.CyclicBarrier;

public class OddEvenRunnable implements Runnable {

private final Counter counter;
private final int partyId;
private final CyclicBarrier barrier;

public OddEvenRunnable(int partyId, Counter counter, CyclicBarrier barrier) {
this.partyId = partyId;
this.counter = counter;
this.barrier = barrier;
}

public void run() {
try {
System.out.println(Thread.currentThread().getName() + ": Waiting for GREEN signal from main guy...");
barrier.await();
} catch (Exception e) {
e.printStackTrace();
}
while (true) {
synchronized (counter) {
while (!(counter.isMyTurn(partyId) || counter.isMaxValueReached())) {
try {
counter.wait();
} catch (InterruptedException e) {
System.out.println(partyId + ": Got Interrupted. Continuing for my turn...");
}
}
if (counter.isMaxValueReached()) {
// make sure other-threads don't keep waiting for my signal when I'm
// leaving
counter.notifyAll();
break;
}
int value = counter.increment();
System.out.println(Thread.currentThread().getName() + ": Counter Value=" + value);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
// ignored
}
counter.notifyAll();
}
}

System.out.println(Thread.currentThread().getName() + ": DONE!!");
}

}




Here's the Main class that drives the app.


/**
* Jan 28, 2009
* @author abhi.sanoujam
*/
package sample;

import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.BrokenBarrierException;
import java.util.concurrent.CyclicBarrier;

public class Main {

private final int numParties;

// @Root
private final Counter counter;
// @Root
private final Set runners = new HashSet();
// @Root
private final CyclicBarrier barrier;
private final Thread[] parties;

public Main(int numParties, int maxCounterValue) {
this.numParties = numParties;
counter = new Counter(numParties, maxCounterValue);
barrier = new CyclicBarrier(numParties + 1);
parties = new Thread[numParties];

for (int i = 0; i < numParties; i++) {
parties[i] = new Thread(new OddEvenRunnable(i, counter, barrier), getThreadName(i));
}
}

private void runInSingleJvm() throws BrokenBarrierException, InterruptedException {
// start all the counting parties
for (int i = 0; i < numParties; i++) {
parties[i].start();
}
startCounting();
}

private void startCounting() throws InterruptedException, BrokenBarrierException {
System.out.println(Thread.currentThread().getName() + ": Sleeping for 1 secs....");
Thread.sleep(1000);
System.out.println(Thread.currentThread().getName() + ": ... And letting all the counting threads go!!");
// let thy parties proceed
barrier.await();
}

private String getThreadName(int partyNum) {
String prefix = "";
for (int i = 0; i < partyNum; i++) {
prefix += " ";
}
return prefix + "Party-" + partyNum;
}

public static void main(String[] args) throws Exception {
Main main = new Main(2, 30);
if (args.length == 0) {
// run in a single node/jvm
main.runInSingleJvm();
} else {
if (args.length != 1) {
printUsageAndExit();
}
if ("odd".equals(args[0])) {
main.startFirstThread();
} else if ("even".equals(args[0])) {
main.startSecondThread();
} else if ("main".equals(args[0])) {
main.startMainThread();
} else if ("reset".equals(args[0])) {
main.reset();
} else
printUsageAndExit();
}

}

private void reset() {
for (int i = 0; i < numParties; i++) {
parties[i] = new Thread(new OddEvenRunnable(i, counter, barrier), getThreadName(i));
}
synchronized (runners) {
this.runners.clear();
counter.setValue(0);
}
System.out.println("Reset Done.");
}

private void startMainThread() throws Exception {
if (runners.size() != 2) {
System.out.println("Make sure that you have started both the odd and even threads.");
printUsageAndExit();
}
synchronized (runners) {
runners.add("main");
}
startCounting();
}

private void startSecondThread() {
if (runners.contains("even")) {
System.out.println("You have already started the even-printing thread.");
printUsageAndExit();
}
synchronized (runners) {
runners.add("even");
}
parties[1].start();
System.out.println("Started even thread");
}

private void startFirstThread() {
if (runners.contains("odd")) {
System.out.println("You have already started the odd-printing thread.");
printUsageAndExit();
}
synchronized (runners) {
runners.add("odd");
}
parties[0].start();
System.out.println("Started odd thread");
}

private static void printUsageAndExit() {
System.out.println("USAGE: java Main [odd | even | main | reset]");
System.out.println(" No-arguments - Starts 2 threads printing odd and even values in single jvm.");
System.out.println(" odd - starts the odd-number printing thread in this node.");
System.out.println(" even - starts the even-number printing thread in this node.");
System.out.println(" main - starts a thread which lets the odd and even threads go ahead.");
System.out.println(" reset - Resets all states so you can start all over again.");
System.exit(1);
}

}




When running in a single-node (without Terracotta), the only method of interest is runInSingleJvm().

You can check-out the code, a maven project, from http://abhi-sanoujam-blogspot-posts.googlecode.com/svn/trunk/terracotta-in-action/
using "svn checkout http://abhi-sanoujam-blogspot-posts.googlecode.com/svn/trunk/terracotta-in-action/ "

You can run in a single node like:


$ mvn exec:java -Dexec.mainClass=sample.Main


Or you can even use regular java for doing it:


$ mvn compile
$ java -cp target/classes/ sample.Main



You should be seeing some output like:


sample.Main.main(): Sleeping for 1 secs....
Party-0: Waiting for GREEN signal from main guy...
Party-1: Waiting for GREEN signal from main guy...
sample.Main.main(): ... And letting all the counting threads go!!
Party-0: Counter Value=1
Party-1: Counter Value=2
Party-0: Counter Value=3
Party-1: Counter Value=4
Party-0: Counter Value=5
Party-1: Counter Value=6
Party-0: Counter Value=7
Party-1: Counter Value=8
Party-0: Counter Value=9
Party-1: Counter Value=10
Party-0: Counter Value=11
Party-1: Counter Value=12
Party-0: Counter Value=13
Party-1: Counter Value=14
Party-0: Counter Value=15
Party-1: Counter Value=16
Party-0: Counter Value=17
Party-1: Counter Value=18
Party-0: Counter Value=19
Party-1: Counter Value=20
Party-0: Counter Value=21
Party-1: Counter Value=22
Party-0: Counter Value=23
Party-1: Counter Value=24
Party-0: Counter Value=25
Party-1: Counter Value=26
Party-0: Counter Value=27
Party-1: Counter Value=28
Party-0: Counter Value=29
Party-1: Counter Value=30
Party-0: DONE!!
Party-1: DONE!!



We create two threads and the two threads wait on a barrier. The main thread when calls "startCounting()" method, makes the two waiting threads proceed.
You can see that the two threads are running sequentially, one by one. Each thread printing the next value on the counter, one thread prints odd numbers and the other thread prints even numbers.

Nice and cool. For testing, we can even change the number of parties when calling the Main constructor by passing in the required number of parties. E.g. if we pass in 5, then there will be 5 threads and each thread will print the counter sequentially, the threads executing one after another.

Now here comes the wow-after-cool part :)
With Terracotta, you can make the threads run on different jvm's and the threads will co-ordinate with each other -- while running in different JVM's. Imagine doing Object.wait() on one jvm and another thread Object.notify()'ing from another thread...

OK, lets have a look at the tc-config.xml for this..., oh by the way, if you are new to Terracotta, you don't need any code-level change to make it work cross-jvm, you just plug in a tc-config and start up with terracotta.

Lets have a look at the tc-config for this app:


<?xml version="1.0" encoding="UTF-8"?>
<tc:tc-config
xmlns:tc="http://www.terracotta.org/config"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.terracotta.org/config http://www.terracotta.org/schema/terracotta-4.xsd">
<servers>
<server
name="localhost"
host="localhost">
<dso-port>9510</dso-port>
<jmx-port>9520</jmx-port>
<data>target/terracotta/server/data</data>
<logs>target/terracotta/server/logs</logs>
<statistics>target/terracotta/server/statistics</statistics>
</server>
</servers>
<clients>
<logs>target/terracotta/clients/logs/%(tc.nodeName)%D</logs>
<statistics>target/terracotta/clients/statistics/%(tc.nodeName)%D</statistics>
</clients>

<application>
<dso>
<roots>
<root>
<field-name>sample.Main.counter</field-name>
</root>
<root>
<field-name>sample.Main.runners</field-name>
</root>
<root>
<field-name>sample.Main.barrier</field-name>
</root>
</roots>
<locks>
<autolock>
<lock-level>write</lock-level>
<method-expression>* sample.Main.reset()</method-expression>
</autolock>
<autolock>
<lock-level>write</lock-level>
<method-expression>* sample.Main.start*()</method-expression>
</autolock>
<autolock>
<lock-level>write</lock-level>
<method-expression>* sample.OddEvenRunnable.run()</method-expression>
</autolock>
<autolock>
<lock-level>write</lock-level>
<method-expression>* sample.Counter.increment()</method-expression>
</autolock>
<autolock>
<lock-level>write</lock-level>
<method-expression>* sample.Counter.setValue()</method-expression>
</autolock>
<autolock>
<lock-level>read</lock-level>
<method-expression>* sample.Counter.isMaxValueReached()</method-expression>
</autolock>
<autolock>
<lock-level>read</lock-level>
<method-expression>* sample.Counter.getValue()</method-expression>
</autolock>
<autolock>
<lock-level>read</lock-level>
<method-expression>* sample.Counter.isMyTurn()</method-expression>
</autolock>
</locks>
<instrumented-classes>
<include>
<class-expression>sample.Counter</class-expression>
</include>
<include>
<class-expression>sample.OddEvenRunnable</class-expression>
</include>
</instrumented-classes>
</dso>
</application>
</tc:tc-config>




You define roots and provide auto-locking for your methods in which you are synchronizing. You should probably go here if you want to read more.

Roots are basically objects that get shared across the cluster. You can note that the counter object is a root, thats how the threads in different nodes are going to get the updated value in each node.

For demo purpose, I've written the Main class to take arguments to start up in a parametrized way across the cluster, so that you can start the threads individually across different nodes.

Here's the script that I'm using to run with Terracotta (runWithTc.sh). (Again, you can find the whole project here)


#!/bin/bash

TC_INSTALL_DIR=/Users/asingh/terracottaKit/terracotta-2.7.1

mvn compile

CP_FILE=cp.txt
mvn dependency:build-classpath -Dmdep.outputFile=$CP_FILE
echo ":./target/classes" >> $CP_FILE

$TC_INSTALL_DIR/bin/dso-java.sh -cp `cat $CP_FILE` sample.Main $*

if [[ -f $CP_FILE ]]
then
rm $CP_FILE
fi


Change TC_INSTALL_DIR with the path where you have installed Terracotta.

First, you need to start the Terracotta-server. Go to $TC_INSTALL_DIR/bin and type


./start-tc-server.sh


This should start the Terracotta server in localhost (note that we have referred to 'localhost' in the server-section in tc-config.xml. You can always run this in multiple machines too if you so desire).

To start the Odd printing thread, run it like:


$ ./runWithTc.sh odd


You should see an output something like:

Started odd thread
Party-0: Waiting for GREEN signal from main guy...


Now on another terminal/console, start the Even-printing thread like:


$ ./runWithTc.sh even


Again, you should see some output like:

Started even thread
Party-1: Waiting for GREEN signal from main guy...


Now you have 2 different threads started in different jvm's and both the threads are waiting on the shared barrier.
Lets start the main thread which will make the threads go ahead. Open another terminal/console and type in the following:


$ ./runWithTc.sh main



You should be seeing an output like this:

main: Sleeping for 1 secs....
main: ... And letting all the counting threads go!!


Now, at this point you can see that the other two threads in the other jvm's have started to run. The output in the console's should be like this:

Started odd thread
Party-0: Waiting for GREEN signal from main guy...
Party-0: Counter Value=1
Party-0: Counter Value=3
Party-0: Counter Value=5
Party-0: Counter Value=7
Party-0: Counter Value=9
Party-0: Counter Value=11
Party-0: Counter Value=13
Party-0: Counter Value=15
Party-0: Counter Value=17
Party-0: Counter Value=19
Party-0: Counter Value=21
Party-0: Counter Value=23
Party-0: Counter Value=25
Party-0: Counter Value=27
Party-0: Counter Value=29
Party-0: DONE!!


And for the even-printing node,

Started even thread
Party-1: Waiting for GREEN signal from main guy...
Party-1: Counter Value=2
Party-1: Counter Value=4
Party-1: Counter Value=6
Party-1: Counter Value=8
Party-1: Counter Value=10
Party-1: Counter Value=12
Party-1: Counter Value=14
Party-1: Counter Value=16
Party-1: Counter Value=18
Party-1: Counter Value=20
Party-1: Counter Value=22
Party-1: Counter Value=24
Party-1: Counter Value=26
Party-1: Counter Value=28
Party-1: Counter Value=30
Party-1: DONE!!


You can see that the two-threads are co-ordinating across jvm boundaries.

Apart from the thread-coordination, this app contains the data-sharing across nodes also.
In the tc-config, you can see that we have another root called runners. This is a Set<String> which just remembers which threads have been started.
If you run runWithTc.sh odd multiple times in multiple consoles, it won't let you start multiple odd-printing threads. It remembers this fact by just putting "odd" in the runners which gets shared across the nodes.


Hope you have a great time playing around ... and enjoy clustering your app with Terracotta :)

P.S. Someday soon, I'm gonna really post something to change things like < to &lt ...