Tuesday, June 29, 2004

Unit testing our server code - List of tools we're using...

So we're starting the push to get our unit / system test cases as automated as possible... We've already got a set of test plans - but as the system has grown in complexity, testing is now taking over 2 man days per release of manual testing...

We're driving the running of testcases from Ant... Ant 1.6.1 - we've already got our build / packaging process in place here, and it's working well ...

Obviously we're using Junit (3.8.1) ... for our unit test framework...

We've got the surprisingly excellent and free (commercial option available too) JCoverage 1.0.5 to provide code coverage...

And we're using Http Unit 1.5.4 (the stable version ... looks like a new version is coming soon, though) ... I can't say enough about the HttpUnit toolset ... I love it... I'll get a big gushing mini-review out when I have a moment ... This really does everything I'm looking for on the system-test front...

A couple of guidelines for unit test cases:

  1. They must be repeatable / reproducible - they should always pass or always fail (until you make them pass...) ... none of this failing sometime, but not others...
  2. They should leave the system in a known state, and not count on the order in which they're run, the success of previous test cases, etc...


Our server reads (several) config files on startup, and there are many many different states that the system could be starting in... We’ve created data-sets of config files that the system could be starting in… We’ve created a BaseTest that extends junit’s TestCase – it implements a consistent set-up / teardown so we meet criteria #2 of unit test cases… We had initially been using ant to setup the directories / copy datafiles around / etc – because ant has such strong file manipulation support… But we couldn’t always count on the test cases being run via ant – and required separate test cases files for each test case… Too many dependencies for our liking, so we flipped things, and had the BaseTest make calls directly to Ant’s tasks… And this has worked great…

Another tricky area that we ran into… We’ve got hooks to call directly into the framework for our unit tests, but I also wanted to use HttpUnit for system tests, to take advantage of HttpUnit’s javascript support (which I’ve been SO happy with…)… This communicates over http, and once again our testcase criteria demand that the system be started in a known state, and left in a known state… So we created a helper thread which we can use to start and stop our server… It took some tweaking, but it looks like things are working really well now…

Over time we’ll build up our unit and system test cases, and our library of startup configurations… We’ve put hooks into our email notification so that we can even test that notification is working correctly, without actually sending / receiving email…

Monday, June 14, 2004

My Search For a New Dedicated Linux Server...

We've outgrown our existing Aktiom server.. Pseudo server really... they take a great big box, and create a bunch of smaller virtual boxes... we get root, and it appears to be our own box except when we're checking free memory running "top" we see 6 gigs, and we still get out of memory errors - ah well...

So I spent the weekend looking at alternatives ... They include building our own box and putting it in a local data center, or renting a dedicated linux box from someplace down in the US (most of our customers are in the US, and it's important that they've got low latency to the box...) ... I narrowed things down to servepath and netsonic.net ... I decided to go with netsonic after looking for opinions about them on groups.google.com, and finding that the owner has in the past helped others with Linux issues as far back as 1997... I also used netcraft to get an idea of what kind of uptime their customers were getting (servepath & netsonic), and who else they're hosting...

I ruled out the local solution for 3 reasons:
1) Time it would take us to put a box together - ordering / getting the machine, getting a working Linux kernel installed, etc... As much as 8 hours of work, and a week to get a box
2) Hardware failures... Really if we build it ourselves, we should get a second to allow us to swap out the machine in the event of hardware failure... so there's the added expense of having some hardware sitting around... A big dedicated hardware place has lots of spares handy, and will repair on our behalf...
3) Price... Local I'm looking at $300 Cdn (~ 220 US) + the cost of a machine (amortize over a year or two, but we're still looking at what would be around $100 / mon)... Vs. $99 US / month for bigger bandwidth...

I've spent the evening getting the new box setup, and it looks like we're good to go... When I have a spare moment tomorrow, I'll vent about how out of date / crappy the debian/java docs are... :)

Wednesday, June 09, 2004

Coding Guideline #1: Save your coworkers sanity - comment your code!

I would have to say that I couldn't disagree more with this blog entry I read today - I spent most of my day shaking my head ... Perhaps this is a joke - but others are chiming in in agreement, so I'm left shaking my head...

I'll start by saying why comments are important ... Code changes (as is mentioned in the blog above)... And because it changes frequently (diverging from all documentation written about it)... Almost always, code is the only current view of the system - and this is the reason that commenting code is so important...

A bit more background about myself... I work primarily as a consultant... On lucky occaissions I come into a project at the start, but most of the time, I'm coming into pre-existing projects that are on the brink of disaster ... The systems I'm working on are usually 100,000+ lines of code... They've been around for years, they've been touched by upwards of 20 developers, and numerous business analysts / projects have tweaked requirements over this time... If I'm lucky I can get my hands on a UML tool, to give me a high level view of the relationships between major objects in the system... A good tool, will suck out the javadoc class / member comments... Without a UML tool, I can still javadoc the system (which I'll do with a tool or not)...

Here's a nice summary about How to Write Doc Comments for the Javadoc Tool ... Every Java developer should read this over, and if you haven't read it lately, go and skim it again...

On top of the guidelines for javadoc, it just makes good sense to place comments where-ever you feel someone would ask: What? Why?

When someone is making an esoteric change to a piece of code, it takes them 2 seconds to add:

// Stripping trailing phone extensions for contact numbers for other group X, system XYZ...

rather than:

/**
* I'm too lazy to write 10 words, figure this out yourself... This is in the middle of a 1000 lines of
* code, in one of 100 java files...
**/
String newNum = "";
keepGoing = true;
for(int c=0;c char ch = oldNumber.charAt(c);
if (ch >= '0' && ch <='9' && keepGoing) {
newNum += ch;
} else if (ch == '(' || ch == ' ' || ch == 'x' || ch == 'X')) {
keepGoing = false;
}
}


... And I'm being generous by using semi-meaningful variable names...

A junior programmer is not going to learn anything from stepping through that code by hand... it's not rocket science, but it's not great code...

When I start work on a new project, I get all of the source, as much documentation on the system as I can get my hands on ... If I'm lucky, there's a system design doc that's only a few years old, requirements a few years older than that... And that's typically all of the external docs... The requirements doc was written by an analyst that didn't initially understand the business process fully, so they missed documenting about 1000 business rules ... Those rules were added after users opened bugs (if we're lucky enough to be using a bug tracking system), otherwise someone got an email from someone else who sent it from someone else, mentioning that group X can't handle extensions in phone numbers and we have to strip them off... So a developer, before one of the releases added some code to do that... No requirements docs were revisited, the testing group wasn't notified, the behaviour is deep within the system, and the change went un-noticed for 2 years, until someone in group X makes starts accepting extensions on phone numbers, and wonders why our system isn't sending them to him... Now some poor junior programmer has to comb though a 100K lines of totally uncommented code...

To sum up:
Java provides a framework for documentation... Use it... Keep your in code comments up to date... I've never in my career heard... "Damn that Chris - he wrote too many comments in his code - he makes my job too easy... "

Friday, June 04, 2004

Big corp open sourcing great opportunity for small companies...

Today I read a blog entry discussing why they thought big company open sourcing policies would be bad for small companies in the open source community...

I'd like to comment on the open sourcing of an IDE (like Eclipse)... And how the open-sourcing of Eclipse is great for small companies... 2 examples:

  • A small company building a tool for developers that wanted to provide an IDE had to build it from scratch, which would be multi-man-year task to produce a proprietary IDE that would probably not be very useful beyond whatever task they had in mind... Now they can leverage all of the effort that went into Eclipse and just build a plug-in...
  • There is still a huge opportunity in developing plug-ins, etc for Eclipse... Some small companies will choose to open source their plugin such as the tremendously great Spindle plugin for Tapestry, developed by Geoff... Others choose to sell their plugins - and all see opportunities here, without having to build an IDE from scratch...


Going back to a world without Eclipse is like going back to a world without standard windowing environments where everyone was building their own windowing/gui interfaces...