Software Engineering Archives

November 15, 2002

The Value of Automated Unit Tests

I had an intersting hallway discussion with a co-worker last night about unit testing and follwoing coding conventions. The readers digest version is that he is against both of them unless he happens to like them. (Why didn't the get him durring the layoffs? Oh yea, he's a team lead...). His defenses are that (a) he doesn't have time to write them and (b) be tests his code when he writes it. The problem with (a) is that he tops the foosball stats with games played. The problem with (b) is that it is a manual process, and he claims that he doesn't have the problem of unintended consequences because he "takes the time to understand his code." So when confronted with the argument of change of code ownership (which happens here alot, but not with him yet) he says "well then the developer should take the time to understand the code before he changes them." Holy Concieted CMM Level 0 Heroic Effort Batman! That is practically a profession of faith against repeatable engineering process, equvilent in value to the typical high school hackers, who he took the time to slander. I wish I could remember exactly what he said because it was so derogitory to people who wern't as good as he was (apparently he never took the sensitivity training that our ex-VP said he needed in his annual review). This discussion occured while he was packing up his laptop and heading home so it ended with him in a huff leaving.

The irony of this is that this morning I found a perfect example of how unit testing protects against the law of unintended consequences, which he says "good" developers never do. A unit test that I do involving schema validation was breaking. Not only was I fixing the coding conventions in those files but our IT/SCM/Build/Developer guy (layoffs do wierd things to job responsibilities...) was upgrading the tools we use used to build and such (being between release and such) including Xerces up to 2.2.1. Unit tests to the rescue! First I reverted all my edited files, still breakes. Then I use older version of Xerces, still breaks, even the version I had before the upgrades. After some head scratching I descide to revert the whole build tree to the last known good point and run the unit test. It works! (I would have been screaming if it wasn't because that shows there is a SCM hole). So I sync one commit list at a time, and it turns out the point it breaks is when... not my code changed, not when Xerces was upgraded, before both of those even happened. It occured when Xalan was upgraded. It turns out that 2.4.1 ships with Xerces 2.2.0, which has a schema realted bug in it. The way the tools tree was set up the jar loading was taking the xerces jar from xalan ($#@&ing manifest files) rather than what the classpath said. Because of the Automated Unit Tests I was able to narrow down the breakage to a single checkin, and then the "Ah-Ha!" moment came much easier. With the reliable test it took me an hour, but wihtout it I would have beens scratching my head all day over waiting for that "Ah-Ha" to occur. not knowing where the breakage really started. And it wasn't that a bad developer was being a code cowboy either, it was that law of unintended consequences striking again when a seemingly innocent change to the system wasn't so innocent.

IMHO accepting and embracing processes like unit testing is what seperates a Software Engineer from a simple developer or programmer. There is a difference.

December 2, 2002

QA and Testing

No. The reason for lousy software lies in two simple words. Inadequate testing.
If he's talking about QA, then he's dead wrong. According to studies in all other facets of production, a focus on increasing the quantity of QA will actually increase the number of defects. (That must somehow tie into Heisenburg's Uncertainty Principle, right?) Quality software requires testing, yes, but that should be the responsibility of the developer. Want to make high-quality software? Make the developers think that there is no QA. Even if you sneak some in unbeknownst to them. [ /dev/nul ]

Sorry, I have to disagree here. When you send more mine sweepers into a minefield more mines are found. Does the number of mines in the minefield increase? No, there is still just as many, just the number of known mines increases. What certinaly increases is what is known about the minefield, what descreases is the unknown quantity. That is what QA's role is: To make an unknown quanitity a known quanitity. QA doesn't fix the defects, the cataog and catagorize them, so that an intelligent decision can be made about release and which defects are worth sending back to be fixed.

But there is a point in your message that I mus agree with, a large portion of the testing should fall on the developers. The role of QA shold not be as crash test dummies (and you shouldn't hire crash test dummies either). QA should be (a) looking at the problem from perspectives that development hasn't thought of and (b) running the more involved and incricate tests that require a full time effort, like scalability load testing and "burn in" style testing. Given that the QA folks (generally) didn't write the code they are at a disticnt advantage for (a), and (b) generally requires a different skill set than a code cowboy would use.

To think that there should be no QA or that the developers should not know it exists is like wrapping a towel around your head to get past the Ravenous Bug-Blatter Beast of Traal.

February 3, 2003

Code Quality

These code quality metrics proviced by PMD and Eclipse are quite nice, it points out all sort of strange things. But there are some thing it doesn't get...

For example I came across a class written by another developer (not sure who at this point). The first thing that caught my attention about that class is the incessent use of this. to refer to local fields. And he would never use a class name to refer to another static field, which is what tripped eclipse to check. After reviewing the class and parent class in question I came across a doozie... all of the methonds and fields were static, except that the fields were initilized by a non-static constructor! It's not like it's a forced option, since you can use static { ... } to initilize static fields. There are just some stupid patterns that you can never imagine until you see them.

PMD definatly needs a "StaticFieldReference" rule, perhaps they also need a "StaticInitInConstructor" rule, where it barks if you set the value of a static field in the same class inside the constructor of a class. Manipulation is ok.

February 12, 2003

Test Driven Patching

One of the wonderful things about open source software is that you can scratch an itch. Is Something broke or do you wish it did more? You can fix it! But with the latest adoption of several Extreme Progaming techniques in open source projects it occured to me that XP, and in particular Test Driven Development (TDD), as cannocally formed does not work well for external contributions.

Take my two latest patches, one to Marathon and another for the Parallel task on Ant (apparently I am sill a committer, [note to self get login accout pasword...]). Marathon is hardcore on TDD and at Ant it is a Very Good Thing (but not always a deal breaker). With both of these patches I updated the tests, but should the committers take my word for it that they are valid tests?

One of the basic principals of TDD is you write the test first, then run it, and watch it fail. This is before you add the new functionality. It seems like a sublte point but it is a very essential step: it validates the test can fail. It may seem unnesscicary, but it can be critical. All of the other constraints in the system can conspire to make a test uesless. Without validating a failire it is quite possible that you can accidentally code the equivilent of if (false) { fail("I'm broke"; }. Is that a valid test case? Maybe if you are testing the compiler and it is supposed to choke when it encounters dead code, but cases like that are never that clear to read.

So how does a project handle the case of patches for TDD? In the case I sent to Marathon it was simple, it was broken functionality and it added no new APIs. In this case you apply the patch that has the test first, assert the test fails, and then you apply the rest. But what about more complex cases? Like when the patch is adding new functionality? My proposed solution is submit two patch files, the first one adds the tests and needed API changes needed to compile, and none of the new code for those APIs. The second patch actually contains the code that makes the APIs work. With this the core commiter can take the two patches, apply the first, clean, compile, test, and look at the new failure and say "this is good." Then (and only then) does the committer apply the second patch, clean, compile, test, and see no test failures (or only failires that were there before the patches were applied).

Generating two diffs from the CVS is a pain, but if you are practicing TDD anyway then you could grab your first patch after the test starts failing. This makes developing a two step process, but TDD is already a two step process: (1) create failing test (2) make test pass. Test Driven Patching reflects that as well.

June 16, 2003

The Follies of IDE Selection by Checklist

Back on the issue of IDE bloat. Sometimes corperations pick IDEs by commitee. And that leads to the check list syndrome. Group A requires Feature A and Groups B, C, D ad infinatum. Add in another common feature of "corperate standards" where development "needs" to standardize on one IDE. So what usually winds up getting picked is the IDE that can meet the biggest check list. In the interest of not losing revenue the IDE developers try to fill all of thoese check list items as is inhumanely possible. But the problem is that the IDEs become a Jack of all Trades, and are sadly the master of none. More often than not what winds up getting picking is an IDE that no one is happy with. But it was done in a very egalitarian fashion, meaning that everyone's equally screwed in that case.

Situations like these remind me of the early days at my current job. We initally had some big fuzzy discussions about coding practices and build environments and one of the things that came up was IDEs. One person with alot of clout (that immediatly eliminates me) insisted we all use Visual Cafe because it integrated with WebLogic. At the job I had just before this one I tried to use what was a current copy of Cafe for a week. I had liked it before, but then again this was after not using it for two years while I was experementing with Linux in College (don't we all experement with stuff in college? Since I didn't experement with drugs I had to experement with something...). Anyway, I quit rather quickly because of it's horrible JDK 1.2 integration. That, and it crashed alot. I voluntered that information but it was freely ignored. Weblogic Integration was the most importaint thing (perforce integration was second) and thus the standard was Cafe (I didn't use it, I stuck with NetBeans).

This was before WebGain crashed and burned. In the year since I had abandonded it Cafe hadn't seen any major changes (despite the new major version number since WebGain bought it). A year and half later I walk into a phone confrenece where he's whining about Cafe crashing again and about what a piece of crap it was. I had to have a chuckle to myself because that's exactly what I had told him over a year earlier. It was still a piece of crap, but the piece of crap integrated with WLS.

Moral of the story? Trust experience over a marketing checklist.

June 17, 2003

GUI Design Tools are Not Just for the Weak

Now on issue #2 that I see with monsuier BileBlog's rant on IDEs. The most relevent poriton of that previous quote is as follows.

Serious swing developers I suspect are likely to scoff [at IDEA's GUI designer tool] and stick to using their own hand rolled magic. Still, I'm sure java needs yet more ignorant mediocre inexperienced developers dabbling in swing eh? [The BileBlog ]

I suspect that he's never done serious Swing development, I also suspect that a serious OSS developer would use an an OSS version control system as well (and by suspect I mean "I'm speaking out of my ass here"). There is no doubting that hand-rolled magic is sometimes the only difference between a sophomore CS project and a shrink-wrapped product. But if I were going to write a GUI in nothing but hand-rolled magic I would need at least two to three times the schedule to accomplish it.

Why? Well, as I blogged nearly eight months ago productivity matters. There are times that you want to spend a lot of time tweaking the little tidbits, and there are lots of times you could honestly care less. That's why I like start out in the GUI designer rather than blank java class. Yea, I could do all the GridBagLayouts by hand. I could also make the vertical FlowLayouts line up nicely inside multiple BoxLayouts by hand, and I could even write my own layout manager to deal with my personal nits if I wished (in fact that was one of the first things I did with Java GUIs more than seven years ago). But why should I visualize the insets, gridwidths, and weightxs in my mind when I can visualize them with my eyes? GUI designers also speed up the tweak/compile/run loop we all get in just to see if you got a titled border on the right component or to see if you fill propertu on a component needs to be GridBagConstraints.HORIZONTAL or GridBagConstraints.BOTH or whether that box layout looks silly with an alignmentX of .96.

GUI designers are not just about layout stuff either (something the reviews indicate the IDEA designer focuses on exclusively). It's about basic wire-work as well. Do I really want to write the boilerplate code to attach some method to a actionPerformed listener added to a JButton or would I rather jump right into the code preping the data for the SOAP call? (the latter, because it's gonna have to be written anyway). When code is truly boilerplate (like button presses and menu selections) and can be automated it should be, wether it's as a language feature, an API convienece, or as a feature of the development tool.

Believe me, there are many times where hand-rolled magic is the way to go. But that time is not when you are saying "JTree here, JButton there."

June 28, 2003

Use Resource Files for GUI Text

This may seem like an obvious statement for people who do a lot of GUI work, but I will say this because it simply cannot be said enough. Don't hardcode strings! That falls into the good old "no magic constants rule" (unless that constant is the 32-bit signed int -1258207934). How do you go about it?

  • Take all your strings and save them in some file that is not part of your main code. There are several options: you could put them all in a code file full of constant values, you could put them in a properties file, or (my favorite) you could put them in an internationalzied resource bundle.
  • Use meaningful names. SC_EN_000854 may be just the same to a compiler as LOGIN_DIALOG_LABEL_USER_NAME, in fact it's shorter and probobly compiles just a millisecond faster. But take my advice: use the human readable form
  • Use some consistent strategy to create these names. Ther are at least three things it should include: what part of the application it's in, what particualr GUI element it applies to, and some indication of what it says. The order doesn't matter and you can even use short varients or (for location and type) encoded varients. But be consistent! The previous exampel could just as easily be LB_D_LOGIN_UID if you are coding it as type/location/meaning.

So why are these importiant? You mean I can't say "that's just the way you shold do it?" Ok, ok. There is a point to this blog entry rather than random rules. Why do I consider this importaint? Any guesses? Umm... you there, yea the one waving his hand up and down and shouting out "internationalization," you can sit down and calm down because you're wrong. Yes it is a very nice side effects but I think there are better reasons. You there, on the front row. "Maintainable code?" Well, it is maintanable code, but that's not the only reason we do it. Anyone else? How about you. Yes, the cynical one sitting in the back corner. "Management!" Ok, close enough, I'll take it.

It's not just management; it's tech writers, QA, marketing folks, and even other coders with better grammer and speling skills then you. To be more precice it's everyone else who wants to put their fingers in the GUI that isn't actually writing the GUI code. If you mis-spel a word and the strings are all over the code good luck finding the string. You could just search on the mis-speling and fix all occurances of that one. But in those cases people only catch the problems they can see, so the biggest stinkers may get over looked because they don't come up that often (for error dialogs they may never come up). With a centralized set of string files you can run a spel checker over it, but don't try this with java code bacuse signal to noise goes down. You can even hand it off to someone with linguistic skills who can fix all the bad grammer as well. Oh yea, I almost forgot. It's easier to internationalize as well, but you'll probobly have the other problems sooner if you are in anything but the smallest development teams.

Even though my reasons for this are entierly prgamatic it won't end all the problems. While some people care about colons v.s. hyphens for separator cues others care about what shade of grey the background is. Should it be windows grey, swing grey, or bike shed grey?

April 22, 2004

Still more than 3 ways to loop it...

A discussion on looping in C# misses my favorite approach to looping, of course this only works if order of execution does not matter...

for (int i = foo.Length - 1; i >= 0; i--) {}

The big advantage here is that we are not going up to some unknown number that has to be stored on the side or accessed from a (potentially mutable) field. This saves one value on the register stack

It also works if you are trying to index the first occurance, although going forward and breaking out of the loop seems to work better in those cases.

21 Apr 2004 9:15 pm MDT: Edited to explain what I think is novel about it.

January 27, 2006

Waterfall 2006

Ahh... not since my freshman year have I seen such interest in the Waterfall methodology. Too bad it's a joke, because it's a conference I might actually be able to get my company to pay for.

What makes this so funny and sad at the same time is that I have worked with people who are so thourhly characterized by some of these bios, except there use nicer terms that are friendlier to their point of view. I mean, who writes a 600 page object model without implementing it once? And then when we do implement the smallest part of it using his choice of data model he complains about the performance?

I only wish that the "it all flows downhill" mentality existed only in software engineering.

January 30, 2006

Dilbert: Why Waterfall is such a bad model

Sunday's Dilbert exposed just why Waterfall rarely works in the real world. Given a blank slate most product managers have no idea what they want it to do.

The real world follow on is when Sally finally does write the software the PM will say "why doesn't it do [feature X our competitor has]?" And anyone who has been in this flow before will know just how the rest of the conversattion will go.

April 12, 2006

Joel's Myopic view on Software Developers as Prima Donnas

Don't mess with Joel Spolsky's Air Conditioning. That is probably the most correct thing he has said on his most recent posting about Developer Abstraction. The sad thing is that the impression I get from the article is that good developers are myoptic Prima Donnas. What bothers me the most is the subtext that it's OK.

Jumping in the Way Back Machine this reminds me of a Sales Engineer I once had the pleasure of working with nearly 10 years ago when I was in Silicon Valley (on the subject of Prima Donnas... but I digress). After the sales team hired him at a rather nice salary he needed to work with those of us in development to understand our application. He sure loved to talk and thought a lot of himself, but he uttered one gem that got us all going: "There are really only two roles in a Company: Sales and Sales Support." While true from a certain viewpoint, it got at me and the VP of R&D in the wrong way. It got at us not because of the sales centric view of the company (which for a sales person is OK) but because of how dismissively it viewed the rest of the organization. That guy didn't last long. Other more cordial people lasted longer, like the guy who couldn't install our application to save his job: he lasted nearly a year. Moron yes, Prima Donna no.

Later Joel places a big analogy on Developers as lead singers like Dolly Parton. But I would like to make a distinction between Artists and Professional. Dolly happens to be both a Professional and an Artist, while singers like Billy Joel are great Artists but have some really unprofessional tendencies. But they are all musicians. Dolly may be the star of the show, and there are people who are instrumental in getting he music out there and making sure that she doesn't have to worry her pretty little head over where the XLR cords go so her microphone works. But there are also other musicians who are also instrumental because they play the instruments that make the balance of the music. They often aren't as prominent as the lead singer, but believe me they know how to setup, tear down, pack, and tune their own gear (although once you get good enough you don’t have to). Without the other supporting musicians (and tour staff) the Prima Donna is just another wannabe at the American Idol tryouts.

So while Joel thinks that his developers shouldn't worry their pretty little heads over air conditioning issues and that they should just sit down and spin their code, I disagree. If all you know is the development abstraction IMHO you don't know enough, and can cause unneeded friction on the company and can wear things down. If you hide the other functions of the company behind a facade, you only feed the tendency to become a Prima Donna . For most companies, one Prima Donna is one to many.

January 9, 2007

Negative Testing

No, no, I'm not being Negative on testing, in fact for teh most part I am positive about testing. But what I want to talk about is a "Negative test" where the desired result is a failure. Charles Miller calls it an Anti-Test, but is sounds a little too much like the Anti-Christ. Negative Tests are a Good Thingtm and they way he describes them is almost exactly how they should be done.

In a previous job I had a rather complex bundle of code that I had the nasty habit of fixing one bug only to break something else, often that I had fixed before. This code was complex, and I often set it down for months at a time while I went and chased the latest sales demand of the week. But every so often an actual paying customer would be scheduled to bring a new shipment of code online and certain bugs QA would find that had to be fixed.

So I developed my own little test suite of code and seeded the test almost entirely with bug report tests, often negative tests. Things like "this shouldn't take more than 500ms." Creating a test case that came up red was often the first step, and for bugs where the bug repost was a positive statement, i.e. "Application Does Foo when you go Bar, Baz, Bif" I would write a test would do that, detect it. The last step was to invert the final return value, or if there were other code branches that would trap out the bug not being their, calling fail("adfadfg").

These tests can the be named something like BUG_12345_Negative to show (a) that is testing something that shouldn't be happening and also the name indicates where to look for verification. Standard fare for QA pros I am sure, but I spend most of my day oggleing over loops, xpaths, and diffs.

About Software Engineering

This page contains an archive of all entries posted to ... And They Shall Know Me By My Speling Errors in the Software Engineering category. They are listed from oldest to newest.

Personal is the previous category.

Technology is the next category.

Many more can be found on the main index page or by looking through the archives.

Powered by
Movable Type 3.33