Stop me if you’ve heard this one before!

The web application has excellent coverage in unit tests and integration tests that run continuously, but some time ago (weeks actually) some number of tests began failing with strange state errors. In our case, out of 138 test classes and 1176 tests, [...]]]> (Finally a tech blog post …)

Stop me if you’ve heard this one before!

The web application has excellent coverage in unit tests and integration tests that run continuously, but some time ago (weeks actually) some number of tests began failing with strange state errors. In our case, out of 138 test classes and 1176 tests, 82 would error out.

The errors were all strange platform related things, like:

org.springframework.transaction.IllegalTransactionStateException: Pre-bound JDBC Connection found! JpaTransactionManager does not support running within DataSourceTransactionManager ...

Or:

java.lang.NoSuchMethodError: org.hibernate.cache.CacheException.

Naturally the failing tests all work when they’re run individually. Heard that one before?

I started out trying different combinations of tests. I could make a list of all the running test classes by grepping for “^Running ” in the test output log. I started out using the maven option “-Dtest=TestClassOne,TestClassTwo,…” to try tests in different combinations. Most of the time, the erroring tests would work perfectly. When they didn’t, the errors would occur in different tests or be different errors.

The failure now was non-deterministic! One of the difficulties is that Maven/Surefire would run the tests in whatever order it wanted to. That approach wasn’t going to work at all.

From studying the Spring references a little, I understood that Spring would cache the application contexts created for unit tests in order to improve the run time of tests. Wiring up a large application is slow when it’s done once — multiply that by 138 test classes and a slow test suite becomes glacial.

Clearly some test class being run prior to the error tests was corrupting the cached Spring context, and ruining the downstream environment. Spring provides a @DirtiesContext annotation specifically to label tests that require Spring to reload the application context. The problem is finding the test doing the dirty work!

I needed to make the test runs deterministic — run the test classes in the same order, and start eliminating classes one at a time from the top of the order. Surefire doesn’t have a property to exclude a test on the command line, so it required editing the POM file to exclude each test class in order from the top.

It was a tedious task, as many hidden software problems can be. I had to keep careful track of the list of test classes, and change the <exclude>TestClassExample</exclude> element for each test run. Fortunately each test run only required about two and a half minutes. After each test with one class excluded, I would examine the final result line for any change.

I was pretty confident that the culprit had to be an early test in the sequence, so I should only have to go about halfway through the successful test classes. Finally thirty-four classes into the list, I had my culprit.

Ironically enough, the test class causing all the problems was named TestSpringConfigurations. It had two tests that would simply verify that all of our wiring would successfully produce an application context. Marking the tests with the @DirtiesContext annotation made all of the following error tests run successfully.

Actually the @DirtiesContext annotation wasn’t necessary: The tests themselves included one fatal line: context.close(). By not closing the contexts after the test load, the cached application context was just fine for all the following tests.

One might argue that this class is pointless when run as part of a large test suite, since earlier tests have already loaded the application context. When Surefire arrives at TestSpringConfigurations, it is only using the already-cached context rather than loading a new one. Good point. But having the test in the suite also gives us a quick way to verify changes made to the application context configuration without running the whole set.

And finally the punch line: When the Spring configuration errors were finally vanquished, four test failures were revealed that were legitimately testing application code. Those test failures were completely masked by the Spring context corruption error.

Oh yes, that TestSpringConfigurations class has been in the suite for many months. Why did we only recently find it causing this corruption? No one here is quite positive, but the only major platform change we can point to is a switch from Java 5 to a Java 6 runtime. Maybe that triggered the original problem, and maybe sometime I’ll be interested enough to test that proposition.

Rod addressed some emerging best practices based around annotation-based application configuration using Spring 2.5 and later.

The emerging best practice is to use the @Autowired annotation rather than @Resource. He demonstrated how to disambiguate bean references with @Qualifier. He also showed how to create your own custom annotations [...]]]> Notes from a session at TheServerSide Java Symposium March 2009

Rod addressed some emerging best practices based around annotation-based application configuration using Spring 2.5 and later.

The emerging best practice is to use the @Autowired annotation rather than @Resource. He demonstrated how to disambiguate bean references with @Qualifier. He also showed how to create your own custom annotations and use them to disambiguate similar types:

@Autowired
public void setOrderServices(@Emea OrderService emea, @Apac OrderService apac) { ... }

@Emea
public class EmeaOrderService implements OrderService { ... }

@Apac
public class ApacOrderService implements OrderService { ... }

(What was unclear to me at the time is whether or not these custom annotations required some definition elsewhere, but I’m not entirely conversant on how annotations work.)

He also spent some time on the @Component meta-annotation, and the concept of stereotypes.  Spring comes with several pre-configured stereotypes:

• @Service
• @Repository
• @Aspect
• @Controller

It’s a powerful concept.

So here’s the big question for me. Why are we configuring with annotations now? It doesn’t do away with the need for context XML files. He indeed mentioned the pros and cons for using Spring 2.5 annotations:

Component scanning pros:

• No need for XML (potentially)
• Changes picked up automatically
• Works great with annotation driven injection
• Highly configurable (whatever that means)

Component scanning cons:

• Not a 100% solution (can’t do everything with annotations)
• Requires classes to be maintained
• Need to take care not to scan an excessive number of classes (use filtering)
• Don’t get valuable application blueprints documented in application XML

He makes a decent case for mixing and matching XML configuration with annotation configuration.

The emerging best practices (he says) include:

• Use XML for classes that you can’t or won’t annotate, such as third party components (data sources, message queues)
• Use annotations and classpath scanning for application objects
• Jump into XML for complex injection behavior and per-instance configuration (like injecting lists and maps)

He suggests the Spring Web Flow example application for examples.

He recommends moving away from the old Spring MVC inheritance model in favor of the @Controller stereotype. (Need to go back to his slides for other recommendations.)

The rest of the presentation covered features coming up in Spring 3.0. One nice feature is the ability to embed property value injection with a @Value annotation and a new expression language. Example:

@Value("#{systemProperties.favoriteColr}")
private String favoriteColor;

@Autowired
public void init(@Value("{systemProperties.databaseName") String dbName)

I was happy to see this presentation, because I’ve really been resistant to using annotation based configuration and component scanning. I do like the XML blueprint for an application, and I like having one place to look for how my application is configured. (I even try to avoid using multiple application context XML files, which avoids having to hunt through several files to find my bean configuration.)

Things that happen by magic like autowiring make me a little uncomfortable. He pointed out that the Spring IDE is a good practical solution for actually browsing through your app configuration.

At a keynote talk, you expect sweeping generalizations and “big picture” insights.

Rod pointed out some things that I have watched happening over the past two decades, namely the rise and fall of complex monolithic software systems as an expensive luxury that falls in favor of simplicity along with economic cycles.

“Enterprise [...]]]> Notes from TheServerSide Java Symposium March 2009

At a keynote talk, you expect sweeping generalizations and “big picture” insights.

Rod pointed out some things that I have watched happening over the past two decades, namely the rise and fall of complex monolithic software systems as an expensive luxury that falls in favor of simplicity along with economic cycles.

“Enterprise Java” has a reputation for complexity, based on the early (failed) J2EE platform, and giant enterprise software systems like those from IBM and BEA. “Complexity is the single biggest killer of projects,” he says. I concur.

“Economic downturns reduce software complexity. Truly bad ideas need a booming economy to survive.” I never thought of it like that, but it should have been obvious.

Service Oriented Architecture (SOA) is one of those complex frameworks that he claims is doomed at this point.

So the movement now is to “lean software,” simpler modular systems that don’t try to solve every problem but focus on doing one or two things well.

His examples, past giants of complexity: IBM, BEA, Sun. Today’s leaders: Spring, Hibernate, Ruby on Rails, Django, Grails, Eclipse ecosystem.

In terms of business management, this means that companies move away from the large portfolio software solutions that require huge quantities of support and complexity. The movement is toward point solutions that are smaller and more targeted. This is a pendulum that directly follows the economic cycle.

Nicely done.