Java web frameworks: how to choose?

On every web project I've worked on, the discussion what web framework to use rises at an early development stage. In my experience, team members often have strong opinions on this particular subject. "JSF sucks, JSF2 is nice though, let's use it because it is a standard"; "Spring MVC worked really well at my last project, I will not settle for anything else"; "My friend says Wicket is cool and now I really want to try it". Often these claims are driven by personal preference based on immediate experience, or worse, resume building tactics. This can lead to anything between a passionate discussion and a flame war. How can we create some consensus on a selection, that is in the customer's best interest?

Website families
The first question to answer is "What kind of website are we building?". On one end we have resource based web applications. We can browse them to read news, shop for products, connect with friends, listen to music, etc. Typically we use REST style URLs to identify resources. The content on the page can be related to the URL. Typically these websites are build with frameworks that stay close to the HTTP request-response cycle (e.g. Spring MVC or Struts 2).

On the other end we have (single page) rich internet applications to handle email, process documents, create mashups, etc. The application typically has complex interactions and a rich model. The URL is often used as a single entry point into the application and is not necessarily in sync with the state of the application on the screen. Although these kind of applications can be created with the same HTTP request-response oriented frameworks, a component based framework (e.g. Wicket, JSF, GWT, Vaadin) is usually preferred to reduce developer complexity and improve maintainability of the code.

Assessing features
When building a new website it is important to discuss general features to assess the consequences of choosing an approach before even mentioning possible frameworks. This gives the team focus and often rules out quite a few options.

Following below are ten items worth checking before starting to talk about specific technologies. Especially when you are considering a component based framework, make sure you can accept all trade-offs.


1. Caching resources
Pages or partial updates loaded with a GET request can be cached by the browser or a proxy. This will decrease load on the server and improve performance on the client. A lot of component based frameworks rely on POST operations for interaction, so be sure to check this.


2. Session size
Is it always required for the user to have a session? How many users do you expect? Component based frameworks maintain a lot of state information for each user, just to function correctly. Usually this data is stored in the session. The amount of data is largely based on the complexity of a page, but I have seen instances where over 1Mb of session data was stored per user, in an application largely optimized to minimize the memory footprint.

Some frameworks offer an option to store this state in the client to reduce memory requirements. This puts a heavy load on the infrastructure though. For every request this state needs to be send to the server using POST, where it is decoded (Base64) and decrypted (security). On response the state is encrypted and encoded again and send back as part of the HTML.

Make sure there are no bad surprises here, that your framework can easily support the expected number of users on the available infrastructure. When unsure, do some proof of concept before committing to a solution.


3. Page size

Some frameworks generate a lot of HTML or rely on huge amounts of Javascript, often in an obtrusive way. This can lead to large, complex pages, frustrating users with low speed connections. The complexity of the page can also have a huge impact on rendering time. Always check if what your framework is generating is acceptable.


4. Styling requirements
When building a web site, separating markup (HTML), style (CSS) and behavior (Javascript) is considered good practice. Following some guidelines developers have maximum flexibility styling a website. When generating HTML using a framework, sometimes styling is mixed in with the generated markup. This can lead to problems when the website needs to be 'pixel perfect' in relation to the original design.

5. Quality components

Many frameworks offer components. Date selectors, autosuggestion fields and ajax forms are some examples of out of the box functionality. Having high quality components can reduce development time a lot. Make sure though that the offered behavior is acceptable to the customer. If not, check the required effort to change it. Telling your customer you can't fix a bug because the framework doesn't easily allow it gets old really quickly.

6. Client support

Even today, developing websites for multiple clients can be a pain. There are some good options like jQuery to make your Javascript cross browser compatible. What does your framework offer and how? Does it solve styling incompatibilities as well? Does it support al browsers? Does it use feature detection or browser detection? I've seen a case where all generated sources where dependent on the user agent string from the client. If the framework didn't recognize a specific user agent it would cause an internal server error, causing a lot of headaches for the developers every time a new browser was released (which is quite often these days).


7. Progressive enhancement

The term progressive enhancement refers to a strategy to build websites in a way so that all users have access to the basic functionality of the site. A lot of component based frameworks rely heavily on Javascript though, basically ruling out all clients without Javascript support enabled. Make sure the requirements of the framework are in line with the customer's expectations.


8. Bots
There are a lot of bots on the web. How are your pages interpreted by search engine spiders for example? Do they index your content correctly? Perhaps you can make a sitemap? What about other crawlers? Do you want to encourage other sites to scrape our content? Just some thing to consider when looking at the generated pages.


9. Pretty urls

On some projects I've had the specific requirement to use pretty urls for pages available to the public. Most component frameworks generate very cryptic urls. A lot is possible with rewrite rules, but make sure to check if this is a requirement and how this can achieved.


10. Browser functions

Every browser has a back button, and people will use it. If I want to go back to a previous page, I mostly use the back button, ignoring any option an application provides. I should know better, but still I expect it to work, and very often it does. I find a lot of developers to be annoyed with the back button. They keep complaining that it is a stupid function and shouldn't be used. If it is important to your customer though, better make sure your framework can support it. Same goes for bookmarking, or copying links from you address bar to share.

Final thoughts
There are probably more things to consider, comments are very welcome. Usually it is quite easy to agree on the requirements mentioned in the previous paragraph and explain them to the customer. When the boundaries are set it will be a lot easier to agree on a framework. People are very often aware of limitations or they can be looked up quickly. When there are multiple options left, factors like standardization, developer availability, learning curve etc. can be considered as well.

Sometimes people ask me why I don't like component based web frameworks, because for some projects a lot of trade offs have to be made and my argument is very much against using them. The point is not that I don't like them. In past projects I've used several frameworks like JSF or Vaadin to great success. I also worked in some projects where using a component based framework would have been a better fit. If you are building a resource oriented web site or expect high volumes of traffic, I seriously doubt a component based framework to be a viable option though.

To wrap up, I think it is a good idea to document the decision making process. Briefly write down the weighted factors and trade offs. When new developers are added to the team or unforeseen problems arise it is good to look back why a certain approach was chosen. This can greatly help accepting a limitation or speed up a reassessment.

Retrospectives

One important aspect of agile methodologies is having feedback loops. Feedback loops allow a team to make improvements to their process, by keeping what works and changing what doesn't.

In my experience, feedback loops have a huge, positive impact on teams. Continuously improving processes, allows people to work more effectively. Making changes requires less negotiation. People are less reluctant to try out something new, because a change can always be undone in a next iteration.

Another positive result is team motivation. Everybody is encouraged to contribute. This leads to less complaining and more commitment. Especially in times of stress, looking back and clearing up the air can prevent a lot of frustration between team members.

So what is a good time to provide feedback? If immediate results can be obtained, it is best to make improvements directly. Some shortcomings are less apparent though. One way is to have a retrospective meeting with the core team. I found that just after the start of a new iteration is a very good time for this. The last iteration is still fresh in people's memory and there is plenty of time left to finnish the current iteration. Adding another meeting takes precious time though. I'd suggest a retrospective meeting can be tried at least once. If the team finds it ineffective, it can always be cancelled at the next retrospective.

Having a good retrospective meeting is hard. It can easily get out of hand and become ineffective if it has no clear structure, It can be tempting to complain about other teams, or talk about the weekends. It is most effective however to look for improvements that can be made by the team itself. A good retrospective meeting results in clear actions and an agreement on when they must be executed and by whom. It is also satisfying for every participant.

The Retrospectives Wiki has some formats to do retrospectives. Ilja's blog post on retrospective anti-patterns is a good read as well.

Effective documentation

In most software projects there is some requirement for documentation. In the few projects where there are no specific requirements it is a matter of good craftsmanship to determine just the right amount of documentation to deliver. At a minimum, it should be enough to enable another person to deploy and maintain the delivered software.

Three types of documentation I encounter the most are requirement specifications, technical designs and source code. Though I find it crucial that source code is well documented in order to maintain it properly, this post is about the external types of documentation (from the source code perspective). Browsing through source code is very time consuming. How can we write good external documentation to enable project members to quickly learn about scope, design considerations, accidental complexity and other important functional and architectural decisions?

Functional requirements

I exclusively work in agile environments. Shorter development cycles, instant feedback and iterative thinking require a different requirements process than the traditional waterfall approach. This also leads to different kinds of requirement specifications. Requirements are more and more captured in separate use case documents or stories, instead of being lengthy, hard to read documents.

In some cases however, requirements may need some form of background documentation. Consider for example a case where several functional options are explored and weighted in great detail. It can be very helpful to document these options and the decision making process for future reference.


Technical designs
For technical design documentation this is even more important. During the lifetime of a project lots of technical decisions are being made. For most technical decisions several options are explored. It is a huge waste of knowledge if these options are not documented. When new developers are added to a project I am often asked why a particular approach has been taken. Having a correct amount of documentation available does not only clearly show pros and cons for every option, It can also lead to new insights, because the reader is actively going through the reasoning process, instead of reading outcomes.

example of a classical technical design: lots of text and concepts spread over multiple pages....

Guidelines
So how to put all this information into a single document, that is still readable? In my experience, the accidental complexity of the document structure limits the allowed complexity of the documented subject. So I keep the structure as simple as possible.

maximize value
Every page must be self-explanatory. This forces the writer to be short and precise. It also provides an immediate overview of the scope and does not force the reader to browse through pages, getting out of flow. Single pages can be used for reference and taken to meetings to support conversation.

define scope

Every page covers no more than a single topic. Adding more, increases document complexity and might unnecessarily confuse the reader. If a topic is too complex to fit on a single page, I split it up in simpler concepts, or use different view levels. 

document the process

When there are multiple solutions to a problem, all considered options are described on a single page with a short analysis of each option and a judgement of important aspects. This documents earlier considerations and allows for a better general understanding of why a specific option was chosen.

be concice

A picture is worth a thousand words. It gives immediate overview and can hold large amounts of data. I try to capture every topic into a single image, using up most of a page's real estate. I add a notes section on a page to provide some context or explanation that does not go very well into the main image.

appeal to the reader

The reader has to absorb a lot of information. I always make sure that every page is visually appealing and requires little reading to keep the reader inspired and focussed, instead of easily distracted.

use proper tooling

I prefer to use graphical tools over word processors, because they allow for more creative freedom.

invest in writing

Most documents are written once, updated regularly and read often, so I spent quite some time optimizing for reading at the cost of writing speed.

write together... and enjoy!

Writing documentation should be fun. A solid foundation is the best incentive to construct a high quality document to which everyone in a team can contribute.

example of a visually appealing design that weights various options to solve a specific problem

Getting started

As a starting point I used some templates available at http://www.visualspec.org/. They can be downloaded or serve as  a source of inspiration to build your own.

My thoughts for blog

When I was about eight years old I got a second handed C64. Playing games was fun, but more and more I became curious about how these games actually worked. I borrowed some books about programming and started to create my own demos. I became heavily inspired with the challenges and satisfaction that came with building software. I knew I was going to be a software developer for the rest of my life.

Flash forward to 2010. I am 31 years old and I enjoy building business critical software over 10 years already. As a developer I led various projects with varying success. Why are some projects more succesful than others?

Over the years I discovered that building great software involves more than great developers. It requires great processes that help developers and organizations get real work done. Proccesses that challenge people to be inspired and creative. To communicate and share knowledge. To have fun and be proud of their collaborative effort. To get to new heights and super productivity.

This blog is no goal by itself. It is a means to organize my thoughts, share experiences and come to new insights. By writing I hope to learn, get more questions and explore new areas. The goal is "Kaizen Mind", to excel in what I do.

This is my first blog and I am not used to writing. Am I ready to share my thoughts? Am I comfortable showing my weaknesses? I think I am. Will I have enough topics of interests? Time will tell. I expect this post to be the hardest, so the first hurdle is taken.