Software Development from Ryan

The conference was amazing. The only negative point, and this is the most tiny negative relative to everything else, would be that there was a bit of a pro Java slant at times which alienated developers using many other technologies. I guess its a sure fire way to get a positive response from a largely Java based audience. This however, was not something that any speaker from the GWT team did making a point to convey Java as a pragmatic, rather than religious, choice.

On the largely positive side of things, some of the most interesting people I’ve ever talked to were there. Bruce Johnson is an extremely practical no-compromise web purist and his dedication to GWT makes me feel lazy and Joel Webber’s depth of thought about the most specific details was wonderful. The rest of the GWT team were just as incredible. They all evoked a great amount of trust in the development of the toolkit. Their priorities are for the users of web apps and nothing else, not even Google itself - except in the sense that what’s good for the web is good for Google (of course helping the developer helps the user in many cases). The people attending the conference weren’t slouches either. Some of the brightest and most progressive developers were there. I found myself enthusiastically listening day in and out.

Adding to my positive experience was seeing my GWT book in print for the first time, watching it sell out by the second day and climb to #5 on Safari’s top books. Woo hoo!

Originally they had a touch of Ajax with their “digg” button where a click would submit your digg asynchronously to their server and update the digg count without a page refresh. This feature had been around for a long time and proved to be elegant and easily usable. Of course the lack of a page refresh also saves quite a bit of server bandwidth, but this performance optimization was shadowed by the features usability. Now, Digg.com has just added a little bit more Ajax icing to their traditional web app cake.

Basically what they have done is replaced the threaded comments with comments that are dynamically loaded when the user expands the list. All of the technical details are discussed by the implementor Joe Stump:

Probably the coolest, technically speaking, portion of the new comments is the manner in which most of the page is created. No longer do we create static HTML in PHP and send you a huge HTML page. Instead we give you the basics and, via AJAX/JSON, we make requests to the API and dynamically create the DOM using the FlyDOM jQuery plugin. The FlyDOM JSON templates are a stroke of genius if you’re looking at loading JSON dynamically into the DOM. The advantage of this is that initial page loads are much snappier and you can load the threads you wish to read on demand.

In simpler terms, Joe Ajaxified the comment system using the jQuery library. If you look past a lot of the technical coolness that Joe is talking about you’ll find that the greatest improvement here is performance based. Initial page loads are dramatically reduced by incrementally loading the page. If you look past any Ajax hype this performance improvement is half of the Ajax promise. The other half is usability. And although usability shined with the initial bit of Ajax on Digg.com’s digg button, digg members have declared this new comment system unusable.

In all fairness Digg.com is working actively on fixing the usability issues. It is clear however that usability in this instance came second, which is a mistake.

I work a lot with GWT building full Ajax applications. This is a bit different than Digg. In my case Ajax is the cake. The things that the GWT team does makes me happy. Seeing this Ajax issue at Digg reinforces that the GWT team is moving the project forward in the best direction. To understand this we can simply compare two quotes, one coming from JQuery and the other coming from GWT:

You start with 10 lines of jQuery that would have been 20 lines of tedious DOM JavaScript. By the time you are done it’s down to two or three lines and it couldn’t get any shorter unless it read your mind

Dave Methvin, JQuery front page

This sounds great to me. As a software developer I know the best code is no code. Or less is more. Digg.com made a good technical decision to use JQuery. But this is only half of the story and the half that developers usually tackle the best. Where is the other half?

We want great results, where “great” is defined by how much it benefits end users. Sometimes there are conflicts between what is easy for us developers and what benefits end users the most. When the two are in conflict, end user experience almost always wins.

GWT Team

I don’t think GWT would be the best solution for the Digg comment system since Digg just needed icing for their traditional web application. However, this is really the proper additude to have when developing applications that you want people to use.

Coincidently I’ve just finished writing a GWT example application that acts as an Ajax database administrator for a social news website like Digg.com. It’s a full Ajax application and the fifth example application in my book on GWT Applications. It illustrates how to dynamically load and save all data using JSON or XML on REST or Actions , or using GWT RPC…. basically a lot of server interaction techniques. Its tested on PHP scripts, Ruby on Rails, and a Java Servlet. Althought most of the technically interesting things are not visible, here is a somewhat underwelming screenshot of the applicaiton:

Adobe Apollo, now called AIR, (Adobe Integrated Runtime), the runtime environment that allows you to use existing web technology for apps on the desktop is an obvious next step from building desktop like Ajax applications that seem to be awkwardly confined to a browser window. The technology requires a runtime installation of approximately 10MB, similar to the need to install the Java or .NET runtime to run apps in these technologies. However, AIR stands apart by allowing you to develop your applications using web standard technologies. It also adds desktop application like features including storage on your hard drive.

Also providing storage is Google’s recently announced Gears browser plug providing SQL database like storage from web applications. Both Gears and AIR technologies are responding to a web user’s need for offline storage for their web applications. In favor of gears, the download size is under 10% of the download size for AIR. It appears to be a better solution if you only need to add offline storage to your web application.

On top of offline storage, some of the sample applications for AIR offer a mix of Flash vector based and DHTML user interfaces. Its good to see Adobe taking advantage of Flash, which Macromedia made the most successful browser plugin of all time. And it has had relatively no competition. Until now. It had its crown taken as the king for responsive web applications with the rise of Ajax. Flash is now primarily used for ads, music, video, and games. However, Ajax has not yet won in the category of web application interfaces. A big battle is going on between Google Maps, an Ajax map web app, and Yahoo! Maps, a flash based map web app, (I have to side with Ajax in this case. I still feel like I’m in an emulated environment when I use a flash interface. The buttons aren’t real. This is also one of the downfalls of older Java client apps).

Of course Microsoft sees this battle going on and wants a piece of the action. They have always had great development tools that have really pushed client application development on windows and are now trying to push web application development in the browser with its Silverlight plugin. Silverlight is a very interesting technology that supports a subset of the .NET framework. This means that it is like AIR providing better integration with OS features but differs by running in the browser. The download size is also a subset of the size of the .NET framework at approximately 10MB.

With Microsoft getting in on the action Sun is reminded of an old grudge. It is the company behind Java and once had a compelling browser plugin like Silverlight that provided richer desktop app like features inside the browser. Java applets still crop up from time to time but have nowhere near the usage that Flash has. Recently Sun announced JavaFX as a lightweight framework to build rich applications that seems to be a response to Microsoft’s Silverlight.

As a developer of client side web applications this can be very confusing. Do you write an Ajax application with web technologies like JavaScript and leverage Gears for offline storage, or use ActionScript to build an app based on Flash, or use .NET languages for Silverlight, or Java for JavaFX. Thankfully Adobe AIR has decided to leverage existing web technologies. This is one reason the web has become a better application platform for many apps than the desktop. However, Silverlight and JavaFX use non web standard technology because Java and .NET are better for building and maintaining larger and more complex applications which web standard technologies were not built to handle.

This brings us to where GWT fits in. It uses Java to allow developers to build larger applications and compiles the code down to web standards. It can’t be used to build the multimedia applications that Flash is known for, and Silverlight and AIR are aiming for, but it is the most compelling solution for building Ajax applications.
From a corporate perspective, It’s refreshing to see these large companies providing innovation on the web. It is important for them to have a decent amount of web technology real estate. And with many multi million dollar acquisitions of Web companies this web technology competition fits the place of the Netscape/IE competition in a modern day dot com bubble.

Links:

• Adobe AIR
• Google Gears
• Microsoft Silverlight
• JavaFX
• Google Web Toolkit

Dietrich Kappe has a nice post on the Pathfinder Agile Ajax blog about Bruce Johnsons remarks on the complexity of building tools for dynamically typed languages.

I think its important to note that, along with the halting problem, the reason that dynamically typed languages like javascript don’t have great tools is because they started out as languages for small tasks. Their advantage is that they don’t need complex tools - maybe just a text editor. You can whip up a script, debug, and maintain it very fast.

In contrast, typed languages are designed from the start for large maintainable projects. The extra language features they have are there for more than the support of their tools. They support readability, organization, and compile time error checking.

In other words, why make a tool for a language that wasn’t designed for tools? Why use a scripting language for a large project? Also, why build applications on web technology designed for documents? I can actually answer that last quetion… but another day .

I had the opportunity to join in on a round table discussion podcast with Bruce Johnson, technical lead of the GWT project at Google, and Robert Hanson, keeper of the GWT widget library, hosted by Java Posse.

GWT Round Table

So aparently serialization was taking 100% of my application
cpu time. I’ve made some changes to the serialization and my apps cpu usage has gone from 100% with about 7 tables running, (approximately 60-80 clients constantaly serializing)
down to 10%. So with the same load on the server the cpu has 90% free
cycles now!

This is from making a few simple changes to
ServerSerializableTypeOracleImpl. Here’s what I did:

1)  Add a new static map to the class…

private static final Map CUSTOM_FIELD_SERIALIZERS = new HashMap();

2)  Add static initialization to the class…

static{
try{
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.boolean_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.boolean_Array_CustomFieldSer­ializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Boolean”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Boolean_CustomFieldSerialize­r”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.byte_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.byte_Array_CustomFieldSerial­izer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Byte”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Byte_CustomFieldSerializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.char_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.char_Array_CustomFieldSerial­izer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Character”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Character_CustomFieldSeriali­zer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.double_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.double_Array_CustomFieldSeri­alizer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Double”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Double_CustomFieldSerializer­”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.float_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.float_Array_CustomFieldSeria­lizer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Float”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Float_CustomFieldSerializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.int_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.int_Array_CustomFieldSeriali­zer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Integer”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Integer_CustomFieldSerialize­r”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.long_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.long_Array_CustomFieldSerial­izer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Long”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Long_CustomFieldSerializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Object_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Object_Array_CustomFieldSeri­alizer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.short_Array”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.short_Array_CustomFieldSeria­lizer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.Short”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.Short_CustomFieldSerializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.lang.String”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.lang.String_CustomFieldSerializer­”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.util.ArrayList”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.util.ArrayList_CustomFieldSeriali­zer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.util.Date”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.util.Date_CustomFieldSerializer”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.util.HashMap”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.util.HashMap_CustomFieldSerialize­r”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.util.HashSet”,
Class.forName(
“com.google.gwt.user.client.rpc.core.java.util.HashSet_CustomFieldSerialize­r”
) );
        CUSTOM_FIELD_SERIALIZERS.put(”java.util.Vector”, Class.forName(
“com.google.gwt.user.client.rpc.core.java.util.Vector_CustomFieldSerializer­”
) );
        } catch (ClassNotFoundException e) {
      // purposely ignored
    }

}

3) Change hasCustomFieldSerializer to use this map….

 public Class hasCustomFieldSerializer(Class instanceType) {
    String qualifiedTypeName;
    if (instanceType.isArray()) {
      Class componentType = instanceType.getComponentType();
      if (componentType.isPrimitive()) {
        qualifiedTypeName = “java.lang.” + componentType.getName();
      } else {
        qualifiedTypeName = Object.class.getName();
      }
      qualifiedTypeName += “_Array”;
    } else {
      qualifiedTypeName = instanceType.getName();
    }
    return (Class)CUSTOM_FIELD_SERIALIZERS.get(qualifiedTypeName);

}

Thats it! Although this limits the custom field serializaers
extensibility it works very well for me since i don’t have any custom
serializers.

I had some trouble yesterday getting this done. Here's what happened:

1. I installed the jar files, both my jar file and gwt-user in /WEB-INF/lib through ftp. However tomcat loads these jar files when the server is restarted daily at 1am. This is a bit too long to wait for updates. The server does load class files immediatly. So I extracted both jars to /WEB-INF/classes.
2. There was no need to play with a web.xml file. You can access your servlet at http://www.domainname.com/servlet/servletname, (replace domainname and servletname, not servlet).
3. Running an HTTP GET on the servlet worked. So I ran GWT generated javascript to access the server. The RPC call failed. Debugging the javascript showed that it was a 500 Internal server error: Check the server logs for details.
4. Checking the server logs revealed nothing. I Called godaddy a couple times, (thank god for free long distance on skype). They didn't know why the error didn't show up in the logs and the guy I spoke with the second time didn't know what a servlet was.
5. I dug into the GWT source. The RemoteServiceServlet catchs all exceptions in the doPost method and passes them to the respondWithFailure method which adds the exception message to the tomcat log and returns the generic 500 internal error. I modified the source to return the exception and the stack.
6. The exception was a java security restriction on java.lang.reflect and happened in ServerSerializationStream.serializeClass. In this method objects are serialized and each field is set to be accessible, that is if they are private fields they become accessible for serialization. This is what you can't do on godaddy's tomcat. The code is written such that every field's accessibility is changed whether its private or not.
7. Changing the code so that it only sets the accessibility when the field is not accessible allows applications to get by with serializing classes with only public fields when on servers with this type of security restriction. Also, if you only serialized simple types, String, int etc. you most likely won't have this problem, although I haven't tested this.

It's not easy getting out of the ghetto. Today gpokr is moving on up, off my home machine to a hosted account for $6.99/month.

It didn't work for a while but after digging fairly deap there were security restrictions and the GWT source had to be modified and recompiled for the hosting to work.

Where did gpokr go today? The answer is a string of unlikely events:

1. The gpokr server is running on my home machine because its a java servlet running on tomcat and I don't have any web server access with tomcat.
2. Rogers, providing good home internet service, changed my IP address after years with the same one. Causing my dynamic domain redirection service no-ip to point to the wrong place.
3. The point of dynamic redirection services are to update the ip address for a domain dynamically when it changes. But not today. Upon logging to to my no-ip account I see this message:

   2006/06/20, 09:17 AM : Updating hosts for Free and Enhanced services is temporarily disabled due to a small outage. It should be available again in the next few minutes. There is no need to resubmit your updates, they will go through as soon as this is rectified.

   A few minutes turned into several hours.

Its back up but the situation is so ghetto. Lets get gpokr out of the ghetto. Please recommend a good tomcat host. 

I uploaded a new version of gpokr today with some bug fixes. Check it out…

gpokr.com