Tuesday, April 3, 2007

java.io.StreamCorruptedException message = invalid stream header: The problem is solved

A short history of the problem: our VoIP solution is installed on some servers and some Java object are serialized and the serialized object is stored to database (in mysql TEXT field). When it is needed, the text filed is read from database, and when we are trying to deserialize it, this exception is thrown:

java.io.StreamCorruptedException: message = invalid stream header
java.io.ObjectInputStream::readStreamHeader [ObjectInputStream.java:764]
java.io.ObjectInputStream:: [ObjectInputStream.java:277]
com.ttechgroup.loggers.app.Logger::_loadSerializedObjects [Logger.java:389]
com.ttechgroup.loggers.app.Logger::getSerializable [Logger.java:199]
com.ttechgroup.loggers.SystemRestarterLogger::getCause
[SystemRestarterLogger.java:59]



My statistics are that on 50% of servers with our application, this exeption is never thrown, but on the rest, it is always thrown by my function _loadSerializedObjects.
Problem is that the bute block where is stored serialized objects, starts with 2 magic numbers (they are the header of invalid stream header)  with values > 127, so they are not standart ASCII values. Our database and database connector are set-uped to use utf-8 but the problem is in convertions of String => byte[] =>String (mey be). Here is a solution with using Base64 encoding:


A very useful article is http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4968673 Here is my "cache" of this article:

The provided test code serializes an object to a ByteArrayOutputStream, converts the generated byte array into a string using the ByteArrayOutputStream.toString() method, converts the string back into a byte array using the String.getBytes() method, and then attempts to deserialize the object from the byte array using a ByteArrayInputStream. This procedure will in most cases fail because of the transformations that take place within ByteArrayOutputStream.toString() and String.getBytes(): in order to convert the contained sequence of bytes into a string, ByteArrayOutputStream.toString() decodes the bytes according to the default charset in effect; similarly, in order to convert the string back into a sequence of bytes, String.getBytes() encodes the characters according to the default charset.

Converting bytes into characters and back again according to a given charset is generally not an identity-preserving operation. As the javadoc for the String(byte[], int, int) constructor (which is called by ByteArrayOutputStream.toString()) states, "the behavior ... when the given bytes are not valid in the default charset is unspecified". In the test case provided, the first two bytes of the serialization stream, 0xac and 0xed (see java.io.ObjectStreamConstants.STREAM_MAGIC), both get mapped to the character '?' since they are not valid in the default charset (ISO646-US in the JDK I'm running). The two '?' characters are then mapped back to the byte sequence 0x3f 0x3f in the reconstructed data stream, which do not constitute a valid header.

The solution, from the perspective of the test case, is to use ByteArrayOutputStream.toByteArray() instead of toString(), which will yield the raw byte sequence; this can then be fed directly to the ByteArrayInputStream(byte[]) constructor.


The solution

I use convertion to Base64 format before writing everything to the database. The cons of this method is that standart format of charset in database is UTF-8 or LATIN-1, 8 bit formats, so in our case we use 6 bits (64 = 2^6) so this is 33% less storage efficency, but this is most portable solution and if the space is not crytical, you can use it too!

For this I use slightly modified version of Base64InputStream and Base64OutputStream classes of package org.mozilla.jss.util. It is modified only for disatatching Asserts (and may be other little things) of no importance in this case. Here they are  Base64InputStream.java and Base64OutputStream.java

The change of source code:
Here are my changes of the source code for the
loadSerializedObjects-change.JPG
setSerializableObjectParams-change.JPG


Thanks to my boss Assen Stoyanov (softa) from TTechGroup for the idea of the solution!
If you have any difficulties with the solution, please write a comments here!








Monday, March 12, 2007

Getting started with Struts 2 and Tiles 2

I do not recommend you using Struts 2 for any reason. For more information, read the last paragraphs of the article.


I suppose this would be a easy work for 2, 3 hours but I spend some days in it, there are little information yet on internet, so I'll try to correct this.
What I've done for you - a simple war file, it is called struts2+tiles2-blank-2.0.6.zip (edit: sorry, I lost the file), so you can use it as template, base to start. It's a some kind of HOW-TO, first steps or something when you decide to build applications on Struts 2 framework with the new Tiles 2.
Versions tath I use are struts 2.0.6, tiles 2.0.6.
In zip file you have mine project with lib included, and some jsp. What it can do? You have a one page with listing fo imputed news in application, and one page for editing them. They use actions & form beans.
I disabled some jar files beceuse I will not use their funcionality - JSF, String and Plexus.
Here is a list of more critical moments for me, where I spend a lot of time:

web.xml

struts.xml

tiles.xml

struts.properties

stdLayout.jsp

listNews.jsp

editNews.jsp

NewsAction.java

News.java



One of the tricks was with tiles TLD: copy tiles-core.tld from tiles-core.jar (open it with WinRAR) and extract the TLD file to WEB-INF directory, and add in web.xml
Other trick is to remove some jars: spring plugin, jsf plugin and plexus plugin. In version struts 2.0.6 they do problems.

And a little hint, Struts 2 is based on WebWorks framework, you can search with this keyword for more results in google.

If you have questions, post comments here. I'll try to update this howto with your experience.

P.S. for UTF-8 charset managment:  In version 2.0.6 there are bugs in tiles, so setting charset / encoding of page to UTF-8 is problematic. If the lastest GA release is 2.0.6 yet, please update to lastest snapshot of struts2-core, struts2-config-browser-plugin, struts2-tiles-plugin, tiles-core, tiles-api from http://people.apache.org/builds/jakarta-struts/nightlies/2.0.x/ and http://people.apache.org/maven-snapshot-repository/org/apache/struts/tiles/tiles-core/2.0-488434-SNAPSHOT/ 
This bug is reported in https://issues.apache.org/struts/browse/WW-1724;jsessionid=8E3A7086E59002AC8802744730F17487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel




EDIT: I don't know when but I have removed the archive for kickstarting :(    I searched my linux box and my desktop PC for it, but with no result...

I will strongly recommend you NOT to use any Struts framework for your new projects, cause for time I have spend with it, I have faced with a numerous number of bugs! In my case using of General Availability releases (in simple words this means it is stable enough to be used) and a lot problems go out while doing only simple things. For example with mentioned version of Struts 2, I encounter problems with outputing text strings from resource bundle - I have used the simplest usage of that tag. Later I sam that many people complained from this! Very unserious for stable version.

Other problem with the release was using the plugin which would allow me to use Struts 1 and Struts 2 together. Another dead end - this does not work. Ohhh, shit.

Later I switched to the other big framework - Sun's JSF. For a kickstart I used AppFuse. I highly recommend you to do this way - using AppFuse. Easy, high-tech, fast, feature-rich solution! Many examples come with default installation. Everything I needed.

For JSF I can tell you only good things - stable (!!!) framework, authors have thought for many programmer needs before designing the backbone of it, so many pleasant tricks expect you if you switch to it. I have found many of my workarounds I made in Struts 1, have been implemented in a clear way in JSF. And in better way than I could imagine. So the feeling of going to JSF framework was as I have replaced my old Trabant with a new spaceship.

For a conclusion, I will highly recommend using JSF in place of Struts 2 again.

No, no one give me money to tell you this :)