I have been working away at a problem related to japanese strings in a database, character encoding, and java servlets for the past week. Not continuously but the problem has resulted in an unacceptable amount of my time being consumed and even some emotional bruising (from the java sun forum). Basically, I was getting a utf-8 encoded string from the database and I wanted my servlet to return an html or xml page, also encoded in utf-8, containing the string. I probed my program at every step to determine when the data I was getting from the database (a byte array of utf-8 character values) was turning into “???????……”. Could not figure it out until just now. Even though I had this set:

res.setContentType(”text/html; charset=utf-8″);
res.setCharacterEncoding(”utf-8″);

I also needed to specify utf-8 encoding when I made my printwriter (how the html or xml page strings are sent out):

PrintWriter pw = new PrintWriter(new OutputStreamWriter(res.getOutputStream(), “UTF-8″));

yeah…that last little bit there…, “UTF-8″ was all I needed.

I am sooooo happy that I finally figured out what I was missing but…really…all of this time. It is things like this that make me think about my life on the time/production/satisfaction scales. If I were indexed I would draw some nice little diagram resulting from these reflections but…time to move onto other things (like double checking that things I think work now are *actually* working).