Russell Beattie

Okay, so here's where I am on my J2ME app using sockets on my 7650. Stuck. :-)

To save some poor soul from throwing a week away on this topic like yours truly, I'll explain here what the deal is since it's pretty much not to be found anywhere else on the net and DEFINITELY not on Nokia's site (the bastards).

First note that I've discovered that the J2ME development platform is pretty damn cool. There's something quite satisfying about the smallness of the API, the ease of which you can throw screens together and the result of seeing your app run on your phone. It's limited in lots of ways, but for many, many things it'll actually be perfect. I'm much more excited about J2ME that I have been in the past because of this past week. Add to this the fact that there's going to be hundreds of millions of J2ME phones out there and suddenly I'm kicking myself for not spending more time developing with it last year.

Okay, that's about the API, which is cool. But the second thing I've discovered is that if you think "write once, debug everywhere" applied to regular Java apps, J2ME apps are doubly painful! Emulators are only that, emulators. And ONLY in the lightest possible use of the word "emulate" you can possibly imagine. At no point in the past week could I get my emulators to agree (I've been using Sun's, Nokia's, Siemen's and IBM's). The best bet is to get something somewhat working on your PC, then throw the app on to your phone and hope. As for development environments, I wanted to "live" in Eclipse 2.1, but I could NOT get it to "step through" the code running in the emulators. It could stop the emulator at a certain line, but that was it. So I ended up using a combination of IBM's Websphere Device Developer Edition which is basically Eclipse 2.0 plus about 180 megs of extra crud (I kid you not) and CodeWarrior's Wireless Toolkit. Both allow you to step through the code which is a life saver.

Here's an overview of my app, and others may vary but J2ME development is more or less like developing an Applet. I have one Midlet which is the main class that instantiates all the other classes which are used to draw the UI and do "stuff". There is an automagic Display class which is instantiated for you and is a singleton. This allows you to swap "screens" for the UI by calling something like display.setCurrent(screenClass); So what I did is when my midlet loads up, I instantiate all my screens (which are separate classes normally subclassing the Form class) and their buttons and then call something called JabberComm to manage the connection to Jabber and another couple classes called JabberConnection which is the connection thread and JabberParser which handles the XML processing part. I've kept all the logic out of the screen classes - all they do is set up the UI. The Items which are placed on the screen have no placement parameters, they're just popped on one after the other. It makes screen drawing pretty damn easy (if not particularly flexible).

Okay, so the problem at hand. All this would be hunky-dory if my phone acted like it should. From the mainScreen (which I loaded up initially) I have a few options, one of which is "connect" - this calls JabberComm.connect() and sets off the process of opening a socket connection to the server and opening input and outputs. Here's what the vanilla code looks like right out of a J2ME book:

public final static String CONNECT_STRING = "socket://198.78.65.25:5222";

connection = (StreamConnection) Connector.open(JabberComm.CONNECT_STRING);

outputStream = connection.openOutputStream();
inputStream = connection.openDataInputStream();

This is all wonderful, but I started out doing this in the main thread right from JabberComm. The results were that it worked in the emulators, but died on my phone, the Nokia 7650. And not died with an error, but "blocked". The J2ME app stopped working and I couldn't choose the "cancel" button I added. Nothing except Nokia's default "exit" button worked. It sucked. That's what happened on Wednesday morning.

So first solution: after a long period of trial and error, I discovered you CANNOT even look sideways at the Connector.open() method in the main thread of the Nokia app:

On the Nokia 7650, a call to Connector.open("sockets://...") in the main thread will STOP your application from running without an error.

It has nothing to do with the input or output streams, but the main StreamConnection itself. Okay, so that's where I created the JabberConnection class to be run as a thread from JabberComm. In the run() method of the JabberConnection, I put the same code as above and then I do this in JabberComm:

midlet.appendConnectingText("Starting Connection");

JabberConnection conn = new JabberConnection(this);
Thread connThread = new Thread(conn);
connThread.start();

boolean connected = conn.isConnected();
while(connected == false){
                connected = conn.isConnected();

}

if(conn.isError()){
        connectResponse(false);
        return;
}

Basically I run a loop to check to see if JabberConnection has made a connection or if there's an error. (I could probably clean this up on second look, but bare with me as I've not had any chance to go over any of this again).

This solved problem number one that the Nokia had, it was no longer locking up on me. But then I had another problem, I wasn't able to send xml to the server. (This is on my phone, the emulators are working - more or less - the entire time).

Here's what the deal was: I'm using the KXML2 libraries and they want an InputStream or a Reader as the input, then you can watch for parsing events from there. So I set off the XML parsing into another thread called JabberParser and I start it from JabberComm as well (that way JabberComm is the "parent" of both the connection and parsing threads). JabberParser takes the inputStream from the JabberConnection and passes it to the KXML2 parser and then does a loop looking for tag events. When the parser receives XML from the JabberServer, it massages it a bit and then calls back to JabberComm to present the message to the UI, etc. Still with me?

Okay, so the first thing I do after I set up both threads is send the initial XML String to the Jabber server using the JabberConnection class:

String msg = "<stream:stream xmlns:stream='http://etherx.jabber.org/streams' to='" + HOST + "' xmlns='jabber:client' >";

conn.send(msg);

The send() method simply places that String into a queue called messages (a Vector) which the main JabberConnection thread loop will pick up as it goes by:

if(!messages.isEmpty()){

        String message = (String) messages.firstElement();

        if(!message.equals("")){
                outputStream.write(message.getBytes());
                outputStream.flush();
        }

        messages.removeElementAt(0);

So again, on the emulator this works like a charm. But on the phone it doesn't. Why? Well, this is what took me ALL day yesterday and last night to figure out. The problem lies in the fact that I've passed off the inputStream to the KXML2 parser. What happens deep inside the parser is that eventually it gets to the point where it calls inputStream.read() which will grab the next byte in the downstream connection, and if it doesn't, just like any other InputStream, it'll block. Well that's all well and good, but here's the catch:

On the Nokia 7650, if the inputStream is blocked waiting for a read(), then the outputStream is blocked also.

Nice, hey? It comes from the fact that if you look above to the first bit of code, both the inputStream and the outputStream are both derived from the StreamConnection. Obviously on the Nokia 7650 has problems sending/receiving on the socket at the same time, at least from J2ME. Once I figured this out, I was able to get and receive data on my phone by not letting the read() block the write() - this is when I wrote the DID IT! post from earlier. Which as you see, was a bit premature.

The solution to this "reading blocks writing" problem was to put the inputStream.read() in the same loop as the outputStream(), then as I get characters from the inputStream, to capture the characters into my own Reader and to pass that off to the KXML parser, like so:

if(inputStream.available() > 0){

        int inputChar = inputStream.read();
        char c = (char) inputChar;

        jabberReader.addChar(c);

}

This worked like a charm in the emulator. (That phrase, by the way, is now going to be part of my general snide remarks for future projects... )

So that should solve the problem right? I should be done right? Wooohoo? Yippy? Well, nope. We come to the third and final major sockets bug on the phone:

On the Nokia 7650, a call to InputStream.available() always returns 0, regardless of how many bytes are actually available.

Joy. On the phone the loop never enters the inputStream code because available() always returns 0, if I skip that part and just call read() and there's no bytes left, the thread will stop there until there are more bites to read, but I can't REQUEST more bytes to be sent from the server because the outputStream is also blocked as well.

And THAT is where I'm at now. The solutions on the phone's side of things seem to have been run out, so now I'm looking at opening up two StreamConnections to the server (which is possible) and running some sort of JabberProxy there to read from one and write to the other. Or another idea is a server-side proxy which simply produces a byte a second of garbage which I can filter out on the phone side - the idea being to break the read() block so I can send data again. I may also see if I can somehow figure out a way of detecting if the inputStream has data on the phone end, but it's looking bleak.

The days of my vacation are growing short - just two days left and I've got less than one bit of the grand plan under way, oh well. This is how development goes. I had a choice every day when I got up: scratch the J2ME app because it was taking too long, or keep pushing through... well, it's 2:41 a.m. on Saturday morning and I never got through. Urgh.

Thanks to everyone who helped me this week - especially Jock - WHO RULES - but to all the 29 people who said hello.

I now know why Unix gurus have beards... I haven't shaved in several days now. I didn't want to waste the time, if you can believe it... I may actually go outdoors tomorrow, we'll see.

-Russ