Russell Beattie

Okay, so here's how I got Ant to compile without dying. I case you hadn't read the previous posts, I'm trying to compile 7000 files and that I'm stuck using JDK 1.2.2 which isn't efficient at all, so it would run out of memory while compiling. Actually, I've come to find out that the compiler would choke only on certain sections of the files for some reason. I narrowed it down to two directories which contained (in subfolders) only a 1000 files... so you never know what the deal is.

Anyways, I needed to break it up and asked for help on Ant and got some great responses. Finally, here's how I ended up working this bad boy out:

First, I created a standard Ant build.xml and build.properties files. Nothing fancy. The trick is I had to go get and compile a "ForEach" task from the ant-contrib project in SourceForge. I compiled it and plopped the jar into the Ant directory and then I could use it from the build.xml as an additional task. I have NO idea why this isn't included in the build except for some esoteric "loops considered bad in make files" opinions...

With a little help from Jim, I got a nice list of the packages inside my source directory ("dir /ad /b /s") that I could copy, paste and massage into a comma separated list of directories and put it into the build.properties as a variable called packages.

Here's the vital parts of my ant script:

        <taskdef name="forEach" 
                classname="net.sf.antcontrib.logic.ForEach"/>

        ...

        <target name="compileAll" depends="init, createDirs">
                <record name="log-${DSTAMP}-${TSTAMP}.log" action="start"/>
                <forEach list="${packages}" 
                                target="compileEach" 
                                param="package" 
                                inheritall="true"/>
                <record name="log-${DSTAMP}-${TSTAMP}.log" action="stop"/>
        </target>

        <target name="compileEach">
                <echo message="Compiling ${src}${package}" />
                <javac  srcdir="${src}${package}" 
                        destdir="${classes}" 
                        classpath="${classpath}"
                        sourcepath="${src}" 
                        verbose = "false" 
                        excludes="*/**/*.java"
                        fork="true" 
                        memoryInitialSize="100m" 
                        memoryMaximumSize="256m" 
                        listFiles = "false"
                        failOnError = "false"
                />
                <copy  todir="${classes}">
                    <fileset dir="${src}" 
                                includes="**/*.properties"/>
                </copy>

There's a couple tricks in that script. First the ForEach calls a task for every item in the packages variable. It's important to include the inheritall, otherwise your other script variables (like classpath) don't get passed through. Then, because javac normally recursively compiles, in order to make sure that I don't hit a parent folder with some classes, but that also has a million files in subfolders, I added the excludes="*/**/*.java" which makes sure that I only compile one directory at a time. Then to make sure that the compiler can find everything it needs, even if it's only focused on one directory, I added the sourcepath. Then I wanted to make sure I do everything so I can see all the errors in one list, and that's why I turned off "failOnError". The other stuff is in there for testing.

It works pretty well. Before they had a dedicated machine with a Gig of RAM and wacky DOS/Unix scripts which took several hours to compile everything. Now I can do all this on my machine with 256megs of Ram in 17 minutes. Yeah!

I'm hoping to push for JDK 1.4 which has a much more efficient compiler and doesn't need to do these tricks. Jikes compiled the whole thing without (many) problems also in 10 minutes. Jikes is a bitchy compiler though and gives me errors on mistakes like accidental ";;" in the code which don't affect anything.

Let's hope this is the solution and that I'm not missing anything. One thing I noticed is that a directory says it has 68 files, but the while compiling it says it's only compiling 61. I'm hoping this is because it already compiled some of the classes because of links to previous packages. I hope. ;-)

-Russ

P.S. My bad day has nothing to do with technology...