Anyone who programs in Java for a short period of time becomes well aware of Java’s exceptions. To the new programmer, they may be a bit of an annoyance, but to anyone designing enterprise-level code, they are a necessity. Objective-C has exceptions too, but they are so weakly defined in the language and so overlooked, it’s almost as if they were nothing more than an afterthought. In fact, one can argue that the exception handling in C++ is superior to what one would find in Objective-C.

In Java, the exception handling is of the familiar:

try {
  <code that throws exceptions here>
} catch (ExceptionTypeA e) {
  <code to handle exception type a here>
} catch (Exception e) {
  <code to handle other exceptions here>
} finally {
  <code to execute regardless of whether an exception was thrown or not>
}


Of course, in the above, the catch statement is optional if one wants the exception to propagate outside of the block and the finally is optional if one doesn’t have code that needs to be executed in all cases. This block is even more powerful in Java 7, with its try-with-resources ability, but I’ll leave that point out for now. In Objective-C, the block looks like the following:


@try {
  <code that throws exceptions here>
} @catch (ExceptionTypeA e) {
  <code to handle exception type a here>
} @catch (Exception e) {
  <code to handle other exceptions here>
} @finally {
  <code to execute regardless of whether an exception was thrown or not>
}


It looks very similar. At a first glace, one may think that the two are the same, but appearances are deceiving. In Java, there are two different kinds of exceptions, checked and unchecked. Checked exceptions are ones that a function may throw, that all callers must handle. This is the familiar throws declaration. Objective-C only has unchecked exceptions, or runtime exceptions. Additionally, exceptions are heavily discouraged in Objective-C, to quote the documentation:

In many environments, use of exceptions is fairly commonplace. For example, you might throw an exception to signal that a routine could not execute normally—such as when a file is missing or data could not be parsed correctly. Exceptions are resource-intensive in Objective-C. You should not use exceptions for general flow-control, or simply to signify errors. Instead you should use the return value of a method or function to indicate that an error has occurred, and provide information about the problem in an error object.

One part that is missing from the above is that a thrown exception bypasses ARC, meaning that throwing an exception will very likely cause a memory leak. It should be noted that stack objects are freed and even have their destructors called in C++ when an exception is thrown. Here is a case where C++ is superior to Objective-C (I never though I’d see myself write that).

So, from this, one can conclude that the use of exceptions in Object-C is discouraged at best, worthless at worst. Apple’s documentation encouraged using an error object, but this method is very lacking. These error conditions are rarely checked in practice, and the object tends to be used again causing more errors that are not checked, and so forth. In a system using exceptions, where one is forced to handle the exceptions, the error handling can be put in a single location for all the functions in the frame which can throw that exception, avoiding the tedious error checking after every single function call.

Additionally, returning an error code is lacking in terms of context. Java’s exceptions contain stack traces and the ability to nest exceptions. One common method, especially in library code, is to contain a thrown exception inside another exception, providing additional information, and throw the new resulting exception. Then, when this exception is finally caught, a lot more information is available than a simple “this type of error occurred.”

So, what does it take to fix this? The changes are likely very significant. Apple did improve exception handling in Objective-C 2, but they still failed to bring it up to where Java has been in the beginning. Their changes seem to suggest they knew that Objective-C’s exception handling was lacking, but they didn’t invest enough resources to bring it up to par. This either means they don’t care, or that the changes were too significant. I’m going to lean toward the latter since my knowledge of the Objective-C runtimes leads me to believe that it cannot really be made to handle exceptions properly.

Impact on Runtime:
Guessing here: Significant. The runtime would need to be completely reworked to handle exceptions at every stage. ARC would need significant work and throwing/catching exceptions would need to be made significantly cheaper.

Impact on Code:
Adding checked exceptions to the code would require adding more syntax sugar to method calls and a lot of code would need to be reworked to handle exceptions instead of checking error codes. Random errors would be less likely to be swallowed, and users will have a better idea as to the true nature of their problem (no more stepping through assembly in a library tracking down what should have been a thrown exception as I’ve had to do in the past).

So, I changed the server to run Ubuntu. I wanted to be able to make the raid drives spin down, so I needed to boot off different drives. I ordered two USB flash drives, set them up in a software raid 1, and installed the OS on it. I considered running root off ZFS, but the ZFS boot in linux is more primitive. I cannot boot off raidz1 nor raidz2, so booting off the existing raid was out, if I wanted to do so. Additionally, the ZFS boot appears to not be able to boot off USB, so that pretty much killed it. Later down the road, I’ll buy a pair of SSD drives for the boot drive. It looks like there it shouldn’t have issue with booting off ZFS, in a mirror configuration.

In some ways I’m said to see FreeBSD go, but I really need more functionality with the machine. FreeBSD is a nice stable server, but it’s so difficult to make it do anything beyond the standard fair. It’s package management and port system is more difficult to use than apt-get. Also, Ubuntu used by more people than FreeBSD, so it’s a more likely target for peoples’ software development. Lastly, SWT based java projects have no problems running on it, like the one I made to stream to my GoogleTV. So, now my server is running Ubuntu going forward.

So, I bought a Roku a few months ago after my dad bought one. He got one of the first generation devices, as well as a second. It was able to play his ripped media, with a little coaxing, but the feature that caught both of our attentions was the 1080p playback. I decided to buy a second generation as well, since it had the ability to create third-party apps (they call them channels).

I started to program for the Roku, only to be first struck by the programming language. While it has similarities to several other scripting languages, it is a new programming language. Those out there familiar with the realities in computer science are screaming in the heads how this is a poor decision, but they made it anyway. I was then struck by the complete lack of library availability. Unfortunately, I had allowed myself to accept the premise that I had no choice in the matter and tried my best to plow on. My father and I had designed a Java server, which facilitated transcoding of files to be served to the Roku. I then started to add in the metadata features found in Sapphire to this server. I had browsing working from the device as well as playback initiated from the device. It was starting to look like a replacement for my AppleTV running Sapphire.

Then came the updates. Roku issues a firmware update, which caused numerous problems. The most notable was when one would play an MP3 on the first generation device (streaming a radio station for instance), then later played an MP4 with H.264, the device locked up requiring a reboot. This bug took a month and a half to fix. Additionally, the HLS streaming, which is what’s used to deliver the transcoded data to the device, was suffering from numerous audio dropouts. Supposedly, this bug was fixed in the update that was released today, but I’ve found reports of this bug in the firmware betas dating as far back as May. This means that roku was aware of the regression for 5 months before releasing the firmware. Furthermore, try to report this bug on their forums, and the fanbois descend on you like a coming plague, insisting that nothing is broken, or that a fix is coming and you must be patient. I’ve even seen some do both, even though they are diametrically apposed positions. I then noticed a firmware update on my second generation roku including some of the same regressions.

I’ve had it.

I decided to get something else. When I looked at the list of options, I narrowed my choice down to AndroidTV and GoogleTV. AndroidTV is essentially hardware vendors taking the open sourced Android OS (the versions that are), and putting in on their devices. GoogleTV is the Android OS, with some additional addons including a large host of codec/container support. I went with the latter, so I now had to pick which device. Sony makes a BluRay player and several TVs, and Logitech used to make their Revue. I went with the BluRay player since it was not discontinued and I didn’t want to buy a TV right now. Also, the GoogleTV’s new firmware is based on Android 3.1, and the AndroidTV devices are mostly on 2.2 or 2.3.

When the device arrived, I started doing some tests. It played nearly everything I threw at it without issue. I played everything over http, just giving the built-in VideoView class a URL to play, and it played it. I tested MKV, AVI, MP4, and MKV files. All had no issues. It even played the killa sample, which is something that the roku failed to play. The programming language is far better, since they used Java, and the programming environment actually works instead of the occasional lockup of the device in the debugger from which the roku suffers. It’s also much faster, and stable. My dad found that Amazon was still selling the Revue, so he bought one based on these results. I don’t know what we are going to do with our roku devices. I expect I may finish the Angry Birds levels, then call it failed experiment and box it up.

In short, don’t buy from roku. Buy a GoogleTV instead. Yes, they are more expensive, but you truly get what you pay for here. The programming environment is far superior, which means the apps will be better. The device works, and it appears that Google actually does regression testing. Additionally, Google appears to be ambitious in the devices future. It looks like GoogleTV truly is the route to go. I just wish I had seen it earlier.

BTW, I plan on fully porting Sapphire over to work on the GoogleTV in a client-server relationship. I’m doing this because I plan to allow sharing a single library across multiple devices. I don’t know yet whether I intend to release it to the public. Sometimes it is nice not having to deal with “customer” support. Browsing and playback is done, as well as an importer to read the previous version’s metadata. All that’s left is the metadata importing of new files. Time will tell on where I go with this.

I started this because one movie, I don’t remember which one, forced me to wait on FBI warning, another copyright violation warning, the same in French, and a disclaimer. Then it displayed two previews and a lengthy menu before I could hit play. This was followed by another lengthy menu animation, studio logos, and finally the movie. In the whole process, the studio logos and the previews were the only portions I was able to skip, the other screens marked as not skippable on the DVD. Now that I have my DVDs ripped, when I want to watch a movie, I browse my collection on the TV, and play the movie. No warnings, no menus, just the movie. I actually moved 8 months ago, and I have yet to unpack the DVD player.

This past weekend, I was at my parents’ house and I played a few of their BluRay disks. One movie, I think it was Flyboys, spent a considerable amount of time on the loading screen, had large sections of menus/screens that were not skippable, even included a section where it tried to download a trailer. Once the movie started playing, I paused it and to help feed some kitties and give them attention for a few minutes only to return to a custom screen saver of some sort. The movie would not resume. Upon repeating the process, a total of 15 minutes was wasted excluding the time spent with the movie paused.

So now, I seriously question the value of keeping media in these physical formats. I was considering BluRay in the near future, but now I think I’ll elect to forgo a player for an external drive and rip every disk that I might buy. Now I just need to find a good external drive that allows easy ripping. Thoughts?

I did consider FreeNAS for a really long time. It is essentially a FreeBSD install with most of the administrative work done for you through a web-based GUI. It hit most of my checkboxes in that it supported ZFS, AFP, Bonjour, and a few others. While this is nice, I found it also to be limiting when one wants to stray off the beaten path. I didn’t want to lose ZFS, but I wanted something where I could tinker. I decided to go with FreeBSD.

One of the first things I tried with FreeBSD, as the topic indicates, is to put the root filesystem on ZFS. I followed an excellent tutorial to start, but decided to make a few modifications:

• I wanted to put the root filesystem some place other than the root of the pool. Since my large media collection is going to be stored on the same pool, I wanted the ability to selectively snapshot the boot filesystem, and possibly revert it, without affecting the media partition. So, I put the root filesystem on on a sub filesystem of the pool.
• The trend in disk storage is looking like it will go to 4K sector sizes in the future. ZFS cannot migrate a pool from 512 byte sector sizes to 4K sector sizes without a reformat/restore. I wanted a 4K sector size pool from the start.
• I wanted to use ZFS swap instead of swap partitions.
• I was going to spent several iterations getting things right and didn’t want to type on that blasted 12 year old keyboard the same commands over and over.

So, I accomplished this whole task through a series of scripts, that I would modify. If the script did something wrong, I’d destroy the pool and start over. The end result is a simple setup that I decided to share with the world:

1. Download and burn/image one of the FreeBSD iso images or USB images. I used 8.2, the most recent at the time of this writing, and did the USB image since I had flash drives to spare but no DVD ROM drives.
2. Boot the machine and enter the recovery mode (Fixit).
3. Start the networking:
   mkdir /var/db
dhclient em0
ln -s /dist/usr/bin /usr/bin

4. Transfer over the script directory:
   scp -r username@10.0.0.10:Desktop/FreeBSD/\* .
   In this case, 10.0.0.10 is my laptop with sshd running.
5. Start sshd: ./FreeBSDSSHSetup.sh
6. ssh into the machine from my laptop and execute: ./FreeBSDInstall.sh
7. Reboot the server and enjoy.

The FreeBSD directory is available for those who would like to copy my efforts, though they will likely need some modification to suit your needs. The gptzfsboot, pmbr, and zfsloader files came from this site which are modified versions that work with 4K drives. The versions available in 8.2 do not work with 4K sector sizes, but future versions will. My install script

• Partitions ad8, 10, 12, 14, 16, 18 to contain the boot loader and zpool information (6 SATA drives),
• Loads kernel modules and sets up a gnop drive with 4K sectors,
• Creates a raidz2 pool using the gnop drive as one of the drives forcing 4K sectors, exports the pool,
• Destroys the gnop drive, re-import the pool, which had been forced to 4K sectors,
• Sets up root filesystem on zroot/root (zroot is the root of the pool, root is the root filesystem, as discussed above), and
• Performs a standard zroot install, except no swap partition.

I actually added the swap later, which is simply:
zfs create -V 2G zroot/swap
zfs set org.freebsd:swap=on zroot/swap
zfs set checksum=off zroot/swap


And that’s it. I have my 8TB raid running, with 2 drive redundancy, booting off the raid. Setting up AFP and the like requires more tinkering than with FreeNAS, but the instructions are easy to find and would be redundant to list here.

For one who has programmed in other object oriented languages, Objective-C stands out with its complete lack of namespace. As a result, classes have a prefix, such as Apple’s common NS and UI prefixes. On the mac side of things, every class under the sun seemed to start with NS, such as NSString, and confusion is added when on the iOS side, several classes start with UI, such as UIView. This is due to the fact that without a concept of namespace, Objective-C cannot have two classes with the same name, regardless of whether the classes are public or not.

If I were designing an application which communicated on a network as well as talked to a database, I would split the two pieces into separate segments, or in Java, packages. Inside each package, I would likely have a Connection class, which has completely different meanings within the network and database packages. Inside those packages, especially if the Connection class was internal to the package only, I could name both classes Connection without any concern and the code inside the package would immediately understand what Connection actually meant. This is possible in Java because the actual class names are com.cod3r.app.network.Connection and com.cod3r.app.database.Connection. Since Objective-C does not have namespaces, I would have to name these NetworkConnection and DatabaseConnection.

Now, take the above scenario, and add another type of network connection. Then take that new network connection and add new specific case to it, making a new class. In Java, the package name gets longer where as in Objective-C the class name gets longer. I have run into cases where this borders on ridiculous if one wants the class name to define what it actually does without ambiguity. The class is within the Sapphire plugin. In the project, all classes had the prefix Sapphire since it was a bundle included in another application and had to make sure there was never a class conflict. The project has the concept of a Directory and a VirtualDirectory was a directory which wasn’t associated with a directory in the file system. There is a type of VirtualDirectory that was Custom meaning it was defined by the user instead of already in the code. Finally, this particular class is responsible for importing these CustomVirtualDirectory objects, making the final class titled SapphireCustomVirtualDirectoryImporter. In Java, this class would have been Importer within the net.nanopi.sapphire.directory.virtual.custom package. The difference is the fully qualified classname would only be used in the import statement, and the short class name would be used in the code, instead of the really long classname used in every location as seen in Objective-C. This becomes particularly relevant when it comes to code completion that produces a terribly long list until nearly the entire class name is typed.

The lack of namespace becomes problematic when one uses third party libraries. Take the above Sapphire example and another plugin developer finds a completely unrelated library titled Sapphire. If that library happened to have a directory titled SapphireDirectory, then both plugins cannot be loaded at the same time. Java goes even further with a two-level namespace such that two jars (bundles) with the same fully-qualified classname still don’t interfere since they are in separate jars.

Adding namespace is necessary for adding inner-classes, which Objective-C outright doesn’t use, but could in proper circumstances. Inner classes are useful for implementing delegate methods. When setting a delegate, one typically passed in the instance of an inner class rather than the current instance. Not adopting this model has the issue when a single class is set to be the delegate to multiple objects, such as being the delegate to two table views. Apple works around this design by making each delegate method have the first parameter be that of the object calling it’s delegate, such as the table view. This means if the code is different between the two, every single delegate method needs have an if statement to determine which table view is being processed, and eliminates the ability of one to skip implementing an optional method for one table but not the other. Both of these problems are eliminated with inner-classes. Finally, the lack of inner-classes severely cripples the ability of a super-class to use the Key Value Observing within Objective-C since the delegate and unregister methods were written in a way that really require an inner-class to work properly (I can see how to easily break some code in a sub-class or super-class since this was designed so badly).

Impact on Runtime:
Small. I don’t know the specifics of the runtime, so I can’t judge the exact impact, but this will result in longer effective class names, which may increase lookup times. Inner-classes would need an instance variable for their outer-class to access it’s instance variables, resulting in additional pointer deference for accessing outer-class variables.

Impact on Code:
No longer need to create necessarily long class names; programmer can use short class name without concern. Frameworks need not have a class prefix. More flexibility in object design.

Objective-C still retains a lot of its heritage from it’s C beginnings. This includes using two files, a header and a source file, for each class. In a strictly object oriented environment, the header file contains the class definition (super-class and instance variables), public property definitions, and any public function declarations. The source file contains all of the function implementations, including synthesize statements. In contrast, Java contains all the functions of both files in a single file. To one who knows better, as in one who has used the single file environment, the two files for each class becomes a pain.

As to why this is a pain, one needs only to visit the following scenario: Add a property called name to the Person object. In Objective-C, one needs to add the instance variable to the class definition and the property declaration to the header file. Then, add a @synthesize statement to the implementation and add [name release]; in the dealloc (I’m neglecting Objective-C’s ability to create the instance variable in modern runtimes since it denies direct access to it in the code). In contrast, Java needs the instance variable added to the class, and in the same file add the setters and getters (which if using Eclipse, can be auto-generated). Java is able to accomplish this because it uses a two-pass compiler. One pass parses the class definition and function declarations, and the second generates the actual code. Any import statements only need the first pass of each file.

In Objective-C, some of this pain can be eliminated using my fork of ObjectiveCAnnotate, but this is only part of the way to what should be done. What Objective-C really needs is to use a single source file, and auto-generate the equivalent header file when the source file is saved (and save it only if it is different to reduce unnecessary re-compiles). This serves as the equivalent of Java’s first complier pass. Then, the actually compile step executes the second pass, using the generated header and the code within the source file. Using the paradigm, several changes should be made to the source file. The obvious is adding a public (or maybe @public) to functions that should be included in the header file. Additionally, the property syntax can be combined with the instance variable declaration and possibly with synthesize to create something like @property (nonatomic, retain, synthesize) NSString *name to take the place of the instance variable declaration, the property declaration, and the synthesize statement. Since this is by far the most common use case, why not simplify it to make things easier for the developer? The instance variable still needs to be in dealloc, but now changes in instance variables has been reduced to two places.

This change doesn’t completely eliminate the need for header files. The old C-style declarations, such as C-functions, structs, enums, externs, typedefs, and defines, would still need to be in a header file on their own. Since most classes do not make these declarations, most classes wouldn’t need a distinct header file. In the cases where such a definition is made, it is often better style to include all of the definitions in a single types header file for the framework/project.

This does necessitate a change to the import statements for a class. In the two file environment, one can separate the import statements between the header and source files, but in a single file, this separation no longer exists. However, the compiler could be made smart enough to track the locations of types it inserts in the generated header file, and include those location in the header file’s import statements. This would mean the header file would contain the import statements for the super-class, and any C-style declarations only. There is no need at all to include the import statements for other classes, as the existing @class declaration tells the compiler all it needs to know at this stage. This would also eliminate the bad coding practice of including unnecessary import statements in the header file, a practice which I’ve seen too often by new programmers who were never taught of its dangers.

Impact on Runtime:
None, this is pre-processor only

Impact on Code:
Reduction in code complexity, ease in modification. Reduction in need to reference two files to understand a single class.

When one is using object oriented design, a common practice is to lump similar classes together with a common super-class and include the common functionality in that super-class. In doing such design, a common problem is for the super-class to require some information that can only be computed by the sub-class. The solution is for the super-class to make a function call on itself which the sub-class implements. For example, I recently designed a class which simplifies storage of an object in a SQL row, but it knows nothing about the actual field names or values stored in the database. In this case, I made a function, which subclasses implement, to retrieve this data. In Java, this is simply done through an abstract method in an abstract class.

Objective-C has no concept of an abstract method or class. Instead, the closest one can get is to provide an empty implementation for a function and then override that implementation in the sub-classes. Unlike Java, there is no error, or even a warning, if the programmer of the subclass forgets to implement the function that must be overridden. The landscape becomes more confusing when one considers properties. Since Objective-C’s gives errors when an implementation doesn’t provide the property accessor methods, it provides a annotation @dynamic which silences the error. Apple mentions this can be used when an implementation is dynamically provided at runtime, but considering the complexity of this technique, it’s more common use is to silence the error in the case where a subclass implements the accessor methods. However, in the inconsistency with which Apple has treated properties, there is no error or warning, or any way to make the compiler throw one, if the sub-class programmer forgets the implementation. This makes @dynamic really dangerous (just as ignoring warnings can be dangerous).

So, what is really required to make abstract classes work in Objective-C? Again, all of the work is in the compiler and doesn’t affect runtime. First, one needs to be able to define a class as abstract, which tells the complier that allocation of that class, such as [AbstractClass alloc], should throw an error. Second, properties and methods need to be able to be defined as abstract. This will silence any errors and warnings about missing implementation in the abstract class and throw the errors and warnings about missing implementation in a non-abstract sub-class. A few changes to the complier, and a programmer can do more advanced designs and avoid simple common mistakes at compile time rather than runtime.

Impact on Runtime:
None, abstract classes and methods are syntax sugar only.

Impact on Code:
Allows more advanced design, and sub-classes become less prone to error.

The other parts can be found here:
Part 1 (Generics)
Part 2 (Abstract Classes)
Part 3 (Single Source File)
Part 4 (Namespace)
Part 5 (Exceptions)

The first item which Objective-C can learn from Java is Generics. Objective-C is a weakly typed language, too weak for my tastes, whereas Java is strongly typed. There are distinct advantages to a strongly typed language, but I’m not going to get into that here. This means that Objective-C’s can make a function call on an object without knowing its type where as Java must have the object typed to an interface or class which defines that function. This requires typecasting in Java before making function calls, where as Objective-C does not. In some cases this throws a warning in Objective-C, and in some cases does not. Missing warnings can lead to programming mistakes, and programmer ignored warnings leads to bad style. In either case, typecasting will change the mistakes to warnings, and the warnings which are not mistakes to no warnings.

Java had the issue in requiring typecasting to make function calls. This lead to numerous typecasts in code, which added to clutter. Java’s solution to much of this clutter came through the use of generics. One of the largest sets of typecasts resulting in fetches from a collection. For example, if one had a list which contains objects of class Person, then it would be convenient if the complier just remembered that for you. This was done by simply declaring the list as List<Person>. Then, when fetching objects from the List, no typecast is necessary since the compiler already knows that all objects in that list are a Person in the first place. I should add, for the C++ programmers out there, this is not a template. Templates define a whole new set of code for that type, ballooning the object binary, where as generics can be thought of as automatic typecasting.

I have found many arguments that state Objective-C doesn’t need generics because it is a weakly typed language (just Google Objective-C generics to see them). These arguments tend to forget one major piece of the language. Consider the following code:


@interface Person : NSObject {
    NSString   *name;
    NSArray    *siblings;
    Person     *father;
    Person     *mother;
    NSArray    *children;
}

@end


In this example, I’ve defined an Objective-C class called Person with a name, an array of siblings, a father, mother, and an array of children. Now, say I wanted to get the first-born child’s name. I could do this with [children objectAtIndex:0].name. Except, this produces an error because [children objectAtIndex:0] returns an id, not a Person *, and properties, unlike functions, are strongly typed in Objective-C. So, the recourse is to either use a temporary variable of type Person *, or to typecast on the same line like: ((Person *)[children objectAtIndex:0]).name. If Objective-C had generics, this would eliminate any need for the typecast or temporary variable since the compiler would already know the type of the object returned by the array.

This is a simple example of where generics become useful in the language, but there are far more. I have actually found Objective-C’s lack of generics limiting my ability to design class hierarchies, and requiring a large amount of unnecessary typecasts or in some cases, additional function calls, which produces slower code.

Should Objective-C adopt all of Java’s generics? I would contend the answer is no. Java’s generics can get very convoluted very quickly, and the biggest source of the mess is in defining a generic function call. While they add convenience, generics in function calls becomes very messy. However, generics on a class level (and used in it’s functions) add so much benefit and cleaner code design that Objective-C is really suffering without them. Additionally, Java produces warnings if a class containing generics does not have it’s generic type defined. For example, List children will produce a warning because I didn’t define the type of object contained within list. In these cases, since Objective-C is so weakly typed, it’d be most appropriate if its generics just defaulted to the base class allowed in the generic, which in the case of NSArray would be id.

Impact on Runtime:
None, generics are syntax sugar only.

Impact on Code:
Generics added where needed, changing the tracking of object type to the compiler instead of the programmer. Cleaner code, and less prone to error.

While fast-cgi is faster, it has inherent issues, such as any memory leak can result in ever growing memory usage, which is exactly why Apache has a setting for each child to serve a limited number of requests before exiting. Trac.fcgi has no such directive, and has the equivalent of a large memory leak, a non-expiring cache. While it’s not as bad as a memory leak, which will indefinitely grow instead of reaching a limit, if the cache size is larger than the available memory for trac to use, it’s just as serious. The only solution, without fixing trac’s caching mechanisms, is to restart trac periodically, but during the time trac is restarting, all requests are lost, causing bad gateway errors to the user. Additionally, the restart needs to be done manually. Clearly not an ideal solution.

The ideal solution would be for the trac process to be periodically restarted, but all requests be successfully completed. This is what Apache accomplishes with its children, but trac has no such mechanism or even the support for one. So, I had to build it in myself.

My Solution:
First piece is to create a parent process, which holds the fcgi socket, and restarts a child trac process when it dies. This ensures that all waiting requests will be served by either the old or new process. Such a parent process absolutely must have no memory leaks, and so I created one that has only 1 explicit allocation, and it is executed only once.

The second piece, and one that’s considerably harder, is to make the trac process exit gracefully when it’s memory usage gets too big. The first step was to create a subclass of WSGIServer and override its _mainloopPeriodic to run a periodic check. In this check, I do a memory usage check, and if it’s over 90MB, set itself to exit. The problem is there’s no easy way to figure out the memory usage on linux. There is a function, getrusage, which is supposed to give resource usage information, such as memory, but linux gives all zeros (unlike a proper kernel). The only way to get this is to read the information out of /proc, and parse that data. Since this becomes a more expensive operation, I only conduct the test every 100 times.
After doing this, I was still getting periodic bad gateway errors. It turns out that trac spawns a thread to process the request, and that request hadn’t completed when the process exited, dropping the connection and causing the error. In examining the documentation, Python is supposed to wait for the thread to complete before exit. Since it wasn’t, I put in a mechanism to see if any threads are running before exit. Here lies a big problem with Python. I found out that the thread, while created, hasn’t actually started. Since it hasn’t started, it isn’t running, which is why Python exited. Furthermore, Python’s threading is so brain-dead, there seems to be no way at all to differentiate between a thread which hasn’t started and one that is exited but not freed. This means there is no reliable way to detect if all threads have exited. So instead, in order to work around Python, I created a thread-safe counter to count the number of threads. I increment it when the thread is created (not started), and decrement it when the thread completes. I then only allow the main thread to exit when this counter reaches zero (since the main thread does the allocations, this never lets the process die without starting all threads). Given this glaringly bad threading model, I put in another protection mechanism so that the main thread will exit after 30 seconds even if the count isn’t zero, just in case.

With the above two pieces, trac’s memory usage is limited, and no connections are dropped, in the time between one process deciding to go down and when it actually does, nothing is processing requests. So, the last piece is to make trac signal to the parent process that it has decided to exit, and then have the parent process launch a new trac to take over while the previous is exiting. I did this with USR1 signal, where the parent process sends a HUP to the child (in case someone else sent it the USR1 signal), and start a new child. With these modifications in place, trac has been humming along for nearly a month, being restarted about 2-3 times a day with no issues.

Files:
launcher.c – Requires the environment variable WORKER_PATH to be set to know what process to launch. Best run with something like spawn-fcgi
trac.fcgi – Modified trac.fcgi to incorporate the above mentioned changes, complete with commented code for testing/experimentation.
Enjoy