Objective-C still retains a lot of its heritage from it’s C beginnings. This includes using two files, a header and a source file, for each class. In a strictly object oriented environment, the header file contains the class definition (super-class and instance variables), public property definitions, and any public function declarations. The source file contains all of the function implementations, including synthesize statements. In contrast, Java contains all the functions of both files in a single file. To one who knows better, as in one who has used the single file environment, the two files for each class becomes a pain.

As to why this is a pain, one needs only to visit the following scenario: Add a property called name to the Person object. In Objective-C, one needs to add the instance variable to the class definition and the property declaration to the header file. Then, add a @synthesize statement to the implementation and add [name release]; in the dealloc (I’m neglecting Objective-C’s ability to create the instance variable in modern runtimes since it denies direct access to it in the code). In contrast, Java needs the instance variable added to the class, and in the same file add the setters and getters (which if using Eclipse, can be auto-generated). Java is able to accomplish this because it uses a two-pass compiler. One pass parses the class definition and function declarations, and the second generates the actual code. Any import statements only need the first pass of each file.

In Objective-C, some of this pain can be eliminated using my fork of ObjectiveCAnnotate, but this is only part of the way to what should be done. What Objective-C really needs is to use a single source file, and auto-generate the equivalent header file when the source file is saved (and save it only if it is different to reduce unnecessary re-compiles). This serves as the equivalent of Java’s first complier pass. Then, the actually compile step executes the second pass, using the generated header and the code within the source file. Using the paradigm, several changes should be made to the source file. The obvious is adding a public (or maybe @public) to functions that should be included in the header file. Additionally, the property syntax can be combined with the instance variable declaration and possibly with synthesize to create something like @property (nonatomic, retain, synthesize) NSString *name to take the place of the instance variable declaration, the property declaration, and the synthesize statement. Since this is by far the most common use case, why not simplify it to make things easier for the developer? The instance variable still needs to be in dealloc, but now changes in instance variables has been reduced to two places.

This change doesn’t completely eliminate the need for header files. The old C-style declarations, such as C-functions, structs, enums, externs, typedefs, and defines, would still need to be in a header file on their own. Since most classes do not make these declarations, most classes wouldn’t need a distinct header file. In the cases where such a definition is made, it is often better style to include all of the definitions in a single types header file for the framework/project.

This does necessitate a change to the import statements for a class. In the two file environment, one can separate the import statements between the header and source files, but in a single file, this separation no longer exists. However, the compiler could be made smart enough to track the locations of types it inserts in the generated header file, and include those location in the header file’s import statements. This would mean the header file would contain the import statements for the super-class, and any C-style declarations only. There is no need at all to include the import statements for other classes, as the existing @class declaration tells the compiler all it needs to know at this stage. This would also eliminate the bad coding practice of including unnecessary import statements in the header file, a practice which I’ve seen too often by new programmers who were never taught of its dangers.

Impact on Runtime:
None, this is pre-processor only

Impact on Code:
Reduction in code complexity, ease in modification. Reduction in need to reference two files to understand a single class.

When one is using object oriented design, a common practice is to lump similar classes together with a common super-class and include the common functionality in that super-class. In doing such design, a common problem is for the super-class to require some information that can only be computed by the sub-class. The solution is for the super-class to make a function call on itself which the sub-class implements. For example, I recently designed a class which simplifies storage of an object in a SQL row, but it knows nothing about the actual field names or values stored in the database. In this case, I made a function, which subclasses implement, to retrieve this data. In Java, this is simply done through an abstract method in an abstract class.

Objective-C has no concept of an abstract method or class. Instead, the closest one can get is to provide an empty implementation for a function and then override that implementation in the sub-classes. Unlike Java, there is no error, or even a warning, if the programmer of the subclass forgets to implement the function that must be overridden. The landscape becomes more confusing when one considers properties. Since Objective-C’s gives errors when an implementation doesn’t provide the property accessor methods, it provides a annotation @dynamic which silences the error. Apple mentions this can be used when an implementation is dynamically provided at runtime, but considering the complexity of this technique, it’s more common use is to silence the error in the case where a subclass implements the accessor methods. However, in the inconsistency with which Apple has treated properties, there is no error or warning, or any way to make the compiler throw one, if the sub-class programmer forgets the implementation. This makes @dynamic really dangerous (just as ignoring warnings can be dangerous).

So, what is really required to make abstract classes work in Objective-C? Again, all of the work is in the compiler and doesn’t affect runtime. First, one needs to be able to define a class as abstract, which tells the complier that allocation of that class, such as [AbstractClass alloc], should throw an error. Second, properties and methods need to be able to be defined as abstract. This will silence any errors and warnings about missing implementation in the abstract class and throw the errors and warnings about missing implementation in a non-abstract sub-class. A few changes to the complier, and a programmer can do more advanced designs and avoid simple common mistakes at compile time rather than runtime.

Impact on Runtime:
None, abstract classes and methods are syntax sugar only.

Impact on Code:
Allows more advanced design, and sub-classes become less prone to error.

The other parts can be found here:
Part 1 (Generics)
Part 2 (Abstract Classes)
Part 3 (Single Source File)
Part 4 (Namespace)

The first item which Objective-C can learn from Java is Generics. Objective-C is a weakly typed language, too weak for my tastes, whereas Java is strongly typed. There are distinct advantages to a strongly typed language, but I’m not going to get into that here. This means that Objective-C’s can make a function call on an object without knowing its type where as Java must have the object typed to an interface or class which defines that function. This requires typecasting in Java before making function calls, where as Objective-C does not. In some cases this throws a warning in Objective-C, and in some cases does not. Missing warnings can lead to programming mistakes, and programmer ignored warnings leads to bad style. In either case, typecasting will change the mistakes to warnings, and the warnings which are not mistakes to no warnings.

Java had the issue in requiring typecasting to make function calls. This lead to numerous typecasts in code, which added to clutter. Java’s solution to much of this clutter came through the use of generics. One of the largest sets of typecasts resulting in fetches from a collection. For example, if one had a list which contains objects of class Person, then it would be convenient if the complier just remembered that for you. This was done by simply declaring the list as List<Person>. Then, when fetching objects from the List, no typecast is necessary since the compiler already knows that all objects in that list are a Person in the first place. I should add, for the C++ programmers out there, this is not a template. Templates define a whole new set of code for that type, ballooning the object binary, where as generics can be thought of as automatic typecasting.

I have found many arguments that state Objective-C doesn’t need generics because it is a weakly typed language (just Google Objective-C generics to see them). These arguments tend to forget one major piece of the language. Consider the following code:


@interface Person : NSObject {
    NSString   *name;
    NSArray    *siblings;
    Person     *father;
    Person     *mother;
    NSArray    *children;
}

@end


In this example, I’ve defined an Objective-C class called Person with a name, an array of siblings, a father, mother, and an array of children. Now, say I wanted to get the first-born child’s name. I could do this with [children objectAtIndex:0].name. Except, this produces an error because [children objectAtIndex:0] returns an id, not a Person *, and properties, unlike functions, are strongly typed in Objective-C. So, the recourse is to either use a temporary variable of type Person *, or to typecast on the same line like: ((Person *)[children objectAtIndex:0]).name. If Objective-C had generics, this would eliminate any need for the typecast or temporary variable since the compiler would already know the type of the object returned by the array.

This is a simple example of where generics become useful in the language, but there are far more. I have actually found Objective-C’s lack of generics limiting my ability to design class hierarchies, and requiring a large amount of unnecessary typecasts or in some cases, additional function calls, which produces slower code.

Should Objective-C adopt all of Java’s generics? I would contend the answer is no. Java’s generics can get very convoluted very quickly, and the biggest source of the mess is in defining a generic function call. While they add convenience, generics in function calls becomes very messy. However, generics on a class level (and used in it’s functions) add so much benefit and cleaner code design that Objective-C is really suffering without them. Additionally, Java produces warnings if a class containing generics does not have it’s generic type defined. For example, List children will produce a warning because I didn’t define the type of object contained within list. In these cases, since Objective-C is so weakly typed, it’d be most appropriate if its generics just defaulted to the base class allowed in the generic, which in the case of NSArray would be id.

Impact on Runtime:
None, generics are syntax sugar only.

Impact on Code:
Generics added where needed, changing the tracking of object type to the compiler instead of the programmer. Cleaner code, and less prone to error.

While fast-cgi is faster, it has inherent issues, such as any memory leak can result in ever growing memory usage, which is exactly why Apache has a setting for each child to serve a limited number of requests before exiting. Trac.fcgi has no such directive, and has the equivalent of a large memory leak, a non-expiring cache. While it’s not as bad as a memory leak, which will indefinitely grow instead of reaching a limit, if the cache size is larger than the available memory for trac to use, it’s just as serious. The only solution, without fixing trac’s caching mechanisms, is to restart trac periodically, but during the time trac is restarting, all requests are lost, causing bad gateway errors to the user. Additionally, the restart needs to be done manually. Clearly not an ideal solution.

The ideal solution would be for the trac process to be periodically restarted, but all requests be successfully completed. This is what Apache accomplishes with its children, but trac has no such mechanism or even the support for one. So, I had to build it in myself.

My Solution:
First piece is to create a parent process, which holds the fcgi socket, and restarts a child trac process when it dies. This ensures that all waiting requests will be served by either the old or new process. Such a parent process absolutely must have no memory leaks, and so I created one that has only 1 explicit allocation, and it is executed only once.

The second piece, and one that’s considerably harder, is to make the trac process exit gracefully when it’s memory usage gets too big. The first step was to create a subclass of WSGIServer and override its _mainloopPeriodic to run a periodic check. In this check, I do a memory usage check, and if it’s over 90MB, set itself to exit. The problem is there’s no easy way to figure out the memory usage on linux. There is a function, getrusage, which is supposed to give resource usage information, such as memory, but linux gives all zeros (unlike a proper kernel). The only way to get this is to read the information out of /proc, and parse that data. Since this becomes a more expensive operation, I only conduct the test every 100 times.
After doing this, I was still getting periodic bad gateway errors. It turns out that trac spawns a thread to process the request, and that request hadn’t completed when the process exited, dropping the connection and causing the error. In examining the documentation, Python is supposed to wait for the thread to complete before exit. Since it wasn’t, I put in a mechanism to see if any threads are running before exit. Here lies a big problem with Python. I found out that the thread, while created, hasn’t actually started. Since it hasn’t started, it isn’t running, which is why Python exited. Furthermore, Python’s threading is so brain-dead, there seems to be no way at all to differentiate between a thread which hasn’t started and one that is exited but not freed. This means there is no reliable way to detect if all threads have exited. So instead, in order to work around Python, I created a thread-safe counter to count the number of threads. I increment it when the thread is created (not started), and decrement it when the thread completes. I then only allow the main thread to exit when this counter reaches zero (since the main thread does the allocations, this never lets the process die without starting all threads). Given this glaringly bad threading model, I put in another protection mechanism so that the main thread will exit after 30 seconds even if the count isn’t zero, just in case.

With the above two pieces, trac’s memory usage is limited, and no connections are dropped, in the time between one process deciding to go down and when it actually does, nothing is processing requests. So, the last piece is to make trac signal to the parent process that it has decided to exit, and then have the parent process launch a new trac to take over while the previous is exiting. I did this with USR1 signal, where the parent process sends a HUP to the child (in case someone else sent it the USR1 signal), and start a new child. With these modifications in place, trac has been humming along for nearly a month, being restarted about 2-3 times a day with no issues.

Files:
launcher.c – Requires the environment variable WORKER_PATH to be set to know what process to launch. Best run with something like spawn-fcgi
trac.fcgi – Modified trac.fcgi to incorporate the above mentioned changes, complete with commented code for testing/experimentation.
Enjoy

Edit: My solution that was here previously didn’t work properly. I’ve developed a Safari Extension to correct this, and am currently testing it. If it passes the tests, I’ll likely put it up here.

Sometime soon, I’ll get around to releasing my code for fetching old Escape Pod episodes that I hinted at earlier.

Kevin sent me the code he used for DVD playback inside NitoTV as a place to start. I stripped it down to a smaller piece, and started the framework. After I had it doing basic playback, I worked on overlays to provide feedback to the user. Now, hitting up and down changes the overlays between normal, chapter view, audio/subtitle selection, and zoom.

Chapter Overlay Sceenshot

Also, I added a pretty menu using a blurred image as the resume menu. This is more consistent with Apple’s own playback

Resume Overlay

In addition, I used my work on AC3 Passthrough in Perian to see if I could pull off the same in DVD Playback. The issue is the AppleTV claims to only have a device which can play uncompressed audio, not one that can send Dolby Digital to a decoder. So, I created an audio driver that claims to do exactly that and pipe the data through to the optical/HDMI port. It was a lot of fun getting that working, and then I added support for DTS.

Anyway, now Sapphire uses the common media player framework for DVD playback. In the future, we’ll add other playback mechanisms as well.

In my test case, I worked with the NASB translation of the Bible. The raw text of this translation, minus formatting and book/chapter/verse identifiers is 3.965MB. Since the iPhone already has zlib, using gzip compression is an obvious choice. When compressed with gzip, the file size becomes 1.189MB, a significant savings. Even though bzip2 is not readily available on the iPhone (at least not that I could find), I tested its compression which produced a file size of 0.8548MB. While these mechanisms provide a significantly smaller file, when one desires a certain portion of the file, one must first decompress the entire file up to that point. This is an expensive operation on a small device such as the iPhone.

One compromise is to compress each book individually. This yields a file size of 1.242MB. However, some books, such as Psalms, are still quite large, requiring a long decompression operation to read some of it’s chapters. Since the application displays a chapter at a time, a logical compression block would be a single chapter which yields a file size of 1.628MB. While this is a large increase in file size, a single chapter can be decompressed without requiring decompression of any other parts of the file.

I was not happy with these compression ratios and was determined to find a better way. I looked into Huffman encoding, which uses a variable length string of bits to represent a symbol. Its compression ratio does not change with the order of symbols, which typically means it won’t provide as high a compression ratio. However, if you are told which bit is the start of a string, you may begin decompression without examining any of the file which precedes it. In addition, the decompression scheme only requires walking a binary tree, which means it is also quite fast. So, if Huffman encoding can be made to provide a good compression ratio, it will work well for this problem.

When using Huffman encoded, the question becomes how to decompose the string into symbols. One typical decomposition is to make each character into a symbol. While this is an easy representation, it doesn’t provide a good compression ratio. Instead, I chose to make each word into a symbols. After adding in punctuation, the tree contained 16,246 words. This resulted in a dictionary size of 0.1364MB. When compressed, the entire Bible was represented by a Huffman encoding of 0.9947MB, meaning a total of 1.131MB for the compressed stream and dictionary.

After this experiment, I concluded that Huffman encoding of words is the best for a large quantity of text. It yielded a compression ratio that was better than gzip (though not as good as bzip2), but at the same time a scheme that could decompress individual parts of the file without having to read preceding parts. In this scheme, a single chapter can be decompressed very quickly, and still have a high compression ratio.

So, my options are:

• Only use the command line utility, or use Xcode, on a single repository checkout, but never both.
• Use a horribly out of date version of Subversion
• Hack Xcode’s plugin to work with newer versions of Subversion

I chose the last, as it should give me the best of both worlds. If you were to conduct a quick Google search, you would find a whole host of people posing a solution which involves moving around system libraries!!!!! This is a horrible idea, as it breaks other things such as Apache. Buried somewhere in the sea of bad ideas, I ran across a good one, written by Jean-Daniel Dupas and improved by Philippe Casgrain which only changes the path location of libraries used by Xcode’s Subversion plugin. This script worked well for Subversion 1.5, but recent versions of Subversion would cause Xcode to crash in libapr. The solution is to add libapr and libaprutil to the script. Here is the corrected script for your use.

#!/bin/sh

echo "Updating install path using svn libraries in \"$2\"..."
install_name_tool -change \
/usr/lib/libapr-1.0.dylib \
$2/lib/libapr-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libaprutil-1.0.dylib \
$2/lib/libaprutil-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_client-1.0.dylib \
$2/lib/libsvn_client-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_delta-1.0.dylib \
$2/lib/libsvn_delta-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_diff-1.0.dylib \
$2/lib/libsvn_diff-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_fs_fs-1.0.dylib \
$2/lib/libsvn_fs_fs-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_fs-1.0.dylib \
$2/lib/libsvn_fs-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_ra_local-1.0.dylib \
$2/lib/libsvn_ra_local-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_ra_svn-1.0.dylib \
$2/lib/libsvn_ra_svn-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_ra-1.0.dylib \
$2/lib/libsvn_ra-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_repos-1.0.dylib \
$2/lib/libsvn_repos-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_subr-1.0.dylib \
$2/lib/libsvn_subr-1.0.dylib "$1"
install_name_tool -change \
/usr/lib/libsvn_wc-1.0.dylib \
$2/lib/libsvn_wc-1.0.dylib "$1"
echo "Done!"


Now, how long till Xcode supports Mercurial?