On the search for an easy way to implement a syntax highlighting mechanism for NSTextView I found an easy and fast (regarding runtime) way to do this. By using flex (a tool with its roots in the 1970's) defining the rules of what should be highlighted makes it a lot more easier and structured.
flex is the GNU version of lex, the lexical analyzer generator (Buzzword jackpot!) developed by Eric Schmidt and Mike Lesk in 1975. By defining some rules (mainly regular expressions) it generates a C program which allows us to scan through a text and do something with the text, e.g. divide it into tokens. Tokens are chunks of characters with a special meaning (as defined in our rules). Confused? Read on, this concept will become very clear soon.

In this article I want to show you how we can use this ancient but very powerful tool to implement a basic syntax highlighting within a NSTextView.

## Overview
At first, let's have a look at the goals. In the demo application (see below) we have one single window with a NSTextView where we can write some text. This text is parsed and highlighted in the following way:

* positive real number will be colored green
* negative real numbers will be colored red

A real number can be entered in the following formats:

* 12
* -3.54
* 23e2
* 23E2
* 423.2e-3
* -5e-1

For the regular expression gurus of you the format can be defined like this:

[-+]?[0-9]+(\.[0-9]+)?([eE][-+]?[0-9]+)?

In addition we want to sum up all found real numbers and show the result in our application.

I'd recommend you to download the source code and read the comments there for additional information.
This application consists of three parts:

* NSTextView (for the text) and NSTextField (to show the total value),
* the NSTextStorage delegate and
* the actual scanner created with flex

As you can see, we don't need to subclass NSTextView to add highlighting, everything is done in the NSTextStorage delegate and the scanner.

## NSTextView and NSTextField
The NSTextView is only used for the user to input some text. We don't need to change its behavior nor do we need to subclass it. The NSTextField at the bottom is used to show the total of the collected real number. Nothing special here either.

## NSTextStorage delegate
The NSTextStorage delegte is responsible to pass the entered text to the scanner, to retrieve the token information and to add the string attributes (e.g. the color) to the original text.
The main part is a while-loop which fetches all tokens and if the token indicates a real number, an NSForegroundColorAttribute attribute is added to the text. That's all we have to in the delegate in principle.

## The scanner
The scanner is the heart of this implementation. It takes an ordinary C string and parses through it returning tokens if something interesting is found. Actually, it returns tokens for everything it could find (whether it is text or a real number). This behavior is needed in the delegate to determine the range.
To write such a scanner the flex tool becomes VERY handy, because you can define the behavior of the scanner with a fistful rules. Flex takes these rules and generates a more or less large C file (*lex.yy.c*) which is then the actual scanner.
Xcode supports flex files directly by adding a *.l* file to the project. In the build process Xcode runs flex to "translate" it to C code and it also compiles this C code. We don't have to do anything.
I'd suggest you to read the [flex manual](http://www.gnu.org/software/flex/manual/) to get familiar with flex as I won't explain it in detail. Flex is a really cool tool and to deal with it for some time won't harm.

Unfortunately there is one problem with flex: It doesn't know UTF-8 characters. As you might know UTF-8 characters consist of 2 to 4 bytes and flex thinks every single byte is one character.
But, there is a workaround to this problem. What we can do is to capture all those bytes and return them as on "byte-string". Our delegate will read those bytes in and recognize them as one UTF-8 character.
The designers of UTF-8 were clever and hid the length of the UTF-8 character in the first byte. So, after reading the first UTF-8 byte we can determine how many bytes to follow (more information [here](http://www.unicode.org/versions/Unicode5.0.0/ch03.pdf), page 103). This buffer mechanism requires some additional variables and functions the scanner.

The regular expression mentioned above can easily be split up into more readable parts on the *flex.l* file. Changes to the rules can be applied easily this way.

## Download
The demo application source code can be downloaded via [Git](http://www.git-scm.com):

git clone git://github.com/simonboots/CaptureReals.git

A universal binary is also available [here](http://www.stiefels.net/wp-content/uploads/2008/01/Capture%20Reals.zip).

## Summary
If you're interested in adding a real fast syntax highlighting to your application, doing this with flex should be one option to be considered. It may not be the perfect solution (UTF-8 workaround, missing runtime flexibility) but it is very easy to implement and may fit your requirements.
I hope this article was at least a little bit helpful. Please let me know what you think about it.
Thanks for reading!

[Late Night Cocoa](http://www.latenightcocoa.com) is a great podcast about Cocoa and Mac OS X development and if you're interested in these topics and haven't heard about Late Night Cocoa yet, [fire up your iTunes](http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewPodcast?id=213023580)
and give it a try.

Every week the producer Steve "Scotty" Scott invites well known names in the Cocoa community (such as [Aaron Hillegass](http://www.bignerdranch.com), [Scott Stevenson](http://www.theocacao.com/) or [Daniel Jalkut](http://www.red-sweater.com/blog/)) to have a talk about a Cocoa-related topic. It goes from Xcode projects, over CoreData to Memory Management and so on.
It is very interesting and fun to listen to Scotty and his guests and there are a lots of tips on how you can improve your Cocoa development.

For the last three months (that's the time since Late Night Cocoa is on air) Scotty spends about 12 hours a week (!) to produce one episode of this high-quality podcast. So if you like it (as I do), please go to the [Late Night Cocoa website](http://www.latenightcocoa.com) and make a small donation.

Thank you Scotty!

It's called RegexTester (yeah, innovative name, I know) and is intended to be a help for developers who have to formulate regular expressions.
More information can be found [here](/projects/regextester).

Have fun!

Chris is an [Adium](http://www.adiumx.com) developer and provides a great service called [Weekly Cocoa App Challenge](http://brok3n.org/weeklycocoa.html):
Every week Chris posts a small Cocoa application which you have to (try to) re-develop.
After a week Chris pubishes the source code to this application so you can compare your code with the one from the expert.
Practice makes perfect, with Chris' help even easier.

NSPredicate was introduced in OS X 10.4 and is mainly used in CoreData applications to specify queries and has therefore also some nice functionality to filter arrays.
But there is also a way to use its built-in regular expression mechanism to validate NSStrings.

Let's say you have a string "Hello World!" and you want to check if there are at least two sequent l's in it. The regular expression would be: ".\*l{2,}.\*" (without quotes).
Note, that there is a dot followed by an asterisk (which means "zero or more characters") at the beginning and at the end of the actual expression (l{2,}). This is necessary because (unlike the regular expressions in Perl) the "interpolated length" of the regular expression must be at least the same as the length of the string to test to (I don't know why but that's what I found out during my tests).

In your Cocoa application this test would look like this:

NSString *mystring = @"Hello World!";
NSString *regex = @".*l{2,}.*";

NSPredicate *regextest = [NSPredicate
predicateWithFormat:@"SELF MATCHES %@", regex];

if ([regextest evaluateWithObject:mystring] == YES) {
NSLog(@"Match!");
} else {
NSLog(@"No match!");
}

This is easy, isn't it?
All you have to do is to create a NSPredicate object with the predicate "SELF MATCHES" followed by a regular expression and call its "evaluateWithObject:" method with your string you want to validate. It returns YES if the string matches your regular expression and NO if it does not.

If you don't like this approach or if you don't want your application to require OS X 10.4 you can find some other solutions and documentation regarding regular expressions in Cocoa at [this webpage](http://www.cocoadev.com/index.pl?RegularExpressions).

Great idea and very cool design.
Thank you, Scott.

The first one is an Objective-C class for accessing the keychain service in Mac OS X: [STNKeychainAccess](/projects/stnkeychainaccess).

In the downloadable zip-file you can find a Xcode project with a small cocoa application which shows how this class can be used in your application.

Have fun!