One of the concepts I have came to know and love is that of a closure.  This was first exposed to me when C# started to incorporate more functional aspects into the language such as anonymous methods as first class citizens and later lambda expressions.

You might think this is quite odd that I specially call out closures & anonymous functions as being a functional concept, as closures are used heavily in dynamic languages due to the fact functions can be passed about as first class citizens. Passing functions about is the very nature of a functional language.

This is what Wikipedia has to say:

In computer science, a closure is a first-class function with free variables that are bound in the lexical environment. Such a function is said to be “closed over” its free variables. A closure is defined within the scope of its free variables, and the extent of those variables is at least as long as the lifetime of the closure itself.

The closure aspect is simple… the anonymous function is “Trapping” the local variable within the functions context.  That means the local variable in question will remain in scope for the duration of the lifetime of the anonymous function. In the examples below I declare anonymous functions called “closure” and I en-close around variable called “x”.

So here is an example of a closure in my language of the month Groovy:

def loopTo(n, closure)
{
	for(int i = 0; i < n; i++)
	{
		if(closure(i))
			println i
	}
}

//Our variable to be "Closed Over"
int x = 2

// I like this syntax of '{ }' to create an anonymous method,
// in Groovy we can use 'it' by default
// to reference an undefined but passed parameter.
loopTo(100, { it % 2 == x })

Now in JavaScript:

function loopTo(n, closure){
	for(i = 0; i < n; i++){
		if(closure(i)){
			alert(i);
		}
	}
}

//Our variable being "Closed over"
var x = 0;
// In JS we use the 'function' keyword and specify the parameter
loopTo(10, function(i){
	return i % 2 == x;
});

Now in C#:

static void Main(string[] args)
{
    //Our variable being "Closed over"
    int x = 0;

    // In C# we can use a lambda expression.
    LoopTo(100, num => num % 2 == x );
}

// As C# is strongly typed we define the expected
// expression type as a 'Func<param, return type>'
public static void LoopTo(int n, Func<int, bool> closure)
{
    for (int i = 0; i < n; i++)
    {
        if(closure(i))
            Console.WriteLine();
    }
}

And now in Scala:

// Scala is sweet we store our range rather than writing a loop,
// then iterate over the list passing the index 'i' into our closure.
def main(args: Array[String]) :Unit = {
    val range = 0.until(100);
    range.foreach(num =>  if(closure(num)) println(num) );
}
//Our variable to be "Closed over"
val x:int = 0
// Our function is defined like so
def closure(num : Int) = {  num %  2 == x;  }

We can copy the Scala way in C#.
We had to cheat a tad and write a funky ‘Until’ C# Extension Method.
This gives us a nice fluent way to create a range of integers stored as a list like so:

public static void Main(string[] args)
{
    //Extension method 'Until' defined later in this post
    var range = 0.Until(100);
    range.ForEach(num => { if(Closure(num)) Console.WriteLine(num); });
}
int x = 0;
// Our function definition in C#
private static readonly Func<int, bool> Closure = num => num % 2 == x;

public static class Extensions
{
    // Called like so: 0.Until(100)
    public static List<int> Until(this int lower, int upper)
    {
        var range = new List<int>();

        for (int i = lower; i <= upper; i++)
        {
            range.Add(i);
        }

        return range;
    }
}

This is what the code would look like without using anonymous functions:

private static void Main(string[] args)
{
    LoopTo(100);
}

public static void LoopTo(int n)
{
    int x = 0;
    for (int i = 0; i < n; i++)
    {
        //Our logic for filtering is tied to the control structure
        if (i % 2 == x)
            Console.WriteLine(i);
    }
}

As you can see the code is not only verbose but breaks a the DRY rule (Don’t Repeat Yourself).  Closures are powerful because they encapsulate what might almost be described as a “Strategy”. You have separated out the control structure from the logic. This is the strategy design pattern.

Closures become powerful when your reach the level of understanding that the encapsulation of logic allows for some really powerful composition of “Strategy”.

I find this really intriguing.  I like languages, and I love the idea creating domain specific languages / fluent APIs which make the definition of “Strategy” trivial to the end user.  By combining closures (via currying) or by logically appending delegates together we can create something truly powerful.

If you didn’t know what anonymous functions & closures were when you started to read this post, then hopefully you do now and maybe you can see why they are worth using.

Just some food for thought.

This post will be complemented with a comparative implementation post where I will show the same solution to a problem in C# using Mono and Objective C using Cocoa. (*Thinks to self, I should use C# to write a Mac App with Mono and Cocoa# if I need some UI and the Objective C solution to run on Windows using GNUstep libraries).

C# -  Extreme Growth and Evolution

I do about 90% of my development with C#.  I love that it is such a Swiss Army Knife of a language and that its constantly growing to become an better multipurpose tool. 

The language first came onto the scene in 2001 with it’s 1.0 release. Now in 2010 we are scheduled to see the official release of C# 4.0 which is a completely different animal indeed.  C# 1.0 was very much like Java, even with version 2.0 the same could be said, but as version 3.0 came out we started to see something very different from Java indeed.

Check out the Language Specs by ECMA if you care that much =]

• C# 1.0
• C# 2.0
• C# 3.0
• C# 4.0

With Great Power Comes Great Responsibility

The flip side of a language which is gaining more and more features is inconsistent development practises in said language will become an issue. 

C++ is famous for this.  For example, if you have a relatively simple task and got 5 developers of differing experience and ability to each implement the solution with a feature rich language you will get 5 very different solutions.  The junior developer will look at the experts code and it will seem like some crazy voodoo.  [I hope to go into this topic in more detail in another post in the future.]

I would classify myself as a relatively experienced when it comes to modern C#, I am familiar with 99% of the language features and I would generally use the best tool for the job (but I am young and far from perfect it must be noted!). 

But say I was to write a solution using the powerful functional language features such as passing delegates, composing numerous expressions using Expression Trees in a resolution based approach to the problem, or even using a multi tiered LINQ query which used the lambda syntax for delegates… I would not expect a Junior C# developer to be able to effectively grok and maintain such a solution without the overhead of having to learn and master these aspects of the language (and in many cases the underlying framework).

Of course the answer is to define coding policies (possibly enforcing them using a tool like FXCop), but this will enviably be broken and restrictive (as in my humble opinion this always is even if its not at the point of conception).  The best or most efficient or most elegant tool (in this case language feature) for the job may be a company policy NO NO!

Objective C

As I learn Objective C, I see a language which..

“was created … in the early 1980s” 

http://en.wikipedia.org/wiki/Objective-C

and only reached version 2 in 2006.

“At the 2006 Worldwide Developers Conference, Apple announced the forthcoming release of "Objective-C 2.0," a revision of the Objective-C language to include "modern garbage collection, …” http://en.wikipedia.org/wiki/Objective-C#Objective-C_2.0

I also see design patterns and conventions as a first class citizen in the Objective C and Cocoa world.

What is my point? ..consistency by being so lean.

Teams developing for the Mac & iPhone are aided by this consistency.  It is very likely that developers for these platforms could move to a different company and become productive on a code base very quickly and effectively as the way things are done (at a high level) will not vary a great deal. 

The MVC pattern is an integral part of how applications are developed with Objective C, Cocoa and Interface Builder.  This helps a great deal.  With C#, .NET and the associated View technologies such as WPF, WinForms, WebForms, GTK etc the design pattern in place ranges from MVC, MVP, MVVM to none what so ever which is too often the case to the expense of my (and many others) mental health.  This means that the consistency is not in place and design patterns (i.e. separation of concerns) are not seen as a fundamental as much as they are seen as an advanced topic and higher learning.  Microsoft have released their MVC framework for web development which is called ASP.NET MVC.  This is a step in the correct direction, not to mention they released it under the MS-PL license which allows the Mono Team to take advantage of it as part of the Mono core libraries.

Objective C as a language is what it is, a C variant which introduces object orientation and the notion message passing to call methods on instances using a “infix” notation as opposed to “postfix” which most people are used to.

Example: Calling a method in ObjC, C++ & C#

• [obj method: parameter]; //Objective C – Infix
• obj->method(parameter); //C++ – postfix
• obj.Method(parameter); //C# – postfix

As the language is low level enough, even if the solution is quite complex relative to the equivalent C# implementation there is very little that can’t be done.

In my discussion of C# I tried to talk purely about the language, not the underlying framework be it .NET or Mono. With Objective C much of its power comes from the Cocoa libraries, the same can be said about C# and its frameworks I guess, but C# as a language is far more feature rich.

I am really exited about learning Objective C, its dynamic Small Talk-esk nature intrigues me.  It is lean in language features relative to C# and that I believe is its biggest advantage.

Generalised Comparisons of C# & Objective C

C# is powerful and can be very elegant and highly productive (in the correct hands).  The language features such as LINQ, lambdas, anonymous classes, expression composition, statically compiled dynamic madness, covariance & contravariance… are something special, but the languages many advanced features may baffle and confuse those who are not intimate with it.

The development stacks which C# lives in lack structure out of the box it could be said, this can lead to some nasty un-maintainable evil without the correct guidance.  When done properly C# very plays well with tooling allowing for highly accelerated development and refactoring with ease and efficiency.  Objective C (which I am no expert in) seems to exist in a world of structure, best practises and conventions without taking it to the extreme as Rails does with Ruby (i.e. without a 500 page book on conventions).  The language is light when it comes to features but by its nature it is by no means weak. 

C# lives on a higher level, it is about 4 or 5 generations away from C whereas Objective C is 1 generation about C. 

There low level control fits well with the Apple ethos.  The proprietary nature of Apples hardware and the low level nature of Objective C means you can interact with the hardware components, with a large degree of control and in a consistent fashion without worrying if the hardware will work with your app.  So Objective C is a better fit with the Mac in that regard.

In comparison C# and Java live at a higher abstraction.  They are designed to work with a massive range of machines with potentially infinite hardware configurations so the fine grained control is sacrificed for portability. That is not to say you can’t call into C code from C# or Java when the need is there.

//Todo: This post is in need of Refactoring, but all the tests are passing =] (i.e. I think I said all I wanted to)!

This is due to the fact that when at least one value type is used in a closed generic definition, the CLR when JIT-compiling a generic definition (either a method or a class) will create a separate “machine code page” for each value type definition e.g.

List<int> intList = new List<int>()
List<RandomStruct> structList = new List<RandomStruct>()

Generic types that will be used with multiple different reference types do not affect the memory footprint, as a shared machine code page will be used e.g.

List<string> stringList = new List<string>()
List<RandomClass> classList = new List<RandomClass>()

As the C# compiler will enforce type safety at compile time, the JIT compiler can produce a more optimized version of the machine code by assuming that the types are correct.

The reality is that Generics with value types provide many benefits that out weight the memory cost. These include the compile time type safety, the more concise code (no need to parse / cast).

When it comes to working in environments where every single little drop of memory is important such as mobile platforms, this little bit of knowledge may prove valuable, or indeed it may not, but ill leave that up to you to decide.

• Download GNUstep System & Core from: http://www.gnustep.org/experience/Windows.html
• Use the ..\GNUstep\mingw\bin\gcc.exe compiler
• Set this to an Environment Variable GCC
• Run => gcc -o hello hello.m
• out pops hello.exe

XXL Bean Bag

Lap Desk

Blue Lava Lamp

Blue Plasma Lamp

…I welcome all suggestions! Kthx =]

The first of which is “The Art of Unit Testing (with examples in ‘C#’ .NET)” by Roy Osherove, the only book on this list I have finished reading. This book I found to be essential reading, and I don’t say that often. I rate this book so highly I recommended that it become a required text for the “Agile and Component based Development” Module at my old university.  If you are looking to improve your unit testing, or if you are just starting out with unit testing this book is for you.  The author purposefully avoids the topic Test Driven Development (TDD) in order to achieve clarity and focus on the actually unit testing of code without the confusion which can be caused by trying to do so whilst at the same time explaining the TDD workflow.

C# is the love of my life.. two more books on the language from two highly regarded individuals is exactly what the doctor ordered.

(The MEAP for the second edition (C# 4) is available here)

2010 for me is going to be the year of C# 4, Objective C, Cocoa and Ruby.

When I was a Microsoft Student Partner I got it in the neck from my friends (especially the Linux Zealot housemate) about my favourite language and framework not being portable nor open source.  This really pushed by button (as they well knew) considering I am passionate Open Source, Cross Platform development and learning new languages D= !

I am a massive fan of the Mono Project.  I love the idea of C# being truly cross platform.  I also like the fact that many of the .NET libraries are being ported to run on Linux and OSX.  Over the last year I have been working hard to get all my apps that I write in C# to work as well on Linux (where possible).  But there was one platform I did not work with.. the ever more popular OSX.

I find that Apple have an interesting platform (although I may not agree on principle with their ethos).  But this doesn’t mean that I don’t wish to expand my skill set to be a competent developer on their platform.

I purchased a Macbook this Christmas, against my better judgement, and have purchased these two books:

Why learn Objective C when I could just use C# with Mono?

Because I, as a hacker at heart, am not satisfied with some abstraction over what is going on… I want to be a strong Objective C developer with a sound knowledge of Cocoa and how things work in the Apple world.  I find the difference from Java and .NET refreshing and I love learning stuff that is completely new.  I want to be able to more effectively use and contribute back into open source projects like CocoaSharp and MacRuby / HotCocoa.

I like how its low level and not “as easy”… I have recently graduated and I am still hungry so lets hope this new educational expedition lives up to my expectations.

“I think that it’s extraordinarily important that we in computer science keep fun in computing. When it started out, it was an awful lot of fun. Of course, the paying customers got shafted every now and then, and after a while we began to take their complaints seriously. We began to feel as if we really were responsible for the successful, error-free perfect use of these machines. I don’t think we are. I think we’re responsible for stretching them, setting them off in new directions, and keeping fun in the house. I hope the field of computer science never loses its sense of fun. Above all, I hope we don’t become missionaries. Don’t feel as if you’re Bible salesmen. The world has too many of those already. What you know about computing other people will learn. Don’t feel as if the key to successful computing is only in your hands. What’s in your hands, I think and hope, is intelligence: the ability to see the machine as more than when you were first led up to it, that you can make it more.”

Alan J. Perlis (April 1, 1922-February 7, 1990)

http://is.gd/5EA5f (MIT Press)

I love this quote and I am starting to really love this book.  Although it is a mammoth text I find myself finding lots of brilliant quotes and useful experience which as a junior developer I really cant get enough of.

Another great snippet from the Foreword:

“To appreciate programming as an intellectual activity in its own right you must turn to computer programming; you must read and write computer programs — many of them. It doesn’t matter much what the programs are about or what applications they serve. What does matter is how well they perform and how smoothly they fit with other programs in the creation of still greater programs. The programmer must seek both perfection of part and adequacy of collection”

Alan J. Perlis

http://is.gd/5EAlT (MIT Press)

I even thought the first line was brilliant D= !

“This book is dedicated, in respect and admiration, to the spirit that lives in the computer.”

Alan J. Perlis

http://is.gd/5EA5f (MIT Press)

I knew in PRISM you are free to swop in your own DI container, which is nice but I was wondering how you achieved the trick of having your container available within itself.

What do I mean?  Put simple being able to resolve your DI Container in any class within your application.

Example: Where ‘Foo’ is just a plain old class within the application.

public class Foo : IFoo
{
	public Bar SomeProp { get; set; }
 
	public Foo(IContainer container)
	{
		Bar = container.Resolve<ibar>();		
	}
}

So I asked Glenn Block (one of the main guys behind PRISM at Microsoft) on Twitter how this was achieved and he got back to me with some great material which I thought I would share:

• How to resolve a DI Container in Unity from itself.
• Why this is bad.
• How to do it but limiting the badness.

Note: Remember to read these ‘Blocks’ of tweets backwards as it is Twitter. So with 1 & 3 start at the bottom and read up.

0 – Me:

1 – GBlock:

(START FROM THE BOTTOM & READ UP)

2 – Me:

3 – GBlock:

4 – Me:

So the recommendation is to not program against the container as “It makes parts tightly coupled to host configuration” & “makes code less intentful”.

But as with the deign of modular composition frameworks such as PRISM I found it to be a good fit and it allowed for “reasonably” rapid development of the UI and decoupling of the parts of the View & ViewModel that really mattered.  So I see why it was suggested, but I now also see why they now see it as an anti-pattern.

The universal access to the Container and the Event Aggregator with Composite Events made for very easy composition of a WPF user interface from separate modules, but I would have to agree, now that I have more experience, that programming against the container at this level just felt wrong deep down, and when that is the case you know its just not meant to be.

Please feel free to comment.

I am working in the domain of distributing computing and I am looking into the Microsoft HPC Server platform as a possible option.  One of the most important aspects is the development experience, and to test that I will need to create a suitable development environment.

The environment will consist of a network of virtual machines with the following “Roles”:

• Domain Controller / DNS Host
• HPC Server Head Node.
• ‘N’ number of Child nodes.

So numerous headaches and a ServerFault Question later (Domain Controller / DNS Issue – Creating Network of Virtual Machines for Development), I’ve thrown together this rough guide which should hopefully aid some other poor developer get up and running developing against a Microsoft HPC Cluster.

BEFORE YOU CONTINUE NOTE:

• I am a developer, NOT an IT Professional, and my actions will reflect that.
• This is a network of virtual machines I am using for testing purposes.
• These are running on a SECURE LAN, working on a Private network.
• With little to no security in place.

So in a nutshell if you are trying to install a cluster of machines in an enterprise environment get a professional and DO NOT use this guide.

Creating the Domain Controller & DNS

For this I followed this great guide: Build a Windows Server 2008 R2 Domain Controller.

Configuring machines to join Domain

This is where the first real headache started.  I followed the usual steps to add a machine to the domain, but realised that the machine was point at my router (thinking it was the Domain Name Server (DNS) ).

Step Zero: Add machine to Domain

Error: “Domain Controller for the domain <Your Domain> could not be contacted”

Full Error Message

Note: This information is intended for a network administrator.  If you are not your network’s administrator, notify the administrator that you received this information, which has been recorded in the file C:\Windows\debug\dcdiag.txt.
The following error occurred when DNS was queried for the service location (SRV) resource record used to locate an Active Directory Domain Controller for domain 17B.CO.UK:
The error was: “DNS name does not exist.”
(error code 0×0000232B RCODE_NAME_ERROR)
The query was for the SRV record for _ldap._tcp.dc._msdcs.17B.CO.UK
Common causes of this error include the following:
- The DNS SRV records required to locate a AD DC for the domain are not registered in DNS. These records are registered with a DNS server automatically when a AD DC is added to a domain. They are updated by the AD DC at set intervals. This computer is configured to use DNS servers with the following IP addresses:
192.168.1.1
- One or more of the following zones do not include delegation to its child zone:
17B.CO.UK
CO.UK
UK
. (the root zone)
For information about correcting this problem, click Help.

Resolution: Configure Network Card To Point at DNS Server

Go to: Network Connections > ‘Properties (of your network connection) > IPv4 > Properties > Advanced..

Then: > DNS tab > Add > “Enter the IP of the DNS machine (which is also you Domain Controller).

(Disable & Enable) Your Network Connection. Now you should be able to add the machine to the domain.

You will be prompted to Authenticate

If all is well you should see this screen

Finally Restart Your machine and it will have been successfully added to the Domain

Note: This step will need to be repeated for each machine you wish to add to the domain.

Installing and Configuring HPC Machines

For this I followed this guide  Installing HPC Server 2008 by (Parallel Programming Guru) Daniel Moth.

Conclusion

That was how I got up and running developing on the HPC Platform, hope this was helpful. (Wish I had it when I started).

Get the Rx Framework libs now from here!

• .NET 3.5 SP1
• .NET 4.0
• Silverlight 3.0

Watch the PDC Session

Rx: Reactive Extensions for .NET

Want more? Watch some Channel 9

Rx Video Listing

Some Early Thoughts

This is a game changer!  I can’t wait to get playing with / applying it to my daily .NET life!  I’ll be blogging more on the topic when I have a better grasp of it all and something more interesting to say.

Complex Event Processing

“Complex event processing, or CEP, is primarily an event processing concept that deals with the task of processing multiple events with the goal of identifying the meaningful events within the event cloud.” Wikipedia

I really hope Rx Framework can be applied to complex event processing (CEP)..  *fingers crossed*.   You can see how Rx might relate to “processing multiple events” as its a push model that you can run LINQ over!

Update: Microsoft’s StreamInsight!

StreamInsight takes advantage of the Rx Framework to allow for high performance CEP.

This is how Microsoft describe their CEP offering:

Microsoft StreamInsight is a powerful platform that you can use to develop and deploy complex event processing (CEP) applications. Its high-throughput stream processing architecture and the Microsoft .NET Framework-based development platform enable you to quickly implement robust and highly efficient event processing applications.

By using StreamInsight to develop CEP applications, you can achieve the following tactical and strategic goals for your business:

• Monitor your data from multiple sources for meaningful patterns, trends, exceptions, and opportunities.
  Analyze and correlate data incrementally while the data is in-flight — that is, without first storing it–yielding very low latency. Aggregate seemingly unrelated events from multiple sources and perform highly complex analyses over time.
• Manage your business by performing low-latency analytics on the events and triggering response actions that are defined on your business key performance indicators (KPIs).
  Respond quickly to areas of opportunity or threat by incorporating your KPI definitions into the logic of the CEP application, thereby improving operational efficiency and your ability to respond quickly to business opportunities.
• Mine events for new business KPIs.
• Move toward a predictive business model by mining historical data to continuously refine and improve your KPI definitions.

Nice!  Good times ahead on the .NET platform.