January 2008 - Posts
Charlie Calvert has posted an article that is co-authored by Mads Torgerson on one possible implementation of Dynamic method invocation in a future version of C#: http://blogs.msdn.com/charlie/archive/2008/01/25/future-focus.aspx
There are already a number of comments, and responses from Charlie and Mads. I need to think about it a bit more, and then I'll add some thoughts as well.
If you're like me, you understand new language features better when you see what the new language features generate for you.
Closures in C# are no different. There's quite a bit that goes on under the covers in a C# closure. Looking at all the code that the C# 3.0 compiler generates can really help you understand what's happening. A little help from Reflector and we can learn a lot.
I started with this rather simple C# 3.0 program:
class Program
{
static void Main(string[] args)
{
int counter = 0;
IEnumerable<int> values = Utilities.Generate(20, () => counter++);
Console.WriteLine("Current Counter: {0}", counter);
foreach(int num in values)
Console.WriteLine(num);
Console.WriteLine("Current Counter: {0}", counter);
foreach (int num in values)
Console.WriteLine(num);
Console.WriteLine("Current Counter: {0}", counter);
}
}
public static class Utilities
{
public static IEnumerable<T> Generate<T>(int num, Func<T> generator)
{
int index = 0;
while (index++ < num)
yield return generator();
}
}
The output for this program is pretty simple, but shows us a few things about closures and deferred execution:
Current Counter: 0
0
1
2
...
17
18
19
Current Counter: 20
20
21
...
38
39
Current Counter: 40
There are a few points to see here. First, notice that the value of counter is 0 after defining the sequence. That's because enumerations use deferred execution. The enumeration does not happen until some calling code wants to examine the enumeration. You can see this by looking at the value of counter after the first and second enumerations. Notice that counter has a value of 20 after enumerating the sequence once. Then, you see that counter has a value of 40 after enumerating the sequence again. Also, notice that the sequence returned changes each time you enumerate the sequence. Charlie Calvert covers this concept very well here.
Now, let's look at how it works inside. A Closure is a data structure that holds an expression and an environment containing the variable bindings necessary to evaluate the expression. OK, that's a mouthful. Sometimes, it's easier to understand in code. So let's power up Reflector and see what the compiler wrote. Reflector has a great option where you can specify what version of .NET Reflector should disassemble into. For this post, I chose to have Reflector generate .NET 1.1 code. Now, that caused quite an increase in the volume of the code, its readability, and in fact, the C# compiler won't even compile the disassembled code (more on that in a minute). But, it does show exactly what C# is doing for you with all these new features. Therefore, for the rest of this article, I took the disassembled code, and reworked it so that it was valid C#. Strange compiler generated variable names have been replaced with legal names. Constructs that won't compile have been rewritten.
I'll leap to the conclusion right now: In most cases, C# 3.0 creates classes to handle the state for closures, continuations (enumerator methods), and other new C# 3.0 features. There's no magic, just a lot of generated code.
Creating an Enumerator class
Let's begin with the Generate method. Generate creates the sequence, and it does so using the yield return contextual keyword. Yield Return creates a nested enumerator class to generate the sequence, and it handles all the work to create and use the sequence:
public static class Utilities
{
// Methods
public static IEnumerable<T> Generate<T>(int num, Func<T> generator)
{
GenerateEnumerator<T> d__ = new GenerateEnumerator<T>(-2);
d__.currentNumber = num;
d__.generatorFunc = generator;
return d__;
}
// Nested Types
private sealed class GenerateEnumerator<T> :
IEnumerable<T>, IEnumerable, IEnumerator<T>, IEnumerator, IDisposable
{
// Fields
private int state;
private T current;
public Func<T> generatorFunc;
public int currentNumber;
private int initialThreadId;
public int index;
public Func<T> generator;
public int num;
// Methods
public GenerateEnumerator(int initialState)
{
this.state = initialState;
this.initialThreadId = Thread.CurrentThread.ManagedThreadId;
}
public bool MoveNext()
{
switch (this.state)
{
case 0:
this.state = -1;
this.index = 0;
while (this.index++ < this.num)
{
this.current = this.generator();
this.state = 1;
return true;
}
break;
case 1:
while (this.index++ < this.num)
{
this.current = this.generator();
this.state = 1;
return true;
}
break;
}
return false;
}
IEnumerator<T> IEnumerable<T>.GetEnumerator()
{
Utilities.GenerateEnumerator<T> d__;
if ((Thread.CurrentThread.ManagedThreadId ==
this.initialThreadId) && (this.state == -2))
{
this.state = 0;
d__ = (Utilities.GenerateEnumerator<T>)this;
}
else
d__ = new Utilities.GenerateEnumerator<T>(0);
d__.num = this.currentNumber;
d__.generator = this.generatorFunc;
return d__;
}
IEnumerator IEnumerable.GetEnumerator()
{
return ((IEnumerable<T>)this).GetEnumerator();
}
void IEnumerator.Reset()
{
throw new NotSupportedException();
}
void IDisposable.Dispose()
{
}
T IEnumerator<T>.Current
{
get { return this.current; }
}
object IEnumerator.Current
{
get { return this.current; }
}
}
}
All the interesting additions are in the GenerateEnumerator class. You can see that this class contains an implementation if IEnumerator<T> and IEnumerator. It handles the state relating to the current location in the list. It creates the nested class whenever you call generate. All the IEnumerator methods are handled by the nested class. It's quite a bit of typing, but it's nothing new or magical.
Closures are also classes and objects
The same technique is used for the closure that surrounds the generate method in the main method of the sample:
class Program
{
private sealed class GeneratedClosure
{
// Fields
public int counter;
// Methods
public int GeneratedMethod1()
{
return this.counter++;
}
}
private static void Main(string[] args)
{
GeneratedClosure closureObject = new GeneratedClosure();
closureObject.counter = 0;
IEnumerable<int> values = Utilities.Generate<int>(20,
new Func<int>(closureObject.GeneratedMethod1));
Console.WriteLine("Current Counter: {0}", closureObject.counter);
using (IEnumerator<int> generatedEnumerator = values.GetEnumerator())
{
while (generatedEnumerator.MoveNext())
{
int num = generatedEnumerator.Current;
Console.WriteLine(num);
}
}
Console.WriteLine("Current Counter: {0}", closureObject.counter);
foreach (int num in values)
{
Console.WriteLine(num);
}
Console.WriteLine("Current Counter: {0}", closureObject.counter);
}
}
The compiler created the GeneratedClosure class to contain the bound variables in the environment that the expression needs. This is a simple environment that only needs one field, the counter, so it's a simple class. Note that the field is public, and the closure type contains the method that will be bound to the delegate (GeneratedMethod1). GeneratedClosure implements the environment and the bound variables. All execution of the expressions in the closure take place in the context of this nested class.
You can see what I mean by looking at what the C# 1.1 equivalent of Main() looks like. Instead of a simple int local variable, the compiler creates an instance of GeneratedClosure. Then, the compiler initializes the bound variables (closureObject.counter).
The lambda expression has been replaced by a delegate bound to the instance method closureObject.GeneratedMethod1. That ensures that the delegate is evaluated in the context of the closure environment.
There are a few extra bits of C# behavior to see here. Even though Main enumerates the sequence more than once, the compiler creates only one instance of the closure. That environment is reused each time. That's how counter ends up being 40, rather than 20. The second sequence contains the numbers 20-39 for the same reason. Notice that in both cases, Main examines the bound variable inside the closure. That's how changes in the closure environment are visible (and modifyable) from the outer scope.
Finally, I don't know why the first foreach loop is completely different than the second. If anyone knows, I'd be interested.
I hope this little side trip inside closures has been useful. The bottom line (at least to me), is this: The compiler creates an environment using lots of familiar constructs. While it helps to look inside the code to see what's going on, that's only necessary to help understand the new features. It helps to remove the mystery by peeling under the covers. In most daily work, it's better to use the new syntax, let the compiler do the work, and get stuff done. But now, the next time someone mentions "closures" and debates whether or not they are useful, now you know what a closure is, why it's a useful construct, and how the C# compiler puts one together for you.
I'm really looking forward to this.
We're hosting a 4 day C# Jam April 8 -11, at our office.
There's a lot that I like in the format: I'm co-instructing with Bruce Eckel and Jamie King, two of the smartest, insightful people in our industry. The attendance is limited to 30 people. (That means a 10:1 instructor to student ratio, at the worst). It's all building software, not eyes-forward presentation style.
It also closely follows the Detroit Launch of VS 2008, so you should already have VS 2008 installed.
Bruce Eckel is hosting the signup page here: http://www.mindviewinc.com/Conferences/CSharp3Jam/Index.php
The earlybird deadline is fast approaching, so sign up now.
Oh happy day.
I'm stepping into the .NET Framework source using the VS 2008 debugger.
Shawn Burke has the most detailed post on how to get your VS 2008 enviornment setup.
Chris Woodruf did quite a few interviews during CodeMash last week. They've started to go live. You can see the list of the latest podcasts here
He's got interview with Sara Ford, Dustin Campbell, Michael Rozlog, Keith Elder, Me, Brian Goetz, Jeff McWherter and Josh Holmes live already.
It's a good taste of the kind of content we had at CodeMash. I can't wait for next year.
Carl Franklin and I got together and did an intro to C# for VB.NET developers last month.
The first version is up here: http://www.dnrtv.com/
Or, watch the silverlight version here: http://perseus.franklins.net/dnrtv/0096/silverlight/video.html
This discussion is somewhat introductory, it should be approachable for a beginning C# developer. Carl is a great interviewer. Those who know me know that I'm not particularly fluent in VB.NET. Carl does a great job of leading me toward topics that will make this discussion clear for the VB.NET developer who is learning C#.
The advantage of writing my recap of CodeMash a little later is that I can start by pointing at one everyone else has written. To begin, Jim Holmes shows the strain of organizing CodeMash. After a break, he discusses what happened at CodeMash.
As is usual, Scott Hanselman chooses just the right phrases to describe CodeMash. He gave a fantastic keynote, but what was more impressive was that he spent the entire conference just meeting with people and discussing everything with everyone. He even interviewed Dustin Campbell on F# for Hanselminutes. Dustin gave a great talk on F#. I didn't attend his talk, but I did get a preview of it in the speaker room before hand. He does a great job of presenting a new concept, and getting everyone to feel comfortable.
Sara Ford came from Redmond, and had the guts to show up wearing an LSU sweatshirt. She did open spaces talks on Visual Studio tip of the day and CodePlex. Sara is also one of those people that put a very human face on a very large company. If you spend any time with her, you can tell she enjoys what she builds, and has blast every day she creates something new.
Dave Donaldson was only there one day, talking about RESTful Web Services. Of course, I had to get him to show his OSU tattoo to Sara.
I really like Jay's recap. He discusses what he learned, how the communities built around different technologies differ, and what we can learn from each other. I missed his Castle Project talk, but I hope to hear more about the project during lightning talks at our office.
Patrick Steele wrote about his experience teaching about Monorail, and using it. (You'll not a thread here that Jay discussed the Castle Project, of which Monorail is one component.)
Well, what about my experience? Well, I had two talks. One was on building an IQueryProvider implementation. That didn't turn out as well as I'd hoped. The more I did research on that topic, the more it was clear that there is just too much infrastructure to build in order to make a worthwhile implementation. That always makes a talk difficult. You can't build it there; you can only discuss code you pre-built. The danger in that is that you give people the impression that something is easier than it is. I hoped to avoid that without completely scaring them away from building an IQueryProvider, if it's the right thing to do. (It often isn't, but sometimes it can be worthwhile.) That's a blog post in itself, so I'll leave that for another day.
My second talk, "Real World C# 3.0" was much more fun. I was the first morning session after the keynote. I showed up with a deck containing 10 slides (including the title, agenda, about me, and questions). This was a 70 minute conversation where I spent the whole time building code and discussing the advantages and disadvantages of different solutions you can build for different problems. I hope everyone there enjoyed that style of talk. I got lots of great questions, so it was clear that most of the room was thinking and engaged.
The rest of the time, I was trying to learn new things. I know more about Python than I did last Wednesday (but I'm by no means proficient with it). I know a bit more about Silverlight.
My favorite thing about CodeMash is seeing first hand just how much technical brainpower we have in this region, in many different technologies. I learn a lot by talking to other folks in our area. I learn what they are working on, what the challenges are, and how they are trying to overcome those challenges. Regardless of the tools they are using, we are trying to solve similar problems, and we can learn from each other.
I'm already looking forward to next year.
This has been on the net in a few forms, but On10 just posted a high quality version of the Bill Gates' last day video from CES:
http://on10.net/blogs/larry/Bills-Last-Day-CES-2008-Keynote-Video/Default.aspx
I'm amazed at the number of people he got to put in some cameos
I keep finding more uses for extension methods. This time it's extending IComparable<T>. The API signature for IComparable<T> has been around since the dawn of C: if the left is less than the right, return something less than 0, if left is greater than right, return something greater than 0, and if they are equivalent, return 0. Well, its readability leaves a bit to be desired.
So I wrote this set of extension methods for any type T that implements IComparable<T>:
public static class EquatableExtensions
{
public static bool GreaterThan<T>(this T left, T right) where T : IComparable<T>
{
return left.CompareTo(right) > 0;
}
public static bool LessThan<T>(this T left, T right) where T : IComparable<T>
{
return left.CompareTo(right) < 0;
}
public static bool GreaterThanOrEqual<T>(this T left, T right) where T : IComparable<T>
{
return left.CompareTo(right) >= 0;
}
public static bool LessThanOrEqual<T>(this T left, T right) where T : IComparable<T>
{
return left.CompareTo(right) <= 0;
}
public static bool Equal<T>(this T left, T right) where T : IComparable<T>
{
return left.CompareTo(right) == 0;
}
}
These methods mean I can write this:
if (foo.GreaterThan(bar))
instead of
if (foo.CompareTo(bar) > 0)
personally, I find the former much easier to read.
Ok, this is not rocket science, but it's useful, and it improves the readability of quite a bit of the code I work with on a day to day basis.
As I finish my talk preparation for CodeMash, I'm putting together my schedule.
You can view the silverlight page here: http://codemash.org/sessionscheduler/?1=2&2=28&3=4&4=7&5=3&7=42&8=14
However, that doesn't let me annotate the agenda, and discuss some of the other items at CodeMash. So, here's my take, including some alternative recommendations:
Wednesday evening: 7:00 Experts Panel: Talking Technology to humans. I've been trying to find the answers to this for years. If you're in technology, and your actually interested in making money occasionally, or even just convincing the business folks why your latest technology idea has merit, you need to attend. A number of technology leaders that actually get suits to listen to them will give you ideas on how to get your technology initiatives understood and adopted by your company.
Thursday 9:30: I'm going to be in the Experts Zone during most of the first session, but if I weren't, I'd be at the Coding in Silverlight talk.
11:00: This is a tough one. I want to learn Python, so I'm going to Catherine Devlin's "Crash, Smash, Kaboom Course in Python." Otherwise, I'd be in Jay Wren's Castle talk.
3:30: I'm going to be at my LinqTo<T>: Implementing QueryProvider talk. I don't have much choice here. If I weren't giving a talk now, I'd be going to Dianne's "Introduction to Scala" talk.
4:45: I'm going to Dustin's talk on "Putting the Fun into Functional with F#". Dustin is a great speaker and the topic is pretty cool.
Friday gets really interesting:
9:30: I'm giving my "Real World C# 3.0" talk here. It's a talk about how all the .NET 3.0 and 3.5 code work together to increase your productivity. However, this is a stacked time. I want to see Bruce Eckel's "Why I Love Python" talk. I also wanted to see Dick Wall's "Introduction to Android" talk.
Later in the morning, I'm taking a break.
At 1:30, Patrick Steele is doing a vendor session on "Introduction to MonoRail" that will be interesting. He's showing how he did a non-profit project and MonoRails was the quickest path.
2:30 is the Advanced Silverlight session. This is one of my goals this year: To learn move about Silverlight. If I weren't interested in Silverlight, I'd be doing the Continuous Integration talk.
The last session includes Jim Holmes' "Story Driven Testing". I very intrigued by that topic. It promises to show the way to build test cases that actually include how to create and use tests to validate customer (or user) expectation.
I'll be missing part of that session to head over to the experts zone for more Q & A.
CodeMash has become an embarrasement of riches. There are already too many great sessions, and no matter what I do, I will be missing something interesting.
Yup, it's that time of year. My MVP award was renewed, as was my RD status.
Both are prestigious awards that represent contributions to the technical community around various products.
In addition, Patrick Steele, one of our senior consultants re-joins the MVP ranks. Totally cool.