Author Archive

29
Jan
2013

Thread-based or Event-based?

by Stuart Harris

http://www.flickr.com/photos/jdhancock/4617759902/

Q: What do our 3 favorite open source projects (node, redis and nginx) have in common?  Apart from being uber-cool?

A: They are all single threaded.  

But aren’t they all really fast and highly scalable?  Yep.  So how does that work?

Nginx, redis and node are all event-based.  They have an event loop that will listen for an event saying that an asynchronous operation (IO) has completed and then execute the callback that was registered when the async operation started.  Rinse, then repeat.  It never waits for anything, which means that the single thread can go hell-for-leather just running code.  Which makes it really fast.

In days gone by, when we were Microsoft slaves, we used to wrestle with multithreading as a way of dividing up work.  In web apps every request started a new thread.  We’d also use the Task Parallel Library (TPL) which was not an easy abstraction.  And combine that with some event processing library like Reactive Extensions (Rx).  Now you’re asking for a lot of trouble.  The new await keyword in C# helps out alot, but either way you have to think about thread safety all the time.  And all kinds of locking strategies to deal with concurrent access to the same data.  And even with all that, it isn’t as fast.

The difference between the two worlds lies in the way that pieces of work are orchestrated.

Event-based applications divide work up using callbacks, an event loop and a queue.  The unit of work, or task, is a callback.  Simple.  Only one callback is ever executing at a time.  There are no locking issues.  You can write code like you’re the only kid on the block.  You decide when you’re done and then effectively yield control to someone else.  Everyone is really polite so it just works.

Thread-based applications essentially divide work up in hardware.  Because each piece of work has its own thread, and will block if it needs to (like when it’s waiting for IO), the CPU will suspend that thread and start running another that is waiting.  Every time that happens there is quite a hefty context  switch, including moving about 2MB of data around.  In effect the hardware decides when to yield control and you don’t get much of a say.

Who’d have thought that a single thread, dealing with everything, could be faster than multiple threads each dealing with just one thing?  Well, on a single core, that may be true.  On multiple cores it actually may also be true.  That’s because you’ve probably got nginx and node and redis all running on the same machine – simplistically, on a quad core, that’s one core each and still one left over :-)

But isn’t writing synchronous code for a multithreaded environment a lot easier than writing asynchronous code for a single threaded environment?  Well, maybe, a little.  But some great patterns have emerged within the node community that really help.  

The simplest continuation-passing style (CPS) is the callback.  Which actually is not at all hard when you get used to it.  And it happens to be a great way to encapsulate and really easy to modularise.  The pattern for async functions is that the last argument is always the callback, and the pattern for callbacks is that errors are always the first argument (with results after that).  This standardisation makes composition really easy.

There are a ton of npm modules that can often help reduce complexity.  The best, in my opinion, is still Caolan’s async.  It’s still the most popular and follows the node conventions.  And there are also a few CPS compilers that allow you to code in a more synchronous style.  I wouldn’t have recommended these in the past, but there are a few, such as tamejs and Iced CoffeeScript, that use an “await, defer” pattern that is quite nice.  We’re using CoffeeScript more and more these days, and this “icing” is very tempting (seeing as we’re compiling anyway), but we haven’t strayed that way yet.

We’ve been writing big apps in node since October 2011 and have learnt a lot about how to separate concerns and modularise our code.  It’s a lot different to the object-oriented class-based separation we were used to, but after your head is reprogrammed to use a functional style it becomes second nature and actually much easier to structure.  Caolan’s post on programming style for node sums it up nicely.  If you hear anyone say that node is no good for big projects, tell them that all you have to do is follow a few simple rules and then it becomes perfect.  And fast.

3
Dec
2010

Watching Silverlight Dependency Properties with Reactive Extensions

by Stuart Harris

At yesterday’s Sliverlight Firestarter, Scott announced that Silverlight 5 will have a DataContextChanged event.  Great news!  How often have you wanted to grab the view model in the code behind and been frustrated by the fact that in Silverlight (unlike WPF) you can’t be easily notified when you have a new view model – you can’t even add your own handler by overriding the DataContext property’s metadata.

So we will wait patiently for Silverlight 5 (and all its other goodness – I was very excited by the 3D demo).  But in the meantime we could use something like Prism’s DependencyPropertyListener or BindingListener.  The former uses the traditional approach of using an attached property (with your own handler) to bind to the dependency property in question.  You have to create a binding, attach and detach it in the right place and hook onto (and off) the listener’s Changed event.  Quite a bit of plumbing.  Prism’s BindingListener sits on top of this and allows you to supply a handler and deal directly with elements and bindings.  All very useful.

But when I want the DataContext, or I just want to know when a dependency property’s value has changed, I really don’t want to dig out a pattern from the recesses of my mind and write a ton of code (which is likely wrong first time) just to be notified when something changes.  It’s gotta be one line of code!  Like this:

this.GetChanges(new Binding()).Subscribe(this.DataContextChanged);

Or if I want to know when the Width of a Canvas changes:

this.GetChanges(new Binding("Width") { Source = this.canvas })    
    .Subscribe(args => Debug.WriteLine(args.NewValue));

Obviously I can create any binding I want and pass it to GetChanges(), which is a simple extension method on FrameworkElement that returns an IObservable of ValueChangedEventArgs.  I can subscribe to this with a handler that will be called each time the binding’s value changes.  The extension method looks like this:

public static IObservable<ValueChangedEventArgs> GetChanges(
    this FrameworkElement element, Binding binding)
{
    return Observable.CreateWithDisposable<ValueChangedEventArgs>(
        observer =>
            {
                var listener = new DependencyPropertyListener();
                return Observable.FromEvent<ValueChangedEventArgs>(
                    handler =>
                        {
                            listener.ValueChanged += handler;
                            listener.Attach(element, binding);
                        }, 
                    handler =>
                        {
                            listener.ValueChanged -= handler;
                            listener.Detach();
                        }).Select(@event => @event.EventArgs).Subscribe(observer);
            });
}

So this is the DependencyPropertyListener that was adopted from Prism:

public class DependencyPropertyListener
{
    private readonly DependencyProperty dependencyProperty;

    private FrameworkElement targetElement;

    public DependencyPropertyListener()
    {
        this.dependencyProperty = DependencyProperty.RegisterAttached(
            Guid.NewGuid().ToString(), 
            typeof(object), 
            typeof(DependencyPropertyListener), 
            new PropertyMetadata(null, this.OnValueChanged));
    }

    public event EventHandler<ValueChangedEventArgs> ValueChanged;

    public void Attach(FrameworkElement target, Binding binding)
    {
        if (target == null)
        {
            throw new ArgumentNullException("target");
        }

        if (binding == null)
        {
            throw new ArgumentNullException("binding");
        }

        this.Detach();
        this.targetElement = target;
        this.targetElement.SetBinding(this.dependencyProperty, binding);
    }

    public void Detach()
    {
        if (this.targetElement != null)
        {
            this.targetElement.ClearValue(this.dependencyProperty);
            this.targetElement = null;
        }
    }

    private void OnValueChanged(DependencyObject sender, DependencyPropertyChangedEventArgs e)
    {
        EventHandler<ValueChangedEventArgs> handler = this.ValueChanged;
        if (handler != null)
        {
            handler(this, new ValueChangedEventArgs(e.OldValue, e.NewValue));
        }
    }
}

Presumably the System.Windows.DependencyPropertyChangedEventArgs is not derived from EventArgs because it’s a struct for performance reasons.  It’s up to you whether you use that or create your own EventArgs like this:

public class ValueChangedEventArgs : EventArgs
{
    private readonly object newValue;

    private readonly object oldValue;

    public ValueChangedEventArgs(object oldValue, object newValue)
    {
        this.oldValue = oldValue;
        this.newValue = newValue;
    }

    public object NewValue
    {
        get
        {
            return this.newValue;
        }
    }

    public object OldValue
    {
        get
        {
            return this.oldValue;
        }
    }
}

Binding to the Windows Phone 7 Application Bar

The Application Bar in Windows Phone 7 (and its buttons and menu items) are not derived from DependencyObject and so can’t easily participate in data binding.  Using Silverlight 3 behaviours, we can attach functionality to them, however, including bindings for ICommands (with parameters), Text and Icons.

<phone:PhoneApplicationPage.ApplicationBar>
    <shell:ApplicationBar IsVisible="True"
                          IsMenuEnabled="True">
        <shell:ApplicationBarIconButton IconUri="/Images/appbar.add.rest.png"
                                        Text="Add" />
        <shell:ApplicationBarIconButton IconUri="/Images/appbar.delete.rest.png"
                                        Text="Delete" />
        <shell:ApplicationBar.MenuItems>
            <shell:ApplicationBarMenuItem Text="MenuItem 1" />
            <shell:ApplicationBarMenuItem Text="MenuItem 2" />
        </shell:ApplicationBar.MenuItems>
    </shell:ApplicationBar>
</phone:PhoneApplicationPage.ApplicationBar>

<i:Interaction.Behaviors>
    <Behaviors:ApplicationBarIconButtonCommand TextKey="Add"
                                               CommandBinding="{Binding Path=AddCommand}"
                                               TextBinding="{Binding Source={StaticResource LocalizedStrings}, Path=LocalizedResources.Add}"
                                               IconBinding="{Binding Path=AddIcon}" />
    <Behaviors:ApplicationBarIconButtonCommand TextKey="Delete"
                                               CommandBinding="{Binding Path=DeleteCommand}"
                                               TextBinding="{Binding Source={StaticResource LocalizedStrings}, Path=LocalizedResources.Delete}" />
</i:Interaction.Behaviors>

I’ve attached a sample project that demonstrates binding to the Application Bar buttons.  There are behaviours for binding Prism’s DelegateCommand to the app bar buttons (and menu items) and to regular buttons and other FrameworkElements.  I haven’t included the full Prism source (just the bits needed to run the sample).  Also there is no IoC container so the MainViewModel is instantiated in App.xaml for this.

Enjoy!

Source Code (Visual Studio 2010 sample solution, built against Build 2787 of Reactive Extensions for WP7):

AppBarBinding.zip (44.42 kb)

13
Sep
2010

Inside XPF: Data Binding

by Stuart Harris

Updated 2010-09-20: Reflects the removal of the generic type parameter TOwner from ReactiveProperty (starting Build 20).

Neuroanatomical Connectivity Macaque by Arenamontanus In the previous post in this series, we discussed XPF’s Reactive Properties – the counterpart to Silverlight’s Dependency Properties.  Reactive Properties allow you to data bind to any source that implements the new .Net 4.0 IObservable<T> (for one-way binding) and/or any source that implements IObserver<T> (for one-way-to-source binding).  Two-way binding is simply a combination of the two.

The Reactive Object

In Silverlight, all UI Elements are derived from DependencyObject which the MSDN documentation says “Represents an object that participates in the dependency property system”.  In XPF all UI Elements are derived from ReactiveObject which represents an object that participates in the reactive property system.  ReactiveObject not only stores the current values of all the element’s reactive properties, but also allows you to bind to them in either (or both) directions.

The Bind method has overloads that allow you to bind a ReactiveProperty one-way to an IObservable, one-way-to-source to an IObserver or two-way to both.

The Binding Factory

So what do you do if the object you want to bind to doesn’t expose an IObservable/IObserver?  This is where the BindingFactory comes in.  The BindingFactory will create IObservables and IObservers around properties on either the data context or a specified source.

Binding to reactive properties, or properties on POCOs and objects that implement INotifyPropertyChanged is fully supported.  This enables element-to-element binding in addition to the use of familiar binding patterns often used in WPF/Silverlight such as Model View ViewModel (MVVM).

The observable/observer that the BindingFactory creates for you is internally wrapped in a Binding object which allows for either immediate or deferred resolution of the binding.  Immediate resolution is used when a reference to the source is provided, but if you are binding to an element’s data context then the binding resolution is deferred until the start of the Measure layout phase.  Deferred bindings are great because they allow the data context to be set or changed after the element binding has been declared.  This is essential for controls that use templates such as the ItemsControl.  The Binding’s resolution is handled seamlessly.

All data binding scenarios are supported.  The following table shows the various combinations of source objects/properties in each mode.  Note that binding to POCO properties works in both directions, but without INotifyPropertyChanged (INPC), one-way binding effectively becomes “one-time”.

Data Context
/Source Object
Reactive
Property
INPC
Property
CLR
Property
One-way Supported Supported Supported Supported
(one-time)
One-way-to-source Not applicable Supported Supported Supported
Two-way Not applicable Supported Supported Supported
(one-time and one-way-to-source)

Show me some code

Here are the overloads to ReactiveObject’s Bind method:

Bind

You’ll notice that because it’s a generic method, the parameters are strongly typed to match the target ReactiveProperty specified.  The scenario in the screenshot above shows binding to a source reactive property on the element’s data context.  The specification for this scenario looks like this:

[Subject(typeof(ReactiveObject), "One Way")]
public class when_there_is_a_one_way_binding_to_a_property_on_the_data_context
{
    private const double ExpectedWidth = 10d;

    private static SourceObject source;

    private static ContentControl target;

    private Establish context = () =>
        {
            target = new ContentControl();

            IObservable<double> fromSource =
                BindingFactory.CreateOneWay<SourceObject, double>(SourceObject.WidthProperty);
            target.Bind(UIElement.WidthProperty, fromSource);

            source = new SourceObject();
            target.DataContext = source;
            target.Measure(Size.Empty);
        };

    private Because of = () => source.Width = ExpectedWidth;

    private It should_update_the_target = () => target.Width.ShouldEqual(ExpectedWidth);
}

Because we’re binding to the control’s data context the binding resolution is deferred until the target is measured.

Using the Binding Factory

Here are some examples of how to create Bindings using the Binding Factory.  Those that involve the data context are all deferred and those where the source is specified are all immediate.  An IDualChannel simply encapsulates both an IObservable and an IObserver so that data can flow in both directions (to and from the source).

Notice that you only need to specify type parameters when there is not enough information in the method’s parameters for the compiler to infer the types involved:

To the Data Context

IObservable<double> fromSource =
    BindingFactory.CreateOneWay<double>();

IObserver<double> toSource =
    BindingFactory.CreateOneWayToSource<double>();

To a Reactive Property on the Data Context

IObservable<double> fromSource =
    BindingFactory.CreateOneWay<SourceObject, double>(SourceObject.WidthProperty);

IObserver<double> toSource =
    BindingFactory.CreateOneWayToSource<SourceObject, double>(SourceObject.WidthProperty);

IDualChannel<double> fromAndToSource =
    BindingFactory.CreateTwoWay<SourceObject, double>(SourceObject.WidthProperty);

To a Reactive Property on a specified Source

IObservable<double> fromSource =
    BindingFactory.CreateOneWay(source, SourceObject.WidthProperty);

IObserver<double> toSource =
    BindingFactory.CreateOneWayToSource(source, SourceObject.WidthProperty);

IDualChannel<double> fromAndToSource =
    BindingFactory.CreateTwoWay(source, SourceObject.WidthProperty);

To an INotifyPropertyChanged or CLR Property on the Data Context

IObservable<double> fromSource =
    BindingFactory.CreateOneWay<SourceObject, double>(o => o.Width);

IObserver<double> toSource =
    BindingFactory.CreateOneWayToSource<SourceObject, double>(o => o.Width);

IDualChannel<double> fromAndToSource =
    BindingFactory.CreateTwoWay<SourceObject, double>(o => o.Width);

To an INotifyPropertyChanged or CLR Property on a specified Source

IObservable<double> fromSource =
    BindingFactory.CreateOneWay(source, o => o.Width);

IObserver<double> toSource =
    BindingFactory.CreateOneWayToSource(source, o => o.Width);

IDualChannel<double> fromAndToSource =
    BindingFactory.CreateTwoWay(source, o => o.Width);

Clearing a Binding

To clear a binding simply call the ClearBinding() method on the relevant control, specifying the reactive property whose binding you want to clear:

element.ClearBinding(Border.BorderBrushProperty);

Because an instance of ReactiveObject can only have one binding per property, calling the Bind() method replaces any existing binding.  Internally it calls ClearBinding() before attaching the new binding.

When a binding is cleared, its subscription to the source is automatically disposed.  This releases any attached event handlers (created automatically in order to listen to PropertyChanged events) which helps prevent memory leaks.  This is a great feature of Microsoft’s Reactive Extensions (Rx) that frees the developer from worrying about having to un-hook event handlers when an observer is no longer interested in the event.

Even though there can only be one binding at a time to an element’s property, you can still merge multiple observables together using any of the Rx combinators (Merge, Zip, CombineLatest etc) if you want your reactive property to listen to more than one source.

Summary

We hope that the binding system in XPF is simple to understand and easy to use.  Whilst the underlying property system in XPF is push based (rather than pull), we’ve strived to keep the development experience familiar to Silverlight and WPF developers.

The tight integration with Reactive Extensions makes it extremely flexible and powerful.  The statically typed nature of the binding and property system makes it intuitive, discoverable and refactoring friendly.  The absence of any magic strings means that there should be no run-time binding exceptions when a deferred binding cannot be resolved.  Binding to nested properties is currently not supported, but we plan to introduce this using the same strongly typed lambda expression syntax shown above.

Please download the latest build, play about with it and feedback to us!

Inside XPF Series:

9
Sep
2010

Inside XPF: Reactive Properties

by Stuart Harris

Updated 2010-09-20: Reflects the removal of the generic type parameter TOwner from ReactiveProperty (starting Build 20).

Tetrahedral reflection by fdecomite

One of the design principles behind XPF has always been to provide a development experience that is familiar to Silverlight developers.  Silverlight’s Dependency Property plays an important role in making that development experience so good.  They enable most of the exciting functionality that both Silverlight and WPF bring to the table.

Pipes and Sockets

XPF introduces Reactive Properties that share some of the behaviour and familiarity of Dependency Properties and add the ability to behave like “sockets for pipes”.  Places where you can plug single- or bi-directional “pipes” that channel data asynchronously between your application’s objects.

Modern UI frameworks are heavily asynchronous so that stuff can happen in the background without making the UI unresponsive.  Silverlight helps enable this by providing asynchronous-only APIs for potentially “long-running” activities.  More and more, UI applications are built using objects that communicate with each other asynchronously.

Active Properties

Whilst regular C# Properties are mostly “passive”, Dependency Properties in Silverlight (and their sisters Attached Properties) are most definitely “active”.  Their “effective” values “depend” on their environment.  Instead of providing access to a single value in a backing field, they will actively seek out the most appropriate value for you at the time you request it.  Is an animation in progress?  Yes? Then that’s the value.  No?  Has the value been explicitly set?  OK, use that.  If not, is there a binding in place?  Is the binding one-way? OK, get the value from the source object.  Is there a relevant style to get the value from?  Can the value be inherited from an ancestor in the visual tree?  Also, if it’s an Attached Property it really belongs to an ancestor in the Visual Tree.

Each time a value is requested a complex workflow is instigated to work out what the value should be.  It’s great, it works really well and it’s behind all the Animation, Data Binding, Styling and Layout in Silverlight and WPF.

Reactive Properties

One of the most exciting things that’s happening in the world of software development at the moment is the evolution of Microsoft’s superb Reactive Extensions for .Net (Rx).  The team behind it discovered a duality between IEnumerable/IEnumerator and IObservable/IObserver which is not only extremely elegant, but also proves to be really useful in enabling pluggable, queryable, composable asynchronous pipes.

The same duality exists between Silverlight’s Dependency Properties and XPF’s Reactive Properties.  The former fetch the right value when asked to do so, the latter are given the right value when the time is right.  Data can be pushed to Reactive Properties through pipes – e.g. through a pipe from a data binding or through a pipe from an animation framework.  Data can also be pushed from the Reactive Property to any number of interested objects through pipes that can be joined, split, queried, filtered and processed in any number of ways.  All thanks to the genius of the guys behind Rx.

Show me some code

You’re probably familiar with the way in which Silverlight’s Dependency Properties are registered statically in a central registry.  XPF’s Reactive Properties are declared in a similar same way.  In the following code snippet, UIElement registers its Height Property with a default value of “Not a Number” and a call-back, which in this case, is just a static generic method that indicates to the property owner (UIElement) that it needs to re-measure when the value changes.  Notice that the generic Register method is typed for the property type (of double) allowing for strongly typed get/set accessors (no casting required):

public static readonly ReactiveProperty<double> HeightProperty =
    ReactiveProperty<double>.Register("Height", typeof(UIElement),
        double.NaN, ReactivePropertyChangedCallbacks.InvalidateMeasure);

public double Height
{
    get
    {
        return this.GetValue(HeightProperty);
    }

    set
    {
        this.SetValue(HeightProperty, value);
    }
}

Attached Properties are declared in exactly the same way – the only difference is that the property accessors (which are also strongly typed) are static:

public static readonly ReactiveProperty<double> LeftProperty = 
    ReactiveProperty<double>.Register("Left", typeof(Canvas),
        double.NaN, ReactivePropertyChangedCallbacks.InvalidateArrange);

public static double GetLeft(IElement element)
{
    if (element == null)
    {
        throw new ArgumentNullException("element");
    }

    return element.GetValue(LeftProperty);
}

public static void SetLeft(IElement element, double value)
{
    if (element == null)
    {
        throw new ArgumentNullException("element");
    }

    element.SetValue(LeftProperty, value);
}

Data Binding with Reactive Properties

The next post in the “Inside XPF” series will go into data binding in a lot more detail, but this excerpt from the BDD specs in XPF’s codebase shows how you can bind a property on a source (that implements INotifyPropertyChanged) to a Reactive Property on an element.  Notice how the binding is simply an IObservable of double.  You can plug anything that implements IObservable<double> into the property directly using the Bind() method.

[Subject(typeof(DependencyObject))]
public class when_a_binding_is_one_way
{
    private const double ExpectedWidth = 10d;

    private static TestBindingObject source;

    private static Border target;

    private Establish context = () =>
        {
            source = new TestBindingObject(); // implements INotifyPropertyChanged
            target = new Border();

            IObservable<double> fromSource = BindingFactory.CreateOneWay(source, o => o.Width);
            target.Bind(UIElement.WidthProperty, fromSource);
        };

    private Because of = () => source.Width = ExpectedWidth;

    private It should_update_the_target_property_with_the_correct_value =
        () => target.Width.ShouldEqual(ExpectedWidth);
}

Summary

Properties in XPF work natively with the new IObservable and IObserver interfaces in .Net 4.0, and their implementations in the Rx Framework.  This really improves the developer experience, because now you can connect up your UI properties to any asynchronous source.  Imagine being able to asynchronously bind your Reactive Properties directly to WebClient’s async Download methods?

Download the latest build and have a play!

Inside XPF Series:

3
Sep
2010

XPF Roadmap and Licensing

by Stuart Harris

You Are Here by joelogon

Throughout XPF’s beta we want to keep everyone in the loop so we thought we would set out a high-level roadmap to share our current plans.

These plans are not set in stone – we’re an agile shop and are always adapting to change as users give feedback about how the product is being used and make feature suggestions.

We really want your feedback!

Features

We’ve yet to decide exactly what will constitute the version 1 release, but it’s fair to say that the vast majority of the planned features listed below will be there.

Done Planned (in order of priority) Future
UIElement (base class)
Panel (base class)
RootElement
Grid (Auto/Pixel)
StackPanel
ContentControl
TextBlock
Image
Border
Button
ItemsControl (with ItemTemplate)
ScrollViewer
Data Binding (one and two-way)

Grid: Star Grid Lengths
Canvas
Grid: Row/Column Spanning
ImageBrush
TextBlock: Default Font
ItemsControl: Control Template
Visibility
Clipping
ScrollViewer: Scroll Bars
Brush: Opacity
ListBox
ProgressBar
RadioButton
Slider
TextBox
VirtualizingStackPanel
CheckBox
Pivot Control
Animation (Windows)
XAML Support
Control Templates
Visual State Manager

Licensing

The finer details of XPF’s licensing model are still being worked on, but we’re really happy to be able to say that XPF will be free to use if you are an individual developing non-commercial or open source applications.

Feedback

Once again, your feedback is really important to us.  If there’s a feature you don’t see listed, or want one feature developed first over another, then let us know.  We’re also really interested to see how XPF is being used, so if you’ve got any examples then please do get in contact – we’re already hearing some great stories.

26
Jun
2010

A simple IRepository implementation for MongoDB and NoRM

by Stuart Harris

Neon Code by maistora David’s at the World Cup in South Africa, Cain’s been to Sonar in Barcelona and I’ve been coding!  Mostly around OpenId and OAuth, Raven DB, MongoDB and NoRM.  I reckon I’ve had just as much fun!

I absolutely love NoRM, it makes working with Mongo (which is a brilliant document database) into a dream.  Andrew Theken and the team are doing a fantastic job with it.  And they are extremely responsive – a small change on my fork was integrated in just a few hours, really bringing home the agility of Open Source Software.

So I needed a simple IRepository<T> implementation to make my application unaware that it was working with Mongo – like getting data from down a tunnel where I don’t care what’s at the other end.  And, of course, I want to make my life even easier.  This is what I came up with.

Unlike with Raven there really isn’t a concept of Unit of Work (or transactions) with Mongo.  Documents are updated atomically, though, which takes away half the situations where you would normally need transactions.  This means that the Repository pattern is easier to implement – its sessions don’t need to represent a UoW that is tied to the lifetime of the web request.  We don’t need to track instances or do change tracking in the session.  And NoRM also provides connection pooling so it’s perfectly reasonable to grab a Mongo connection every time you need to do something.

That’s great for atomic methods like Save or Remove that don’t return anything, or that return a single entity (like FindOne).  But it’s a problem for methods that return an IEnumerable<T> or IQueryable<T> because by the time the query actually instigates a connection to the database the session is long gone – and it’s not a good idea to “materialize” these into an array or list in order to return data from the method because then all the flexibility of LINQ is lost.

So there seems no alternative but to pass the session out to client code so that the user can control when it is released (after the query has been used).  Passing it out as IDisposable is great, though, so that consumers of the repository don’t need to know that the implementation is using Mongo.  And other Repository implementations can be slotted in instead.

I wanted to be able to use the repository in two ways.  Firstly for atomic operations I wanted to be able to call the relevant method on my repository instance like this (note that an instance of the repository would normally be injected into the constructor rather than calling the IoC container directly):

var repository = ObjectFactory.GetInstance<IRepository<User>>();
User user = repository.FindOne(userId);

And, secondly, for methods that return an enumerable I need to be able to materialize the query before disposing of the session:

var repository = ObjectFactory.GetInstance<IRepository<Role>>();
using (repository.NewSession())
{
    return repository.AsQueryable().Select(role => role.Name).ToArray();
}

I decided to use NoRM’s HiLo Id generator for my entities so that URLs are more friendly (Mongo’s ObjectId and Guids are not human – although I understand how they can be useful sometimes).  NoRM sees that I have a property called Id, and because it is Nullable<int>, NoRM uses the HiLo algorithm for me by default.  Objects have a null Id until they are saved by NoRM:

namespace RedBadger.Data
{
    public interface IEntity
    {
        int? Id { get; set; }
    }
}

So at the top of the Repository hierarchy is IRepository which simply exposes a method for creating a new session (for use by clients in their using blocks):

namespace RedBadger.Data
{
    using System;

    public interface IRepository
    {
        IDisposable NewSession();
    }
}

Then, from that, is derived an interface that adds familiar methods for interacting with the data:

namespace RedBadger.Data
{
    using System;
    using System.Collections.Generic;
    using System.Diagnostics.Contracts;
    using System.Linq;

    [ContractClass(typeof(IRepositoryContracts<>))]
    public interface IRepository<TEntity> : IRepository
        where TEntity : class, IEntity
    {
        IQueryable<TEntity> AsQueryable();

        void Drop();

        IEnumerable<TEntity> Find(Func<TEntity, bool> selector);

        TEntity FindOne(int? id);

        TEntity FindOne(Func<TEntity, bool> selector);

        void Insert(TEntity instance);

        void Remove(int? id);

        void Remove(Func<TEntity, bool> selector);

        void Save(TEntity instance);
    }
}

And the contracts for this interface are in a supporting class:

namespace RedBadger.Data
{
    using System;
    using System.Collections.Generic;
    using System.Diagnostics.Contracts;
    using System.Linq;

    [ContractClassFor(typeof(IRepository<>))]
    public abstract class IRepositoryContracts<T> : IRepository<T>
        where T : class, IEntity
    {
        public IDisposable NewSession()
        {
            Contract.Ensures(Contract.Result<IDisposable>() != null);
            return default(IDisposable);
        }

        public IQueryable<T> AsQueryable()
        {
            return default(IQueryable<T>);
        }

        public void Drop()
        {
        }

        public IEnumerable<T> Find(Func<T, bool> selector)
        {
            Contract.Requires(selector != null);
            return default(IEnumerable<T>);
        }

        public T FindOne(int? id)
        {
            Contract.Requires(id.HasValue);
            return default(T);
        }

        public T FindOne(Func<T, bool> selector)
        {
            Contract.Requires(selector != null);
            return default(T);
        }

        public void Insert(T instance)
        {
            Contract.Requires(instance != null);
            Contract.Requires(instance.Id == null);
        }

        public void Remove(int? id)
        {
            Contract.Requires(id.HasValue);
        }

        public void Remove(Func<T, bool> selector)
        {
            Contract.Requires(selector != null);
        }

        public void Save(T instance)
        {
            Contract.Requires(instance != null);
        }
    }
}

Now to the implementation.  Note that all the atomic methods (like Save) create their own session and dispose of it as soon as it’s finished with.  This allows the connection to return to the pool for re-use.  However, the methods that return an IEnumerable (like Find) simply ensure that the user has set up an external session and then use that.  Unfortunately, it’s not really possible to make sure that the user creates and disposes of these sessions in a responsible way.

namespace RedBadger.Data.Mongo
{
    using System;
    using System.Collections.Generic;
    using System.Diagnostics.Contracts;
    using System.Linq;

    using Norm;

    public class MongoRepository<TEntity> : IRepository<TEntity>
        where TEntity : class, IEntity
    {
        private readonly MongoConnectionStringBuilder mongoConnectionStringBuilder;

        private Mongo externalSession;

        public MongoRepository(MongoConnectionStringBuilder mongoConnectionStringBuilder)
        {
            this.mongoConnectionStringBuilder = mongoConnectionStringBuilder;
        }

        public IDisposable NewSession()
        {
            return this.externalSession = this.CreateSession();
        }

        public IQueryable<TEntity> AsQueryable()
        {
            this.EnsureSession();

            return this.externalSession.GetCollection<TEntity>().AsQueryable();
        }

        public void Drop()
        {
            try
            {
                using (Mongo session = this.CreateSession())
                {
                    session.Database.DropCollection(typeof(TEntity).Name);
                }
            }
            catch (MongoException e)
            {
                if (e.Message != "ns not found")
                {
                    throw;
                }
            }
        }

        public IEnumerable<TEntity> Find(Func<TEntity, bool> selector)
        {
            this.EnsureSession();

            return Find(this.externalSession, selector);
        }

        public TEntity FindOne(int? id)
        {
            return this.FindOne(x => x.Id == id);
        }

        public TEntity FindOne(Func<TEntity, bool> selector)
        {
            using (Mongo session = this.CreateSession())
            {
                return Find(session, selector).FirstOrDefault();
            }
        }

        public void Insert(TEntity instance)
        {
            using (Mongo session = this.CreateSession())
            {
                session.GetCollection<TEntity>().Insert(instance);
            }
        }

        public void Remove(int? id)
        {
            this.Remove(i => i.Id == id);
        }

        public void Remove(Func<TEntity, bool> selector)
        {
            using (Mongo session = this.CreateSession())
            {
                session.GetCollection<TEntity>().Delete(this.Find(selector));
            }
        }

        public void Save(TEntity instance)
        {
            using (Mongo session = this.CreateSession())
            {
                session.GetCollection<TEntity>().Save(instance);
            }
        }

        private static IEnumerable<TEntity> Find(Mongo session, Func<TEntity, bool> selector)
        {
            Contract.Requires(selector != null);

            IQueryable<TEntity> queryable = session.GetCollection<TEntity>().AsQueryable();
            Contract.Assume(queryable != null);

            return queryable.Where(selector);
        }

        private Mongo CreateSession()
        {
            return Mongo.Create(this.mongoConnectionStringBuilder.ToString());
        }

        private void EnsureSession()
        {
            if (this.externalSession == null)
            {
                throw new InvalidOperationException(
                    "This Repository has no current session.  Create one like this: using(repository.NewSession()) { ... }");
            }
        }
    }
}

I love the way that NoRM uses a URL for the connection string which means that I can use a class derived from UrlBuilder to set defaults for me:

namespace RedBadger.Data.Mongo
{
    using System;

    public class MongoConnectionStringBuilder : UriBuilder
    {
        public MongoConnectionStringBuilder()
            : base("mongodb", "localhost", 27017)
        {
        }

        public string Database
        {
            get
            {
                return this.Path;
            }

            set
            {
                this.Path = value;
            }
        }
    }
}

Finally, wiring it all up using StructureMap is really easy:

ObjectFactory.Configure(
    x =>
        {
            x.For<MongoConnectionStringBuilder>().Singleton().Use(
                () => new MongoConnectionStringBuilder { Database = "MyTestDB" });

            x.For(typeof(IRepository<>)).Use(typeof(MongoRepository<>));
        });

Incidentally, with StructureMap, you can also add more specialised repositories to the container.  At point of use you don’t need to know whether you’re getting a specialised repository or a more generic one, which is really cool:

ObjectFactory.Configure(
    x =>
        {
            x.For<MongoConnectionStringBuilder>().Singleton().Use(
                () => new MongoConnectionStringBuilder { Database = "MyTestDB" });

            x.For(typeof(IRepository<>)).Use(typeof(MongoRepository<>));
            x.For<IRepository<User>>().Use<UserRepository>();
        });

So that’s all straight forward and totally abstracts away the database driver implementation (the specialised UserRepository could easily be an implementation over another data store).  Anyway, enjoy.

7
Jun
2010

ReWork: A great way to spend a fiver

by Stuart Harris

ReWork I’ve just read "ReWork: Change the Way You Work Forever” by Jason Fried and David Heinemeier Hansson of 37 Signals.  It cost £5.49.  Not bad for a refreshing confirmation of our philosophy at Red Badger.  And lots of new insights and succinct rationale for doing business in a radically different way.  It takes about 3 hours to read – an investment that will pay out over and over.

Some of the ideas I found most interesting in the book are around building a business that is open.  Speaking, blogging, tweeting, writing, teaching, making videos.  About everything you do.  A behind-the scenes documentary, if you like.  Lots of progressive companies do this now and it’s incredible, today, how easily the Internet enables us to do this.

We’ve talked about this a lot as a team and are committed to sharing all our experiences with the community – good and bad, warts and all.  This “open book” policy fit’s well with our belief that honesty is the best way to do business.  Everyone wins.  A blog, for example, is a great place to talk about ideas and to crystallise them in your own head.  It forces you to research an idea thoroughly and to fully understand everything about it.  Building a community by giving away product, IP and knowledge is also a win-win, because it builds strong brand loyalty and a ready made channel for super effective marketing.

ReWork introduced me to the Japanese concept of wabi-sabi which describes an aesthetic that is derived from the characteristics of being "imperfect, impermanent, and incomplete".  In essence it’s the cracks and scratches that show you what’s really inside and allows the observer to appreciate the object for what it really is.  Beauty, derived from honesty. And openness. The lines in our faces show the world how we have laughed and cried. Shoot me if I ever have botox!

I recently enjoyed working for 2 years at Conchango which was known for it’s early adoption of Agile principles.  The name Conchango is derived from “continuous change” which embodies the wabi-sabi values of impermanence and incompleteness.  It was a real company, with an honest surface that reflected it’s core.  Small companies can do that well.  When it was taken over by a large multinational its wabi-sabi was lost.  It’s much, much harder for big companies to be agile, open and honest.

Papering over the cracks is the way that most businesses today operate.  They create a facade that is sterile, plastic and opaque – it doesn’t let the real company through.  The authors of ReWork remind us of how many times we have waited in a call queue and been told (by a machine) that we are important to them.  And yet they are willing to let us waste half an hour of our life waiting in a queue.  Just be honest with us, tell us you are under-resourced and call us back.  We would rather see the crack.  Then we can see your beauty, trust you and enjoy doing business.