Creating a responsive Image control
In part 6 we discovered problems even when we follow the guidelines from this series. In this case we find that sometimes it can be the controls themselves that are the cause of the lagging interface. In part 6 it was the Image control that was to blame. I make the assumption that it was trying to perform it's bitmap decoding on my precious dispatcher thread. However not only Image controls can get ugly, Charles Petzold shows some problems you may find when using chart controls (or any Items Control) in his Writing More Efficient Items Controls article.
The first thing I wanted to do was to just sub class Image however, I personally was stumped as to how I would then create my control template for it. Next I actually only wanted to accept URIs as my source so that I can do the decoding from file to an ImageSource myself explicitly and not on the UI thread via a TypeConverter. So I want a sub class of Control with a dependency property of UriSource and then 3 read-only properties ImageSource, IsLoading and HasLoadFailed. UriSource will provide the hook to bind your file name to. ImageSource will then provide the decoded ImageSource to be displayed. IsLoading and HasLoadFailed will be there so that you can update the UI appropriate to the lifecycle of the image.
The Style and control template I came up for this design was this:
<Style TargetType="{x:Type controls:Thumbnail}"> <Setter Property="Template"> <Setter.Value> <ControlTemplate TargetType="{x:Type controls:Thumbnail}"> <Image x:Name="ImageThumbnail" Source="{TemplateBinding Image}" HorizontalAlignment="{TemplateBinding HorizontalAlignment}" VerticalAlignment="{TemplateBinding VerticalAlignment}" MaxHeight="{TemplateBinding MaxHeight}" MaxWidth="{TemplateBinding MaxWidth}" Stretch="Uniform" StretchDirection="DownOnly" /> </ControlTemplate> </Setter.Value> </Setter> </Style>
Where I want to use the Thumbnail control I can have some XAML that hides and shows a loading indicator. This example here just shows the text "Loading Image..." but would probably have a nice animation or an indeterminate progress indicator.
<Border CornerRadius="4" BorderBrush="Silver" BorderThickness="1" Width="150" Height="150" Margin="10"> <Grid> <local:Thumbnail x:Name="Thumbnail" MaxWidth="148" MaxHeight="148" UriSource="{Binding}" HorizontalAlignment="Center" VerticalAlignment="Center" ToolTip="{Binding}"> <local:Thumbnail.Style> <Style TargetType="{x:Type local:Thumbnail}"> <Setter Property="Visibility" Value="Visible"/> <Style.Triggers> <DataTrigger Binding="{Binding RelativeSource={RelativeSource Self}, Path=IsLoading}" Value="True"> <Setter Property="Visibility" Value="Collapsed"/> </DataTrigger> </Style.Triggers> </Style> </local:Thumbnail.Style> </local:Thumbnail> <TextBlock Text="Loading image..." Foreground="Gray" HorizontalAlignment="Center" VerticalAlignment="Center"> <TextBlock.Style> <Style TargetType="TextBlock"> <Setter Property="Visibility" Value="Visible"/> <Style.Triggers> <DataTrigger Binding="{Binding ElementName=Thumbnail, Path=IsLoading}" Value="False"> <Setter Property="Visibility" Value="Collapsed"/> </DataTrigger> </Style.Triggers> </Style> </TextBlock.Style> </TextBlock> </Grid> </Border>
Well that is the easy bit, the public API. Trust me it gets more interesting.
My gut feeling was basically take the code from reflector and use that to decode Uris to ImageSource objects, but perform the action on a background thread. Two problems here:
- Most of the fun code that happens when the URI is being parsed and then decoded, is internal. :-(
- You cant pass most sub-classes of ImageSource across threads. If you create them on one thread they cant be accessed on another thread. So this stops us blindly copying code from reflector as all that code only runs on the dispatcher thread so wont face this problem.
We do have something to work with however: WriteableBitmap.
In some vain attempt at brevity (he says in his 7th post in the series!), I will try to skip over all of the brick walls I faced and jump straight to the solution I came up with. This was a great learning experience for me, and in the spirit of a learning experience the code is fairly rough (with TODO comments still intact). I will try to build this control into a stable control but in its current state is very much demo-ware.
First thing I want to achieve was to have the decoding and if possible the reading from disk happen off the UI thread. Lets start with the easy bit; reading the file into memory. I'm going to keep that simple and just go with grabbing the file as a byte array like this
byte[] buffer = File.ReadAllBytes(UriSource);
Next I want to decode the byte array into some form of ImageSource. To do this I have used a combination of the BitmapDecoder and the WriteableBitmap. First I take the byte array and load it into a MemoryStream, and then pass the stream to the BitmapDecoder factory method Create. This will return me an instance of one of its implementations (BitmapDecoder is abstract/MustInherit). From here I make a bold assumption that we only care about the first "Frame" of the image. I believe that most formats don't support multiple frames but formats like GIF do which allows them to have animation features. Anyway, in my demo-ware code I just take the first frame.
using (Stream mem = new MemoryStream(buffer)) { BitmapDecoder imgDecoder = null; imgDecoder = BitmapDecoder.Create(mem, BitmapCreateOptions.None, BitmapCacheOption.None); BitmapFrame frame = imgDecoder.Frames[0]; double scale = GetTransormScale(maxWidth, maxHeight, frame.PixelWidth, frame.PixelHeight); BitmapSource thumbnail = ScaleBitmap(frame, scale); // this will disconnect the stream from the image completely ... var writable = new WriteableBitmap(thumbnail); writable.Freeze(); return writable; }
From here I figure out the ratio that I want to scale it to. There doesn't seem much point in returning an 8MB image if we only want to see it as 300x300 does it? Next we request the image as what is almost our final product. We have a helper method scale the frame and return it as a BitmapSource. We can't assign this BitmapSource back to our ImageSource dependency property as they don't play nice over thread boundaries. So, the last thing we need to do is take the BitmapSource we just generated and create a WriteableBitmap from it and then call its Freeze method. This puts the WriteableBitmap in an immutable state which then makes it thread safe. Whew!
Just for reference here is the GetTransformScale method
private static double GetTransormScale(double maxWidth, double maxHeight, double currentWidth, double currentHeight) { double xRatio = maxWidth / currentWidth; double yRatio = maxHeight / currentHeight; double resizeRatio = (xRatio > yRatio) ? xRatio : yRatio; if (resizeRatio > 1) resizeRatio = 1; return resizeRatio; }
and the ScaleBitmap method
private static BitmapSource ScaleBitmap(BitmapSource source, double scale) { if (scale > 0.9999 && scale < 1.0001) { return source; } var thumbnail = new TransformedBitmap(); thumbnail.BeginInit(); thumbnail.Source = source; var transformGroup = new TransformGroup(); transformGroup.Children.Add(new ScaleTransform(scale, scale)); thumbnail.Transform = transformGroup; thumbnail.EndInit(); return thumbnail; }
So all things considered, once you get over the decoding stuff and then wrestle with the various subclasses of ImageSource that play their part, its not too bad. But like I said earlier; it gets more interesting. So far we have only looked at how to decode the image, we have yet to consider how to make the call to perform the decoding. My first instinct was to make the call on any change to the UriSource property. However I may not have the MaxHeight and MaxWidth information at that point in time. This caused me much stress over when should I call this decode functionality. If no value is ever going to be set for MaxHeight or MaxWidth then I should just process the image, but if first the UriSource is set then the MaxHeight, then the MaxWidth I would end up creating 3 asynchronous calls to render 3 different sized images. This would be a disaster as it would surely end up with race conditions and most likely the wrong sized image being displayed. The other obvious problem with that is that we would be performing 3 times the work. Hmm.
My solution (and mileage may vary as I am green to concurrent programming models) was to create a stack of render requests for each instance of a Thumbnail. As a property was set then a request would be added to the stack and a request to start processing the stack would occur. Periodically while processing the image I would check to see if the current work was invalidated by any new requests to the stack. If the current request was invalid it would terminate its work. The request to start processing the stack would simply try to pop the last request from the stack and clear out all other requests (effectively ignoring stale requests). On returning from the render request it would loop back and try to pop anything new from the stack. This last part while important is very much implementation details and could vary dramatically from anything you may implement. I have just read over the code myself and I could do with some work. It is amazing what just 3months (and reading Joe Duffy's Concurrent Programming for Windows) does for your appreciation of your code.
As always the code is available for you to have a play. Open it up, pick it to pieces, use what you like. Obviously I take no responsibility for the code if you do choose to use it as this is intended on being a learning exercise. Having said that I do try to produce good code for my demos so if you do see something that is not good enough let me know.
This has been a fun series and I hope you liked it and learnt as much as I did from it.
Back to series Table Of Contents
Working version of the code can be found here
