Saturday, 20 December 2008

Building a Silverlight RSS reader - part 2 - getting the data.

Continuing from part 1, we're now going to extend our Silverlight application to get data (available feeds and actual RSS postings) which we will shortly bind to in the UI.

One of the things we need to work around is the cross domain security in Silverlight. It doesn't allow you to send any requests to services/sites outside of the domain in which the Silverlight xap is hosted without explicit permission from the service/site you're trying to call. This permission is granted by placing a cross domain policy file on the target server, but in our case we can't guarantee that our target RSS feed servers will have one in place.

Silverlight has this restriction for a very good reason - it would be very easy to create a distributed denial of service attack using Silverlight if it was able to call any web resource directly. Imagine making a nice Silverlight app that is used by hundreds of people and putting a payload in there that cripples a target web site.

To work around this we can send a request for data back to a web service in the ASP.NET application that is hosting the Silverlight application. This is then free to send a request to the target server for the data we're looking for (thereby circumventing any DDoS attack possibility as now all requests are coming from the server).

Defining the feeds.
Before we look at that though, we'll start with the simpler task of getting a list of available feeds. For this, create a feeds.xml file within your web solution (in the root) with content similar to the following. (feel free to replace with your own feeds).

   1: <?xml version="1.0" encoding="utf-8" ?>
   2: <feeds>
   4:   <feed id="ScottGuthrie">
   5:     <title>Scott Guthrie</title>
   6:     <url></url>
   7:   </feed>
   9:   <feed id="Deepcode">
  10:     <title></title>
  11:     <url></url>
  12:   </feed>
  14:   <feed id="AyendeRaihen">
  15:     <title>Ayende Raihen</title>
  16:     <url></url>
  17:   </feed>
  19: </feeds>

Now that we have a list of feeds, we need to get this up to Silverlight and populate the drop down list of available feeds. We could use a web request to simply load the XML, but as we're going to be building a WCF service anyway to marshal requests out to feeds, we might as well piggy back on those contracts and have one cohesive service.

Defining the service, operation and data contracts.
Now, under normal circumstances I'd advocate the use of several namespaces to separate and re-use the contracts etc. I'd have separate assemblies for the data contracts (RSSReader.Services.Contracts) another to define the "specification" of the services, or in other words, the operation contracts (RSSReader.Services.Spec) and another for the service itself. (RSSReader.Services). As you can't re-use assemblies in Silverlight though, this seems a little pointless for this particular exercise so we're just going to go with a single assembly - RSSReader.Services.

Let's go ahead and create this "class library" now. Right click the solution and select Add new project and select C# class library. Delete the automatically generated class1.cs file.

Add references to System.Runtime.Serialization and System.ServiceModel. These references allow WCF constructs to be used within your application;


image And finally, create 3 directories within the new project - Contracts, Repositories and Spec. In contracts we will define the classes that will be used to represent the data being sent between Silverlight and our service, in Spec we'll define the operational contracts - the methods we can call on our service and in repositories we'll add the data access code to get data from sources.


We now want to define what data we're going to expect to receive from Silverlight and what data we're going to send back to it. We're going to have two operations on our service - a method to get a list of all available RSS feeds, and a method to get the postings within a selected feed.

For the method to get a list of feeds, we won't have any data passed in, but we want to pass back out a list of feeds. We'll wrap the list of feeds into a response object in case we need to add anything new later. As such we'll have the following classes for querying what feeds we can select;


Notice the use of the DTO suffix. This stands for Data Transfer Object and is standard terminology when talking about what data will be exchanged between services. See the enterprise stack posting for more information about this.

When we request feed articles, we need to tell the service which feed we want to get articles for, and have it return the articles in a list, again wrapped in a response object should we need to expand it later. This gives us these classes;


These contracts are all just plain old CLR objects (POCOs), onto which we add WCF specific attributes to tell the service that the class is a data contract and what members should be serialised and sent over the wire. We do this using the DataContract and DataMember attribute. The code for GetFeedRequestDTO is shown below;

   1: using System.Runtime.Serialization;
   2: using System.Collections.Generic;
   4: namespace RSSReader.Services.Contracts
   5: {
   6:     [DataContract]
   7:     public class GetAvailableFeedsResponseDTO
   8:     {
   9:         private readonly IList<RSSFeedDTO> _feeds 
  10:             = new List<RSSFeedDTO>();
  12:         [DataMember]
  13:         public IList<RSSFeedDTO> Feeds
  14:         {
  15:             get 
  16:             {
  17:                 return _feeds;
  18:             }
  19:         }
  20:     }
  21: }

As you can see, we've marked the class as being a data contract and the Feeds property as being a data member.

The entire listing for the various contracts is shown below;

   1: /***** GetAvailableFeedsResponseDTO.cs *****/
   2: using System.Runtime.Serialization;
   3: using System.Collections.Generic;
   5: namespace RSSReader.Services.Contracts
   6: {
   7:     [DataContract]
   8:     public class GetAvailableFeedsResponseDTO
   9:     {
  10:         private readonly IList<RSSFeedDTO> _feeds = new List<RSSFeedDTO>();
  12:         [DataMember]
  13:         public IList<RSSFeedDTO> Feeds
  14:         {
  15:             get 
  16:             {
  17:                 return _feeds;
  18:             }
  19:         }
  20:     }
  21: }
  23: /***** RSSFeedDTO.cs *****/
  24: using System.Runtime.Serialization;
  26: namespace RSSReader.Services.Contracts
  27: {
  28:     [DataContract]
  29:     public class RSSFeedDTO
  30:     {
  31:         [DataMember]
  32:         public string Id { get; set; }
  34:         [DataMember]
  35:         public string Title { get; set; }
  37:         [DataMember]
  38:         public string FeedUrl { get; set; }
  39:     }
  40: }
  42: /***** GetFeedRequestDTO.cs *****/
  43: using System.Runtime.Serialization;
  45: namespace RSSReader.Services.Contracts
  46: {
  47:     [DataContract]
  48:     public class GetFeedRequestDTO
  49:     {
  50:         [DataMember]
  51:         public string FeedId { get; set; }
  52:     }
  53: }
  55: /***** GetFeedResponseDTO.cs *****/
  56: using System.Runtime.Serialization;
  57: using System.Collections.Generic;
  59: namespace RSSReader.Services.Contracts
  60: {
  61:     [DataContract]
  62:     public class GetFeedResponseDTO
  63:     {
  64:         private readonly IList<FeedArticleDTO> _articles 
  65:             = new List<FeedArticleDTO>();
  67:         [DataMember]
  68:         public IList<FeedArticleDTO> Articles
  69:         {
  70:             get 
  71:             {
  72:                 return _articles;
  73:             }
  74:         }
  75:     }
  76: }
  78: /***** FeedArticleDTO.cs *****/
  79: using System;
  80: using System.Runtime.Serialization;
  82: namespace RSSReader.Services.Contracts
  83: {
  84:     [DataContract]
  85:     public class FeedArticleDTO
  86:     {
  87:         [DataMember]
  88:         public string Title { get; set; }
  90:         [DataMember]
  91:         public DateTime Published { get; set; }
  93:         [DataMember]
  94:         public string BodyContent { get; set; }
  95:     }
  96: }

With our data contracts defined, we can now look at defining the operation contracts - the services themselves. As mentioned above, we're going to have a single service, lets call it RSSProxyService, and it will have two methods. A method to get all of the feeds that are available and a method to get the articles for a selected feed. The signatures of these methods will use the data contracts defined above. Within the Spec folder, create a new class file and name it IRSSProxyService.cs. The code should be as follows;

   1: using System.ServiceModel;
   2: using RSSReader.Services.Contracts;
   4: namespace RSSReader.Services.Spec
   5: {
   6:     [ServiceContract]
   7:     public interface IRSSProxyService
   8:     {
   9:         [OperationContract]
  10:         GetAvailableFeedsResponseDTO GetAvailableFeeds();
  12:         [OperationContract]
  13:         GetFeedResponseDTO GetFeed(GetFeedRequestDTO request);
  14:     }
  15: }

In WCF we define services as interfaces and then tell the service definition which concrete class to use that implements this interface. As you can see our two methods are marked up with [OperationContract] attributes, indicating that these are methods that will be exposed through the service and the interface itself is defined with a [ServiceContract] attribute.

We now need to provide a concrete implementation of the service. As such, create a new class called RSSProxyService.cs within the root of RSSReader.Services project;

   1: using System;
   2: using RSSReader.Services.Spec;
   3: using RSSReader.Services.Contracts;
   5: namespace RSSReader.Services
   6: {
   7:     public class RSSProxyService : IRSSProxyService
   8:     {
   9:         public GetAvailableFeedsResponseDTO GetAvailableFeeds()
  10:         {
  11:             throw new NotImplementedException();
  12:         }
  14:         public GetFeedResponseDTO GetFeed(GetFeedRequestDTO request)
  15:         {
  16:             throw new NotImplementedException();
  17:         }
  18:     }
  19: }

If you build now, things should build, but as we've not hooked anything up yet, it won't do a great deal. Lets make it actually do something and write some unit tests to check our implementation actually works.

Implementing the repository
Before we can get into the guts of the service itself, we need to think about how we're going to get data out of feeds.xml and from the RSS feeds themselves. I've chosen to implement a repository class so that my service concentrates on logic and uses the repository to handle any data persistence/access.

Before we dive headlong into writing a repository class though, we need to give some thought to testing. We want to be able to unit test this service to verify that it is working as anticipated, but we want to test the logic, not test if the internet is up and running (which is what we would be doing if we consumed an RSS feed directly - pass/fail would be dependent upon the RSS feed being available).

As such, we are going to want to replace the repository code when testing with a pre-canned or mocked version of it. This will allow us to control what data is sent to the service when we invoke it. To do this we must define an interface for the repository.

Within the repositories folder, create a new file - IFeedRepository.cs with the following content;

   1: using System.Collections.Generic;
   2: using RSSReader.Services.Contracts;
   4: namespace RSSReader.Services.Repositories
   5: {
   6:     public interface IFeedRepository
   7:     {
   8:         IEnumerable<RSSFeedDTO> GetFeedDefinitionList();
   9:         RSSFeedDTO GetFeedDefinitionById(string id);
  10:         IEnumerable<FeedArticleDTO> GetArticlesForFeedId(string id);
  11:     }
  12: }

In this instance, our feed repository will offer 3 methods. One to get the feed definition list, one to get a specific feed definition record from the list and one to get all of the articles for a specified feed id.

We can then implement this by creating FeedRepository.cs, as follows;

   1: using System;
   2: using System.Collections.Generic;
   3: using System.IO;
   4: using System.Linq;
   5: using System.Net;
   6: using System.Xml.Linq;
   7: using RSSReader.Services.Contracts;
   9: namespace RSSReader.Services.Repositories
  10: {
  11:     public class FeedRepository : IFeedRepository
  12:     {
  13:         private static XDocument GetFeedDefinitionData()
  14:         {
  15:             string filename = Path.GetFullPath(
  16:                 AppDomain.CurrentDomain.BaseDirectory) + "\\feeds.xml";
  17:             XDocument doc = XDocument.Load(filename);
  18:             return doc;
  19:         }
  21:         /// <summary>
  22:         /// Loads the feed definition list from the XML file "feeds.xml"
  23:         /// </summary>
  24:         /// <returns></returns>
  25:         public IEnumerable<RSSFeedDTO> GetFeedDefinitionList()
  26:         {
  27:             XDocument doc = GetFeedDefinitionData();
  29:             IEnumerable<RSSFeedDTO> list =
  30:                 from item in doc.Descendants("feed")
  31:                 select new RSSFeedDTO
  32:                 {
  33:                     Id = (string)item.Attribute("id"),
  34:                     Title = (string)item.Element("title"),
  35:                     FeedUrl = (string)item.Element("url")
  36:                 };
  38:             return list;
  39:         }
  41:         /// <summary>
  42:         /// Gets one feed definition item from feeds.xml
  43:         /// </summary>
  44:         /// <param name="id"></param>
  45:         /// <returns></returns>
  46:         public RSSFeedDTO GetFeedDefinitionById(string id)
  47:         {
  48:             XDocument doc = GetFeedDefinitionData();
  50:             // query it for feed items and create RSSFeedDTO objects
  51:             var matchedFeeds =
  52:                 from item in doc.Descendants("feed")
  53:                 where ((string)item.Attribute("id")) == id
  54:                 select new RSSFeedDTO
  55:                 {
  56:                     Id = (string)item.Attribute("id"),
  57:                     Title = (string)item.Element("title"),
  58:                     FeedUrl = (string)item.Element("url")
  59:                 };
  61:             if (matchedFeeds.Count() < 1) return null;
  63:             return matchedFeeds.First();
  64:         }
  66:         /// <summary>
  67:         /// Gets all of the articles available from the RSS feed
  68:         /// definition id specified.
  69:         /// </summary>
  70:         /// <param name="id"></param>
  71:         /// <returns></returns>
  72:         public IEnumerable<FeedArticleDTO> GetArticlesForFeedId(string id)
  73:         {
  74:             // Find the feed
  75:             RSSFeedDTO feed = GetFeedDefinitionById(id);
  76:             if (feed == null) throw new ArgumentException(
  77:                     String.Format("Feed id {0} not found", id));
  79:             // Get the RSS payload
  80:             WebClient client = new WebClient();
  81:             string responsePayload = client.DownloadString(feed.FeedUrl);
  83:             // Load it into LINQ to XML and find all items, 
  84:             // translate to FeedArticleDTOs
  85:             XDocument rssResponse = XDocument.Parse(responsePayload);
  87:             IEnumerable<FeedArticleDTO> list =
  88:                 from item in rssResponse.Descendants("item")
  89:                 select new FeedArticleDTO
  90:                 {
  91:                     Title = item.Element("title").Value,
  92:                     Published = DateTime.Parse(item.Element("pubDate").Value),
  93:                     BodyContent = item.Element("description").Value
  94:                 };
  96:             return list;
  97:         }
  98:     }
  99: }

As you can see, we're using LINQ to XML to query the XML data from feeds.xml and transform the data into our DTO classes, and when we retrieve data from the physical RSS feeds we're going to use the WebClient object, and again query and extract this with LINQ.

Implementing the service
With our repository in place we can now build our service implementation. Go back to our RSSProxyService.cs and change it's code to match this;

   1: using RSSReader.Services.Spec;
   2: using RSSReader.Services.Contracts;
   3: using RSSReader.Services.Repositories;
   6: namespace RSSReader.Services
   7: {
   8:     public class RSSProxyService : IRSSProxyService
   9:     {
  10:         // The feed repository we will use
  11:         private readonly IFeedRepository _feedRepository;
  13:         public RSSProxyService()
  14:         {
  15:             _feedRepository = new FeedRepository();
  16:         }
  18:         public RSSProxyService(IFeedRepository feedRepository)
  19:         {
  20:             _feedRepository = feedRepository;
  21:         }
  23:         /// <summary>
  24:         /// Interrogates the feeds.xml file to return a list of 
  25:         /// available RSS feeds that can be displayed
  26:         /// </summary>
  27:         /// <returns></returns>
  28:         public GetAvailableFeedsResponseDTO GetAvailableFeeds()
  29:         {
  30:             GetAvailableFeedsResponseDTO feeds 
  31:                 = new GetAvailableFeedsResponseDTO();
  33:             foreach (RSSFeedDTO feed in 
  34:                 _feedRepository.GetFeedDefinitionList())
  35:             {
  36:                 feeds.Feeds.Add(feed);
  37:             }
  39:             return feeds;
  40:         }
  42:         /// <summary>
  43:         /// Gets articles from the requested RSS feed.
  44:         /// </summary>
  45:         /// <param name="request"></param>
  46:         /// <returns></returns>
  47:         public GetFeedResponseDTO GetFeed(GetFeedRequestDTO request)
  48:         {
  49:             GetFeedResponseDTO response = new GetFeedResponseDTO();
  50:             foreach (FeedArticleDTO article in 
  51:                 _feedRepository.GetArticlesForFeedId(request.FeedId))
  52:             {
  53:                 response.Articles.Add(article);
  54:             }
  56:             return response;
  57:         }
  58:     }
  59: }

If you notice our constructors now, we either create or accept an IFeedRepository object depending on which constructor is used - this allows us to write (or mock) a replacement repository, specifically for testing this service. (This is an example of coding for testability as when not thinking about unit testing, you wouldn't write code like this).

Our implementation of the service itself is now complete. We have to now either expose this via our web interface and write all the front end code to consume it in order to test it, or we need to write unit tests to satisfy ourselves that things are working as they should. I'd sooner find problems as early as possible, so lets write some unit tests.

Writing unit tests to check the service
Add a new project to your solution, a test project and call it RSSReader.Services.Tests.


You can then clear out the AuhtoringTests.txt file, and rename the UnitTest1.cs file to RSSProxyServiceTests.cs. Also copy the feeds.xml file into your test project. You will need to tell the test project to deploy the feeds.xml file when it runs the tests, as follows;

Select the Test menu and choose "Edit Test run configurations" and then "Local test run". From the resultant dialog, select the deployment page and select "Add File" - choose feeds.xml from the RSSReader.Services.Tests directory and click Apply to save the changes.


In order to provide our service with a canned set of data from the repository, we're going to "mock" the repository using a mocking framework.

We're going to use the excellent Rhino Mocks mocking framework by Oren Eini (aka Ayende Rahien). You'll need to download this from here, extract it, and then add a reference in your test project to Rhino.Mocks.dll. Also add a reference to the RSSReader.Services project.

Finally change the RSSPRoxyServiceTests.cs to match the following;

   1: using System;
   2: using System.Collections.Generic;
   3: using Rhino.Mocks;
   4: using Microsoft.VisualStudio.TestTools.UnitTesting;
   5: using RSSReader.Services.Repositories;
   6: using RSSReader.Services.Spec;
   7: using RSSReader.Services.Contracts;
   9: namespace RSSReader.Services.Tests
  10: {
  11:     [TestClass]
  12:     public class RSSProxyServiceTests
  13:     {
  14:         /// <summary>
  15:         /// This test validates that the feeds.xml file can be loaded
  16:         /// correctly and that it contains 3 defined feeds.
  17:         /// </summary>
  18:         [TestMethod]
  19:         public void ValidateFeedsLoaded()
  20:         {
  21:             IRSSProxyService service = new RSSProxyService();
  22:             GetAvailableFeedsResponseDTO response = 
  23:                 service.GetAvailableFeeds();
  25:             Assert.IsNotNull(response);
  26:             Assert.AreEqual(3, response.Feeds.Count);
  27:         }
  29:         /// <summary>
  30:         /// This test uses a mocked IFeedRepository to returned canned
  31:         /// data into the RSS proxy service implementation to allow it's 
  32:         /// logic to be tested. It ensures the correct number of articles
  33:         /// are returned when a feed is queried by ID.
  34:         /// </summary>
  35:         [TestMethod]
  36:         public void ValidateArticleCounts()
  37:         {
  38:             MockRepository mocks = new MockRepository();
  40:             // Define the data we want the IFeedRepository to return
  41:             IEnumerable<FeedArticleDTO> feedList = new List<FeedArticleDTO>
  42:             {
  43:                 new FeedArticleDTO{Title="Article 1", 
  44:                     Published=DateTime.Now, BodyContent=""},
  46:                 new FeedArticleDTO{Title="Article 2", 
  47:                     Published=DateTime.Now, BodyContent=""},
  49:                 new FeedArticleDTO{Title="Article 3", 
  50:                     Published=DateTime.Now, BodyContent=""},
  52:                 new FeedArticleDTO{Title="Article 4", 
  53:                     Published=DateTime.Now, BodyContent=""},
  55:                 new FeedArticleDTO{Title="Article 5", 
  56:                     Published=DateTime.Now, BodyContent=""}
  57:             };
  59:             // Mock the feed repository and setup some expectations
  60:             IFeedRepository feedRepository = 
  61:                 mocks.StrictMock<IFeedRepository>();
  63:             using (mocks.Record())
  64:             {
  65:                 Expect.Call(feedRepository.GetArticlesForFeedId("MOCK"))
  66:                     .Return(feedList);
  67:             }
  69:             GetFeedResponseDTO response;
  70:             using (mocks.Playback())
  71:             {
  72:                 // Test the service - the service will use the mock repository
  73:                 IRSSProxyService service = 
  74:                     new RSSProxyService(feedRepository);
  76:                 // Request the MOCK feed
  77:                 GetFeedRequestDTO request = new GetFeedRequestDTO();
  78:                 request.FeedId = "MOCK";
  80:                 // Get the response and test
  81:                 response = service.GetFeed(request);
  82:             }
  84:             Assert.IsNotNull(response);
  85:             Assert.AreEqual(5, response.Articles.Count);
  86:         }
  87:     }
  88: }

This test class contains two specific tests that will test the workings of the service itself - ValidateFeedsLoaded will test that the service can load the feeds.xml file and correctly assemble the XML into the objects we're expecting.

ValidateArticleCounts is a little different in that it's using mocks. We define what data the repository is going to return when invoked, create a mocked object of the repository and then setup some expectations on what we expect the service code to do with our mocked repository.

In this case, we tell the mocking framework that we expect the service to call the repositories GetArticlesForFeedId method with a parameter of "MOCK", and when it does, the mock framework should return the canned data we've setup.

We then invoke the service, passing in the mocked repository so that it is used instead of the default and then we invoke the service with the parameters expected. If all goes well, the expected calls will be invoked and we will have a response from the service with the correct number of articles.

Now, for those eagle eyed among you, you will notice we're not testing the functionality of the repository. We certainly should, but as this series of posts is supposed to be about Silverlight, I'll leave that as an exercise for the reader - but we would have to encapsulate the WebClient so that it, itself, could be mocked so the repository is tested, not the availability of the RSS feeds...

If you now build your application and run it you'll find we have nothing more than we did at the end of the last article!! However we do have a completed service implementation and we can use test view to run the two unit tests and validate that our service is functional.



(NOTE: When you run your unit tests, you may get a "Test run deployment issue" saying "Rhino.Mocks.dll" is not trusted. This is because you downloaded the files from the Internet. The solution is to find the DLL's on your machine using explorer, right click each DLL and click "Unblock" - then do a rebuild and you should be good to go)

That's all for this instalment (it was longer than I expected!) - in part 3 we'll look at exposing the above service through the web project, consuming it in Silverlight and having fun with data binding.

Again, have a good Christmas!

Get the source code here.