Nov 2 2011

SwimEventTimes W7P Development -- Isolated Storage and Tombstoning

Category: � Administrator @ 17:10

At the outset of development I really didn't look into tombstoning and what that entailed but I knew I had to make use of Isolated Storage for keeping the content resident on the phone once it had been "SCRAPED" from the target URL.  You're asking yourself why am I talking about Isolated Storage and tombstoning together.  Well as it turn out, getting your app back to the state that it was in prior to tombstoning requires Isolated Storage.  I have designed my Isolated Storage with a GUID as its key:

Guid SwimmerGuid;
SwimmerGuid = Guid.NewGuid();

Once I have the data from the Target URL I generate a new GUID and that becomes the key that is passed from one page to the next.  In the "navigatedTo" pages, this is passed in the QueryString (think ASP.NET) and the rest of the data values are re-hydrated from Isolated Storage:

this.NavigationService.Navigate(new Uri("/MeetDetails.xaml?selectedGUID=" +
                    curSwimmer.SwimmerID + "&Meet=" + meetName, UriKind.Relative));

This makes easy work of adding new pages since you can construct your called page and know that you'll have the data available from Isolated Storage.

I'm talking about Isolated Storage as if I'm constantly hitting those files but in reality I simply read the stored data (into ObservableCollection) when the App starts up and I rewrite it when I add new data from the target URL:

private void Application_Activated(object sender, ActivatedEventArgs e)
        {
            this.LoadSavedApplicationState();
        }
void LoadSavedApplicationState()
 {
     if (Database.Instance.RequestedSwimmers == null)
     {
         using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
         {
             if (store.FileExists("RequestedSwimmers.xml"))
             {
                 using (IsolatedStorageFileStream stream = store.OpenFile("RequestedSwimmers.xml", System.IO.FileMode.Open))
                 {
                     XmlSerializer serializer = new XmlSerializer(typeof(ObservableCollection<RequestedSwimmer>));
                     Database.Instance.RequestedSwimmers = (ObservableCollection<RequestedSwimmer>)serializer.Deserialize(stream);
                 }
             }
             else
             {
                 Database.Instance.RequestedSwimmers = new ObservableCollection<RequestedSwimmer>();
             }

        }

    }

When I need to write, I add new swimmers to the ObservableCollection and write it back to Isolated Storage:

if (Database.Instance.RequestedSwimmers != null)
         {
             using (IsolatedStorageFile store = IsolatedStorageFile.GetUserStoreForApplication())
             {
                 using (IsolatedStorageFileStream stream = store.CreateFile("RequestedSwimmers.xml"))
                 {
                     XmlSerializer serializer = new XmlSerializer(typeof(ObservableCollection<RequestedSwimmer>));
                     serializer.Serialize(stream, Database.Instance.RequestedSwimmers);

                 }
             }
         }

 

This way Isolated Storage is always up to date.

Resuming from tombstoning is straight forward since all the pages are called with a QuerySring (GUID) and the ObservableCollection has already been re-hydrated from Isolated Storage.

 

 

 

Tags: , , , , ,

Oct 2 2011

SwimEventTimes W7P Development -- HTML Agility Pack

Category: � Administrator @ 08:13

When this app started out, a long time ago, I happened upon the HTML Agility Pack or HAP.  This tool was originally written for .Net and used XPath to search and define the elements that you needed from the original source HTML.  So when it comes time to SCRAPE data from a page, you'll need a tool that provides a robust and almost carefree nature about the HTML structure.  It does so much for you that without it I would have never attempted what I was thinking:

http://htmlagilitypack.codeplex.com/

Kudos to the authors, especially DarthObiwan, of this fantastic piece of work.

Basically it allows you to define what tags you want from a page and search and pull out that piece of the text.  But the twist here is that for HAP to work on the phone it had to work without Xpath since Xpath was not supported in the OS 7.0 release on the phone.

So what took the place of the Xpath is LINQ.  This added yet another dimension to my learning since I had never really used LINQ and had only recently started looking into using LINQ when I thought about converting a project from XSLT.  It takes time to make the mental switch from Xpath to LINQ but there was no other way.  Also, at that time, none of the code releases for HAP worked on the phone but by getting the source and following the comments of the authors on how to re-engineer the code I was able to get it compile and now it works like a charm.

So now let's talk about how we use HAP:

Here we have entire HTML loaded up in htmDoc.

//let's detemine what came back on the response

HtmlAgilityPack.HtmlDocument htmDoc = new HtmlAgilityPack.HtmlDocument();
htmDoc.LoadHtml(responseData);

Next you can start to look for specific items:

string pattern = @".*txtSearchLastName$"; 
var SearchPagenode = htmDoc.DocumentNode
                      .Descendants("input")
                      .FirstOrDefault(x => Regex.IsMatch(x.Id, pattern));


So now I can look at the element and get the id:

CTLID = SearchPagenode.Id;

Other things like pulling out <a> tags out of table contained in the last <tr>:

pattern = @".*dgPersonSearchResults$"; 
var links = htmDoc.DocumentNode.Descendants("table")
           .First(n => Regex.IsMatch(n.Id, pattern))
          .Elements("tr").Last() .Descendants("a")
          .Select(x => x.GetAttributeValue("href", "")).ToArray();


You can go crazy with HAP and as your LINQ gets better you can go further and refine these queries.

HAP provides the foundation and performs the grunt work required to interrogate/parse a HTML source.


Tags: , , , ,

Sep 2 2011

W7P SwimEventTimes development - Fiddler2

Category: Features � Administrator @ 11:19

Back to the beginning.  When you need content and the content exists on other web sites you'll have to devise a way to acquire the data.  And I'm not talking about OData or some other nice structure of data lying around on the internet.  What I'm talking about is, for a lack of better term, "SCRAPING" data from other URL's.  Some would call this Web Scraping but this technique really applies to a whole realm of techniques and not just in use on the web.

Some believe that type of development is fraught with problems since your tying your app to a URL and that web design.  Just know these pitfalls up front and try to mitigate as many problems as possible.

Try and contact the URL's owners and try to get them to come onboard with your development.  That way at least you'll have knowledge of upcoming changes which could cause side-effects on your W7P app.

Disclaimer:  I'm a developer and not a copyright lawyer but be warned that some sites believe that they own the data and have the sole rights to that data.  It's the old "Creative" vs "Collection" argument. Collection of data is not creative but do your own copyright research.

Now let's talk about overall design.  Some designs call for an intermediate server to make the calls to the target URL but I opted to make the Windows 7 Phone software smart enough to exist on its own and pull the data directly from the target.  This minimizes the points of failure. It's either working from the W7P or not plus it also permits the phone to operate anonymously from the target URL's perspective.

Now the main tool that I quickly made use of is Fiddler2.  Fiddler2 allows you to look at the payload when making calls across the web.  This includes everything that is sent from your Request and the subsequent Response.

The site from which the data for this app will be scraped is an ASP.NET site.  I can't be sure but the site appears to make use of a Content Management System. I could have gone further into looking at the CMS since I believe that it's Open Source and pulling apart the code but I decided to spend the time on how and if I could get access to the data across the web from the phone. 

Other sites and their software platforms and how they implement cookies/session data etc. will change the makeup of how you access the pages but you need to start with Fiddler2.

I know IE9/Firefox have similar developer tools but when this application started up, Fiddler2 was my choice.

Get to know Fiddler2 and exactly what pages you need to hit.  Make screen shots of the flow that you'll need to capture and test scenarios that will cover the flow of pages.

You and Fiddler2 will spend many long nights together.

Tags: , ,

Aug 6 2011

W7P SwimEventTimes development - Touch and Hold Gesture vs Discovery

Category: Features � Administrator @ 11:20

I had implemented a "Touch and Hold" gesture for editing and deleting items from the Swimmer's page. 

The "Touch and Hold" would bring up a Context menu for editing and deleting a selected swimmer.

After receiving complaints from beta users that this was not discoverable I thought I needed to provide a better experience for the user. 

What I ended up using was a List box with check boxes and additional icons in the application bar to control the experience.  So before, I had a context menu with "Edit" and "Delete" which was revised into two additional icons on the application bar.

I made great use of the following example:

http://listboxwthcheckboxes.codeplex.com/

Now it's quite apparent on how to delete swimmers and edit the swimmer's date range when you need to refresh and download with the latest meet data.

 

 

 

Tags: , , , ,

Jul 5 2011

W7P SwimEventTimes development -- Tilt Effect

Category: Features � Administrator @ 05:38

As many developers, I've completed most of the development using the phone emulator since I wanted to wait until Verizon was offering a Windows 7 Phone.  While the emulator provides an invaluable tool, you soon realize you missed something when you start developing and testing with a real device.

For this development I immediately noticed that the "Tap" gesture was not fluid enough to give the user a feeling that they had just touched an element on the screen. 

So here I employed the the "Tilt Effect" as so nicely provided by:

http://www.scottlogic.co.uk/blog/colin/2011/05/metro-in-motion-part-4-tilt-effect/

I went with the following value since I think it gave me a better pushed-in effect:

<StackPanel Orientation="Horizontal" HorizontalAlignment="Left" local:MetroInMotion.Tilt="5">

Now touching an element gives a nice feedback to the user so that they can feel as if the element on screen was pushed in.  I've used this in all the places where the action of touching the element would bring you another page.  Specifically I added it to the Swimmer's Page (Add, Edit, Delete) and the Panorama page displaying the "Best", "Meets" and "Strokes" data.

 

Tags: , , ,