Windows Azure

In this episode of “Azure Lessons Learned” Rob Fraser from RiskMetrics talks about the work they’ve done with Windows Azure to scale some of their heavy computational workloads out to thousands of nodes on Windows Azure.

RiskMetrics specializes in helping to manage risk for financial institutions and government services.  The solution they built on Windows Azure is primarily for calculating financial risk for their clients.  Calculating the risk on portfolios of financial assets is an incredibly compute-intensive problem to solve (Monte Carlo simulations on top of Monte Carlo simulations).  There is an ongoing and increasing demand for this type of computation.  RiskMetrics calculations require enormous computational power but the need for that power tends to come in peaks.  That means the required hardware is idle for much of the time.  Windows Azure solves this problem by allowing RiskMetrics to quickly acquire the very large number of required processors, use them for a short time and then release them.

Channel 9: Windows Azure Lessons Learned: RiskMetrics

To give you a sense of the scale RiskMetrics is talking about, the initial target is to use 10,000 worker roles on Windows Azure.  And that’s just a beginning as Rob thinks they could eventually be using as many as 30,000.

While using Windows Azure may help control costs, the real motivation is having the kind of compute power they need to build analytic services for their clients that they just wouldn’t otherwise be able to do easily.

Rob goes in to some depth on the architectural pattern they devised to ensure the efficient flow of work packets from their data center into the cloud for processing and then back again with the results.  The architecture is an interesting hybrid of on-premises and cloud computing.

Rob (along with his colleague Phil Jacob) also presented some of this in a PDC session.

In this episode of Lessons Learned I chat with Rex Hansen of ESRI.  Rex works on MapIt; a product for visualizing enterprise data on maps.  This was recently released as a on-premises product that enables developers to work with the tabular and spatial data in SQL Server 2008 and integrate that data with maps on ArcGIS online and Bing Maps. 

ESRI has been working to extend that functionality to Windows Azure and SQL Azure.  MapIt takes advantage of SQL Azure to consume location-based data.  The MapIt spatial data service can be deployed as a role on Windows Azure and provides spatial data capabilities to applications using SQL Azure.  This provides a valuable service to folks that miss the spatial data types they were used to using in SQL Server.

Channel 9: SQL Azure Lessons Learned: ESRI 

Rex walked me through building a Silverlight application with their Silverlight Control Toolkit in Expression Blend.  As Rex mentions in the video ESRI has released the source code for their Silverlight toolkit controls.  You can find those on CodePlex here: http://ESRISilverlight.codeplex.com/

It sounds like ESRI has some big plans for where they want to take the MapIt product to provide even better integration with SQL Azure.

You can find out more about the ESRI MapIt product here.

Telerik is one of the more popular component vendors building good stuff for ASP.NET, Silverlight, and WPF.  I was intrigued by the interest they have taken in the Windows Azure Platform.  Naturally all the UI components just work on the platform with no changes required.  Telerik has gone further in looking at how they can adapt their Object Relational Mapper (ORM) and Content Management System (CMS) technologies to take advantage of the benefits of Azure.

Stephen Forte is their Chief Strategy Officer (love that title :)) responsible for thinking about their new technology directions.  In this episode of Lessons Learned, Stephen shows me how they’re now able to go from SQL Azure tables to persistent classes when developing applications using their OpenAccess ORM product.

Channel 9: SQL Azure Lessons Learned: Telerik

The process of implementing a SQL Azure provider for OpenAccess was fairly straightforward.  Telerik ran unit tests of their existing SQL Server provider against SQL Azure to find the differences.  They found the few minor issues and were able to get a working version running in three weeks or so.

The biggest difference they found between SQL Server and SQL Azure was the data types.  SQL Azure supports most of the SQL Server data types but some (like spatial, text, etc) are not supported.  Another difference was multiple active result sets (MARS).  This one was a little trickier and with some work they were able to get the same end-result using different t-sql.

The OpenAccess ORM product is available now.  Telerik is also dogfooding it internally using another of their products: Sitefinity CMS.  The next release of Sitefinity will then support SQL Azure as a data store.

A few links mentioned in this episode:

• Telerik team blogs are here
• Stephen’s blog is here
• OpenAccess ORM product page
• Sitefinity CMS product page

In this episode of Azure Lessons Learned I chat with Dmitry Sotnikov, new product research manager with Quest Software.  Quest, as you probably know, is a huge global ISV focused primarily on Systems Management software.

Quest is an early adopter of the Windows Azure platform.  They’ve been working on a new offering for their various management business.  They’ve built out an extensive services framework as well as a few service offerings on that framework. 

Channel 9: Windows Azure Lessons Learned: Quest Software

Quest has hundreds of solutions for the enterprise.  These are your typical on-premises that would normally require hardware and people to install and maintain those solutions.  The Quest OnDemand project Dmitry is working would extend those offerings to the small and medium business by making many of the solutions available as subscription services.  The first 3 offerings are Recovery Manager OnDemand for Active Directory, InTrust OnDemand event log management, and Site Administrator Reports OnDemand for SharePoint.  These are now available in beta.

Dmitry walks us through the experience of setting up Recovery Manager OnDemand to provide backup and recovery of AD.  The solution running in Windows Azure does most of the work in the cloud while sending requests to the local infrastructure via agents that are running locally.  There is no need to set up servers or install anything beyond that simple local agent service.

Dmitry was very open about the process they went through to adapt their existing solutions to this new subscription model.  He mentions that they were able to re-use about 50% of the code for one of their solutions (AD Recovery Manager).

Since Quest OnDemand is meant to be on solution with many different component offerings it was important that there be a consistent framework across all those offerings (including login authentication/access control, portal, subscription/payment/billing).  Even within Quest I think they were a little surprised as to how fast they could port applications over once that framework was in place and available to re-use.

Dmitry also goes into detail about the architecture they have built out for this set of services including the portal, Windows Identity Foundation (formerly Geneva) Security Token Service (STS), and the individual services.  He also is very open about what they learned along the way of this development.  It’s interesting to note that the bulk of the Quest products are on-premises so although they had many large installations there had never been a call for super-scalable super-secure multi-customer/multi-tenant deployments like the one required by Quest OnDemand.

I love that Windows Azure played such a big role in helping Quest to adapt the way that they do business.

Have a look at the Quest OnDemand Service here:

• http://www.quest.com/ondemand/

In this episode of Lessons Learned we talk with Jones Pavan and Gurleen Randhawa of CCH about tax and accounting!  No wait stick around, that stuff can be exciting too.  Yes, really! 

The good folks at CCH (a Wolters Kluwer company) have built an interesting service on Windows Azure.  The solution we discuss here is a sales tax calculation service which they offer to other accounting firms.  This is an existing on-premises product that they are now moving to the cloud.

Channel 9: Windows Azure Lessons Learned: CCH

The existing product was a stateless web service that was designed to live behind the firewall.  The service is meant to be called directly via a plug-in in an accounting firms ERP system (for example, Dynamics AX).  To move that to the cloud CCH wrapped the web services in Windows Communication Foundation (WCF).

They had been using another third party RAD development tool called CA Plex.  The Plex runtime was added to the project and copied out to the cloud.  One of the things they quickly learned is that the nature of the cloud app is to be stateless and that required special consideration when moving on-premises apps (for example the Plex tool was caching db connections behind the scenes).

Anther important consideration was security.  They were not ready to move to ACS so for the initial release they used X.509 certs, ADFS and message-based security to establish trust relationships with the server.

BTW, the Windows Azure marketing folks have already published a case study on the CCH solution (available here).

There are not many solutions that can claim to have saved lives.  In this episode of Lessons Learned I chat with Richard Prodger of Active Web Solutions about the Windows Azure project they’ve been working on that tracks fishermen in real time.  It monitors not only their location but also their status so as to immediately raise the alarm if help is needed (e.g. fallen off the side of a boat or pressed a panic button).  This solution is already credited with saving the lives of 9 fishermen.

Channel 9: Windows Azure Lessons Learned: Active Web Solutions

Electronics on the fishing vessels communicate directly via satellite to the Windows Azure solution.  Those messages are processed via Windows Azure worker roles and routed using the Windows Azure AppFabric Service Bus to various on-premises systems for review and action.  The desktop client overlay marine charts onto Bing maps so that the coast guard gets a visual representation of the exact location of boats that have raised alarms.

The good folks at Active Web Solutions have published some of the source code that they developed to “automatically bridge arbitrary TCP endpoints, handling any intermediate firewall traversal.”  The code is available on CodePlex as the SocketShifter project:

http://socketshifter.codeplex.com/

If this is interesting, you should also have a look at Port Bridge published by Clemens Vasters on his blog.  Clemens describes it as “Socketshifter’s older brother”

http://blogs.msdn.com/clemensv/archive/2009/11/18/port-bridge.aspx

One question that is often asked is how hosters can benefit from the Windows Azure Platform.  While the platform can be used to deploy many types of web apps we expect many partners including hosters to develop on top of the Windows Azure platform infrastructure.  In this episode of Azure Lessons Learned I chat with Paul Lappas, VP Engineering at GoGrid and Mehul Shah and Madhavrao Pachupate from Blue Star Infotech.  GoGrid has been working on a hybrid solution that builds on the GoGrid infrastructure to assist in development and load testing of Windows Azure applications.

For more information on the GoGrid solution for Windows Azure have a look here: http://www.gogrid.com/azure/

In this episode of Windows Azure Lessons Learned I chat with Paul Forney, System Architect for Invensys and Aleksey Savateyev, Senior Architect in Microsoft’s Global ISV group working with Invensys.  Invensys is well known for industrial automation and control systems.  They’ve been working to develop a system for the power industry to manage the large network of smart meters that will be used to build out smart grids delivering electricity from suppliers to consumers.  To do this Invensys is using Windows Azure AppFabric (formerly called “.NET Services”).  The AppFabric Service Bus is the magic that allows this type of application.  It allows those meters not only to connect across the cloud to on-premises systems but also does it a way that can scale to the millions of homes and businesses that will form the smart grids.

Channel 9: Windows Azure Lessons Learned: Invensys

Database tooling is important for many developers and DBAs as they manage numerous databases across the enterprise and the cloud.  In this episode of Azure Lessons Learned I chat with Scott Walz, Sr. Director Product Management at Embarcadero Technologies responsible for the DBArtisan product.

Scott walks us through the DBArtisan product to show how SQL Azure integrates seamlessly into this cross-DBMS product.  It was interesting to hear how quickly the effort to add SQL Azure went.  I think that bodes well for other tooling in general for SQL Azure.  Since SQL Azure is so very close to SQL Server it should be relatively simple for ISVs to add SQL Azure support to products that support SQL Server today.

Channel 9: Azure Lessons Learned: Embarcadero

Scott mentions a DBArtisan download for SQL Azure.  You can grab that here.

As always let me know your comments below.

I’m back with another episode of Azure Lessons Learned.  In this one I discuss building apps that need to be able to scale very highly very quickly.  Jim Zimmerman, the CTO and Lead Developer for Thuzi, walks me through the Facebook application his team built for Outback Steakhouse.

Thuzi specializes in building social media applications for large organizations to connect with their customers on sites like Facebook.  The concern with these types of applications is the possibility that the offer will go viral as it gets passed from customer to customer and potentially swamp the servers to the point that they are not responsive and ultimately disappoint customers.  Thuzi chose to use Windows Azure since the Windows Azure platform has ample capacity and they could scale up or down the solution based on the current demand of the market.  The Outback Steakhouse offer in this case was a free Bloomin’ Onion at any of the thousands of Outback Steakhouse restaurants.

Facebook apps don’t actually run on Facebook.  They are embedded using an iframe into a Facebook page.  The app that is running in that iframe must be hosted somewhere else.  In this case, Thuzi hosts that in Windows Azure. They actively monitor the campaign and turn on or off web and worker role instances as required.  In order to scale they used Windows Azure table storage and queues.  They also use SQL Azure to perform reporting and analytics on the results of the campaign.

Channel 9: Azure Lessons Learned: Outback Steakhouse Facebook App