collective unconscious of the smarter planet

Forget workload optimization for big data. Microsoft makes a case for using arrays of cheap, commodity x86 servers to run big data analytics.

Infrastructure vendors’ vision of big data rigs based on scale-out NAS won’t come to fruition, according to the Microsoft executive heading the company’s big data push for SQL Server 2012.

David Campbell, a Microsoft Technical Fellow in the Data & Storage Platform Group claims personal responsibility for Microsoft’s adoption of Hadoop as both an app on the Windows platform and Azure. In conversation with The Register he declared himself “Diametrically opposed” to big data rigs built on scale-out storage. To explain his reason he challenged your correspondent to “Do a census of big data implementations. Ask how many are built on off-the-shelf products and how many are built on scale-out storage.”

All the examples Campbell could list were built on commodity hardware. “The evidence is there if you look,” he said, asserting that the market has already decided that a RAIS – redundant array of independent servers – approach typified by the way containerised data centres operate is superior to the somewhat exotic hardware involved in scale-out NAS or dedicated analytical appliances.

“I used to think the big online players were outliers,” he added. Now he thinks they got the approach to big data right the first time.

Campbell is also cool on the role of the data scientist, analytical experts who blend hard-core data-crunching skills with an understanding of moving bits at scale and can then translate their efforts into business insights. Such workers are in very short supply, he says, and industry cannot assume that the crop currently taking up the first university courses in the discipline will reach the workplace in the next five years.

Microsoft has therefore tooled SQL Server 2012 so it can satisfy a data scientist’s darkest big data desires, then pass the results of their efforts to lesser folk in IT and around the business.

“People ask me if they need to hire a data scientist,” Campbell asked a Microsoft event in Sydney today. “I say that if they can connect their people to the output a data scientist creates, maybe not all the work needs to be done in-house.” To help things along, SQL Server 2012 therefore includes Power View, a new data visualisation too which makes it easy - with the help of Excel - to turn data into something the average executive can understand.

Campbell is also optimistic that, over time, more suit-wearing types will be happy to drive tools like Power View, as “milennials” fluent in Excel syntax enter the workforce and start to crunch their own data.

Great article on how the CTO of Capgemini is looking at recent developments in enterprise adoption of the cloud. Good stuff.

As Capgemini’s CTO for North America, Joe Coyle hears an awful lot about cloud computing. He hears it from customers that want to evaluate cloud solutions and from vendors that want to win that business. Capgemini, a $12 billion global systems integrator, has relationships with all the major vendors and many enterprise customers, so it’s interesting to hear what Coyle has to say about the current state of the market.

Here are my main takeaways from a recent conversation with him.

1: IBM is cloudier than you think.

Big Blue has a pretty potent set of cloud options but it’s going about its business very cleverly. Given it’s big-iron heritage, IBM rarely talks about the hardware component of its cloud portfolio, Coyle said.

“They’re attacking this from a software perspective. They’ve taken Tivoli and are building this software umbrella so that you can take whatever you’re running in your data center now and put all or part of it in a public or private cloud,” he noted. IBM’s 2010 acquisition of Cast Iron also give it a slick appliance that lets customers integrate in-house apps with SaaS applications running outside.

He doesn’t see IBM cloud penetrating a ton of new smaller businesses, but for many existing IBM shops — and there are a ton of them — IBM cloud is a no brainer.

2: Microsoft Azure has a tough row to hoe

Coyle is of two minds on Windows Azure, the platform-as-a-service (PaaS) underlying Microsoft’s cloud strategy.

“Azure’s been a bit of a disappointment,” he said. “When Microsoft briefed us on it years ago, all the national [systems integrators] were chomping at the bit. But then it stumbled.”

“Then the message was the software would only run on Azure. That’ s fine, but by that point, the world had moved on, companies were already using Amazon,” he said. The usual argument that Azure is a PaaS while Amazon Web Services (AWS) is Infrastructure-as-a-Service (IaaS) simply doesn’t matter to most customers. The big AWS draw is they know they can deploy their applications on AWS now and move them to another hosted or in-house data center, later.

On the plus side, the Azure technology is solid and, unlike previous Microsoft development technologies, forces developers to follow the rules — they can’t design software services that misbehave. ”Azure is extremely powerful and if [Microsoft] can get its act together people will try it,” Coyle said.

But overshadowing all that technical mastery is the perception of Azure as a closed platform — despite its multi-language support.  Microsoft’s single biggest problem is customer suspicion that it will use Azure to lock them into the next wave of Microsoft technologies, essentially replacing the Windows/Office upgrade cycle.

“I’m not saying it’s true, but it’s what people think,” Coyle said.

3: Amazon is Amazon

Amazon Web Services are what they are: extremely flexible and leading the league in public cloud. AWS suffered a couple black eyes in 2011 with an embarrassing four-day outage in April and then a widespread reboot glitch later in the year.

Coyle is pretty forgiving of these miscues. The April outage, he said, was largely due to people  implementing their work incorrectly, something that AWS tried to fix manually. There are things you can do now in AWS to prevent this stuff, to build in more reliability and redundancy, although users will have to pay for it, he said.

The bottom line? Glitches and all, Amazon is the incumbent public cloud power and will stay that way, he said.

4: OpenStack as big-time cloud disruptor

Coyle is also bullish on the OpenStack movement, which is building a standard cloud foundation out of open-source tools. Initiated by Rackspace and NASA, it’s achieved critical mass with nearly every IT provider — from Dell, to HP, to Cisco, to Citrix — aboard and Rackspace offloading management to a more neutral OpenStack Foundation. 

“OpenStack will change the world of cloud computing. As a lot of smaller companies look to build their own clouds, this will be a natural choice,” Coyle said.

Who stands to lose if that’s the case? Ironically, the Dells and HPs of the world — all of which are building their own clouds. “Why do you think they joined?”  His feeling is these hardware companies — many of which were building their own more vendor-specific clouds — are hedging their bets.

Will OpenStack affect Amazon? “No. Amazon is Amazon,” he said.

5: CIOs are getting over cloud phobia

It’s taken time, but the economics of cloud computing are too good for CIOs to ignore, Coyle said. Any doubts they had about moving at least some corporate data to an outside cloud storage provider, for instance, have evaporated in recent months.

And they’re getting emboldened to do more than storage. The advent of Hadoop and NoSQL technologies means that companies could actually get some use out of all that old stuff sitting on tape or in platters, he said.  Uploading that information, and massaging it with the latest analytics means that historical data can be used to test assumptions and new models, for example, seeing what a price change means to sales over time.

Wringing real value out of old data is a pretty good proposition for most CIOs.

Gartner recently held their Symposium, and as they do every year, they offered up a new version of reinventing IT. This time, however, they weren’t very kind to the large IT incumbents, like IBM, HP, and Oracle.

Enterprise technology spending will rise 3.9 percent in 2012 and keep that rate through 2016, according to Gartner, which assumes that a recession is a foregone conclusion. Amid that budget backdrop IT departments will have to creatively blow up systems they’ve spent 20 years to build.

Peter Sondergaard, senior vice president of research at Gartner, laid out the research firm’s views of the years to come. It’s a world where chief marketing officers will have a larger IT budget than CIOs. It’s a world where traditional vendors—HP, IBM, Oracle, Microsoft, SAP and Cisco—will pull end runs around the IT department to gain sales. It’s also a world where enterprise architectures are useless amid mobile, social and cloud computing trends.

Welcome to the annual therapy session that is Gartner’s Symposium. Every year there’s some flavor of reinventing IT. “Architectures of the last 20 years will be obsolete,” said Sondergaard, who urged tech execs to create “post-modern businesses.”

Sondergaard sounded a good bit like NYU professor Clay Shirky, who also had an argument that consumerization was changing everything. While Gartner is urging creative destruction, I can’t help but be skeptical. If everyone could re-imagine IT and blow up old systems to delight customers, there would be no losers in the corporate world. As we know, there are plenty of losers and there will be thousands of companies that flop at people-centric system design.

Among the more notable points from Sondergaard and his merry band of Gartner analysts:

• The strategies of IBM, HP, Oracle, SAP, Microsoft, Cisco—old standbys for enterprise tech buyers—should be viewed as “long-term risky,” said Sondergaard. Going forward, these vendors should be judged on how they embrace mobile, social and cloud. Apple and Google will be disruptive enterprise vendors. You’ll buy from all of them.

My take: If tech giants are judged on mobile, social and cloud there will be some spectacular failures. If these key vendors convince IT buyers to buy more gear in the name of private clouds they can stick around. If not, look out below.

• CIOs will lose 25 percent of their IT spending control by 2014 to business units.

My take: This prediction is a no brainer. Salesforce.com and other SaaS vendors have traditionally sold to non-CIOs. All of the big guns see that sales trend. Expect IBM, HP, Oracle and others to sell to everyone but the CIO.

• Cloud brokers will be necessary to integrate public cloud services. IT departments will spend more than they wanted on cloud services without brokers.

My take: Cloud brokers are a logical step. Today, public cloud services are only 3 percent of IT spending, according to Gartner. That tally will only balloon.

• Social and enterprise systems will become intertwined. Sondergaard said social networking means there will be “mass employee involvement” with enterprise systems. “You must immediately incorporate social throughout your enterprise systems,” said Sondergaard.

My take: Obviously Gartner has watched a bunch of keynotes from Salesforce.com CEO Marc Benioff. I think CIOs get the social thing, but color me skeptical that they will all develop social enterprise systems starting next week. If anything they will buy a bunch of social malarkey from vendors they’ll regret later.

• Companies need to rethink systems for simplicity. The so-called postmodern business will develop systems that put customers first by simplifying. Less is more.

My take: Unless you blow up companies and countries and start from scratch this simplicity movement is pipe dream. There’s a whole generation of enterprise IT that has been raised to complicate systems. The customization curse will live on.

Microsoft recently made some big announcements around the SQL Server platform which contain some really interesting thinking on data structure, cloud, and visibility. ReadWrite Web explores the implications.

Of all the software technologies that are least suited to getting a makeover that’s “about the experience,” you’d think databases would rank pretty far down. The database experience, if there is one, is typically about accuracy, reliability, and speed. Certainly Oracle’s frequent measurements (“5x,” “10x,” “20x” and so on) are all about those metrics.

But Microsoft has found an angle with respect to SQL Server 2012, the second round of announcements for which came this morning in Seattle. The new angle starts with multitouch, but then it runs deeper, touching on the larger problem of data getting fragmented and redundantly duplicated as it gets used and visualized.

On the surface, it would be nicer for businesses to have a more flexible, adaptive system for visualizing large amounts of data and making correlative analyses. At a deeper level, the capability to share and collaborate by way of this visualization system could reduce, if not eliminate altogether, all the unnecessary duplication and versioning that comes from embedding data in e-mails. That’s the two stage goal described today for Microsoft Data Explorer, a touch-sensitive visualization tool that will orbit around the new SQL Server ecosystem.

Perhaps the most commonly used tool for visualizing data in an organization today is Excel. And as Quentin Clark, Microsoft’s own corporate vice president for database systems, tells RWW, that’s not really a good thing. When enterprise users see data from a large database in Excel, they show it to the workforce by cutting and pasting it into an e-mail message. Object Linking and Embedding, one of Microsoft’s oldest Windows technologies, gets cut out of the loop here. Inevitably, recipients save the spreadsheet fragment as an attachment.

Then employees make suggested changes and revisions. Suddenly versioning gets called into play. Fragments of the database abound, none of it in the database. Projections are made based on one or the other fragment. Those projections are loaded back into Excel. Charts are made from those projections, and get pasted into PowerPoint. Tools from PowerPoint and OneNote make further suggested revisions. And none of this involves the use of the original database or its own tools, beyond that initial cut-and-paste job.

“It’s not so much that we don’t want Excel in the loop,” says Clark about a product his team doesn’t manage. “We want to give IT tools to make data sharable, so [people] can stop using Excel as the way data and information gets flowed, over e-mail… [and] ensure that SharePoint can be used as a sharing and collaboration mechanism, as opposed to e-mail.”

Microsoft’s other hope is that Data Explorer will introduce more users to a concept that debuted with SQL Server 2008 R2. It currently goes by the name Managed Self-Service Business Intelligence, which already doesn’t sound like something you’d want to gift-wrap as an anniversary present. But it merges the company’s business intelligence (BI) tools with some of the visualization techniques that emerged from project “Crescent” and then became Data Explorer, with hooks that enable sharing views of the data through SharePoint without duplicating the data.

Does the cloud change the data?

Last week at Oracle OpenWorld, CEO Larry Ellison introduced his audience to several elements of “magic” that he explained could change the texture and behavior of databases as we know them (if they haven’t done so already). One of them involves the use of columnar storage, which twists the order in which the contents of tables are written by 90 degrees.

Imagine in your mind a data table. The fields in tables are always written to a storage device in some sequence. Usually that sequence is record by record (“1,” “David,” “Strom,” “2,” “Joe,” “Brockmeier,” “3,” “Scott, “Fulton…”). Columnar data changes this order (“1,” “2,” “3,” “David,” “Joe,” “Scott,” “Strom,” “Brockmeier,” “Fulton…”). This makes it a bit more difficult for query engines to draw analytical conclusions on the data. But in cases where the objective of a query is to read large quantities of fields in a column, if not the entire column, it expedites the process tremendously.

Depending upon the scale of the database and the types of analysis that need to be performed, the speed gain from changing to columnar could more than offset the speed loss in transaction processing and in write processes. Now, SQL Server 2012 enables a shift to columnar storage for selected tables.

“Instead of compressing things by rows and pages, this [technique] looks at a large set of data,” explains Clark. “Because these data warehousing tables have a lot of repeating data, when you look down columns, you can take advantage of that.” The technology was first introduced for PowerPivot for Excel during SS 2008 R2’s reign, using an alternative storage engine that flipped the contents of very large worksheet tables to one side. Once flipped, it became easier to compress columns with multiple, repeated fields. Imagine a personnel directory, for instance, where there are multiple “Smiths.” It’s much more efficient to compress “Smith,” “Smith,” “Smith…” than a table full of probably unique records adjacent to one another.

“Unlike a lot of our competitors’ columnar accelerators, which are separate products that have to be managed separately, we’re building an index type into the database. So there’s no separate thing to manage… You configure it to be indexed this way, and off you go. It’s a configuration operation, not a development or architectural operation.”

The good kind of fragmentation

Databases are getting larger, in large part (as Microsoft’s Doug Leland explained yesterday) due to the shifting ratio of machine-generated data to human-generated data. The NoSQL movement is grounded on the principle that data structures that make ordinary databases transactional and query-ready serve to stifle scalability.

But Microsoft’s Quentin Clark disagrees with the idea that removing structure from data enables performance increases in all scenarios. As an alternative, Microsoft announced today a sharding technique, where partitions of complete records are divided horizontally (as opposed to by column) and stored in separate regions. The pattern being utilized for programming for “shard-ability” is being called “Federation” (for now), and will apply to SQL Azure-hosted databases.

As Clark explains to us, when databases are designed for the appropriate constructs, they will automatically benefit from sharding in ways that are transparent to the customer. “You’re inherently programming into a partition form, which is what a lot of these new SQL databases do, so you get that scalability and performance built in to help you manage the shards, know where the hotspots are, move them around, split them, and so on.

“It has to do with the programming pattern; it’s not really a performance issue,” Clark continues. “It’s really about how fast you get moving, and how sure you are of scaling once you’ve programmed to a certain pattern.”

Just over the past few years of Microsoft customer briefings in which Clark has taken part, he tells us, the polarity of the cloud discussion has shifted. Whereas he used to have to bring the topic up himself (“Have you thought about the cloud?”), today attendees won’t let him talk as long as ten minutes without forcing the topic on him. “The conversation has shifted with customers,” he remarks. “One of the unique things we bring to the table is our collection of products — whether on-premise, or appliances, or from the cloud — that allows you to have choice… You develop one thing, and then you get deployment freedom from that.”

Amazon is making some headway with purpose-built clouds for government.

Amazon is upping the pressure on both Microsoft and Google in the battle to scoop up cash-strapped government customers into the cloud.

The online bookseller has moved to win over more government customers with the launch of AWS GovCloud, its cloud for US gov departments that crunch super-secret defence data.

AWS GovCloud is a region, Amazon’s sixth, which is designed to meet a tough set of US government rules known as the International Traffic in Arms Regulations (ITAR).

ITAR says controlled data must be stored in an environment where logical and physical access is limited to US persons or permanent residents. That data covers the import and export of a long list of goods and services, ranging from warheads and missile-control systems to aircraft carriers and tanks.

Without ITAR compliance, US departments could not legally have uploaded this data to Amazon’s existing service, given that compute and storage happens in three non-US regions.

AWS GovCloud is based on the US West Coast, Amazon said, while the service already meets the same US regulatory controls as the rest of its regions – including Federal Information Security Management Act (FISMA), PCI DSS Level 1, ISO 27001, and SAS 70.

Now the etailer says it is ready to talk to other governments about adopting GovCloud, potentially adopting similar regulatory restrictions.

Werner Vogels, Amazon chief technology officer, blogged: “We do not envision that over time GovCloud will address only the needs of the US government and contractors. We are certainly interested in understanding whether there are opportunities in other governments with respect to their specific regulatory requirements that could be solved by a specialized region.”

Government is a hot area for cloud providers. Amazon claims more than 100 federal, state and local government agencies already onboard its existing services.

Microsoft and Google, meanwhile, are running PR campaigns against each other, claiming government customers’ scalps to prove their cloud is winning against the others.

National and local government has proved a fertile hunting ground as the US’s economy has stalled and austerity becomes the watchword for the public sector.

Microsoft and Google are pushing hard to sign up agencies to their hosted email and docs. For the public-sector IT people involved, this has meant they get the prospect of brand-new systems without either the up-front purchase cost or the long-term maintenance costs.

Much of the Microsoft and Google fight has been over email and docs; at times, though, it has been point-scoring over precisely the kind of regulatory approval Amazon has on GovCloud. The idea has been to sow uncertainty about their rivals in customers’ minds or to reverse customers’ decisions.

In April, Microsoft accused Google of misleading the market about the FISMA certification of Google Apps for Government. Google shot back saying Microsoft’s cloud services for government wasn’t FISMA-certified. It is now suing the US Department of the Interior for its selection of Microsoft’s Business Productivity Online Suite-Federal – Microsoft’s rival to Google Apps for Government – saying the department had violated government procurement policies.

While Google and Microsoft tussle over details, both are behind on what they’d really like to be: the leading platform for hosting devs apps and data. That crown belongs to Amazon, which is now beginning to serve as a platform for other cloud businesses such as Ruby host Heroku.

Microsoft’s alternative to Amazon is Azure: the company claims a list of customers using the Azure compute fabric and storage layer to deliver apps, but it has yet to find Amazon-levels of traction.

Microsoft this week re-jigged the pricing and packaging on its smallest level of compute instance for the third time since last autumn’s launch.

Beyond the headline, the interesting news here is that many company are deploying multiple hypervisors in their data centers.

VMware is the primary hypervisor for 58% of organizations that use x86 virtualization software, with Citrix and Microsoft’s Hyper-V splitting the rest of the market, a new survey finds.

Commissioned by virtualization management vendor Veeam and conducted by research group Vanson Bourne, the V-Index survey of more than 500 large businesses found that 92% have adopted virtualization, and deployed an average of 470 virtual machines. Although VMware is still the most widely used, it turns out multiple hypervisors in the same data center is fast becoming the norm. And with VMware’s latest price increases, Microsoft and Citrix have a chance to chip into its lead.

Fully 58% of companies using virtualization call VMware their primary hypervisor, while 20.2% put Citrix at the top of the list and 18.6% count Hyper-V first.

Yet when counting all virtualization use, 84% of virtualizing companies use VMware, 61% use Hyper-V, 55.4% use Citrix’s XenServer and 12% use other hypervisors. Only 3% use an “other” hypervisor as their main one.

Vanson Bourne surveyed IT decision makers at 544 organizations in the United States, United Kingdom, France and Germany, each of which has at least 1,000 employees. Exactly half of the companies use virtualization.

The results suggest that Microsoft, until recently an also-ran in the virtualization market, is making progress. But with VMware being counted as the main hypervisor for nearly 60% of customers, it seems VMware is still the top choice for running mission critical applications.

Veeam said it will update the survey every three months.

While Veeam’s survey provides a snapshot of today’s usage, IDC’s latest quarterly numbers show that customers are continuing to buy VMware products at a rapid pace.

In Q1 2011, worldwide sales of new virtualization licenses deployed into production on new or existing servers was 58% VMware, 26% Microsoft and 8% Citrix, with others accounting for the remainder, according to IDC.

Overall, 20% of new x86 servers have virtualization technology pre-installed or installed at the time of deployment, IDC also says. While Veeam’s numbers suggest that most enterprises are using virtualization, the IDC survey shows that doesn’t mean most of their servers are virtualized.

In fact, Veeam’s survey also showed that just under 40% of servers are virtualized, with an average consolidation ratio of six virtual servers per host.

Microsoft’s share could continue to increase because of anger among VMware customers over new licensing prices. In postings on a VMware community forum, one customer professed to be “totally floored” by the new prices, while another used the phrase “still in shock” and claimed to be emailing a VMware sales rep to complain.

VMware increased the number of licenses required to virtualize systems that meet certain thresholds of sockets and memory, causing one customer to call it a “penalty on density.”

“Our team prefers vSphere over Hyper-V, but with this new pricing we might be forced to switch,” one customer wrote.

Another Microsoft post for today. Frank Scavo of Constellation Research discusses how Microsoft is moving from being the perennial “bad guy” to being a champion in the enterprise IT market. Here’s how he sees them doing it.

So, as I’m listening to keynotes and conducting one-on-one interviews with Microsoft executives and partners, I’m getting the feeling that Microsoft is losing its place as everyone’s favorite punching bag. In fact, it has a real opportunity to be the good guys in the enterprise IT marketplace.

I see this in three ways.

1. Offering safe platforms

Microsoft takes a lot of criticism for its proprietary technologies–especially from open source advocates (of which, I am one). I still believe that open source technologies, such as Linux, AJAX, and others, are a great foundation for building enterprise software. But, increasingly, independent software vendors (ISVs) are seeing Microsoft as another safe alternative.

It is not generally known, for example, that SAP–the granddaddy of enterprise software–is using Microsoft Visual Studios as the platform for customers to script custom logic in its new Business ByDesign (ByD) cloud-based ERP system. SAP formerly did nearly all development in its proprietary ABAP language as well as in Java. But now, SAP apparently feels that Microsoft’s C# is a better choice for its end-users, at least for ByD. When I asked SAP about this a few months ago, an SAP executive told me, it’s because most of the target market for ByD–small business–is already using Microsoft technologies.

This week, at the WPC, I ran into an executive of a Tier II ERP vendor, which competes with Microsoft Dynamics. I was surprised to see him at a Microsoft event. Why was he here, I asked. “Because, we’re a Microsoft partner,” he replied. “We use .NET, Lync, Sharepoint, and Microsoft’s business intelligence capabilities as part of our product strategy. We’re also using Azure to build cloud-based mobility apps.”

Later in my one-on-one interviews with Microsoft executives, I asked about this. How can you compete with these enterprise software vendors in your Dynamics business, yet turn around and support them with your technology? The answer, in so many words, is that in the big picture, Microsoft will be more successful by being a safe platform provider for other vendors, than it will be by hoarding its technologies only for use by its own applications business.

So, at a time when other enterprise software vendors are questioning their commitment to Java, in light of Oracle’s acquisition of Sun, Microsoft is starting to look like one of the good guys.

2. Focusing on cloud-value

Despite Steve Ballmer’s talk about being “all in with the cloud,” Microsoft’s actual progress has been slow. From this perspective, Microsoft is seen as a laggard, falling behind cloud infrastructure providers such as Amazon, as well as SaaS providers, such as Salesforce.com and NetSuite. This has been my view for some time, and I still feel this way.

But from interviews with Dynamics executives, it’s clear that there is some deep thinking going on about the cloud. Specifically, what is the value of cloud computing to customers? Is it only in cost-savings through outsourcing the infrastructure to a low-cost platform? Is it with all workloads equally, or with certain workloads? Are there parts of the enterprise suite that customers will more likely want to retain in-house, or with a trusted third party hosting provider, while moving other parts to a shared multi-tenant environment? What scenarios favor multi-tenant as the preferred architecture, due to the relationship between the tenants?

With many enterprise IT vendors today, where you stand on the cloud depends on where you sit. If you are a NetSuite or Salesforce.com, the only valid strategy is to have everything delivered as a pure multi-tenant SaaS offering. If you are a Larry Ellison, SaaS is a myth, or if you are a Harry Debes, the SaaS industry will collapse in two years.

But, with Microsoft there’s no such dogmatism. Rather, it is thinking hard about where customers find the most value in cloud computing and is working to prioritize its migration to the cloud to focus on those value propositions. I just wish they would get there faster.

3. Enabling entrepreneurs

The third way in which Microsoft is the good guy is in the opportunity it offers to entrepreneurs. We all know that Microsoft’s sells a lot of products to small and mid-size businesses. But Microsoft is also small-business-friendly in its partner channel. Attending the WPC is a real eye-opener: thousands of partners, mostly small businesses, many entrepreneurial, enabled by Microsoft’s channel program. During these economic times, when everyone is championing small business as the key to economic prosperity, Microsoft is enabling thousands of entrepreneurs and small businesses worldwide to grow and compete successfully. In fact, IDC recently estimated the total 2010 revenue of the Microsoft partner ecosystem at US $580 billion. Compare that to Microsoft’s revenue of approximately $60B, and you can see that every dollar Microsoft makes results in about 8 or 9 dollars of revenue for its partners. That’s a big opportunity for Microsoft’s 640,000 partners worldwide.

My interviews with three Microsoft partners also gave me insight into how these small businesses are winning in these difficult times. I interviewed Jeff Geisler, owner of Socius, a traditional CPA-type partner, which is growing steadily through the recession by acquiring smaller firms. I also met with Steve Thompson and Jim Sheehan from PowerObjects, a Microsoft CRM partner with strong development capabilities. Finally, I had a sit-down with Paul Tilling and Bob Hadingham at LexisNexis. Paul and Bob’s group is an independent software developer in the UK that has taken its software for law firm practice management and is migrating it to Dynamics AX as its underlying platform. These three businesses have different focuses, but each is betting its business on Microsoft’s partner channel.

With some other enterprise IT vendors, being a partner is a risky bet, as you sometimes find yourself competing against the vendor’s direct sales force. Or, the vendor has a shifting strategy on where it wants to allow its partners to do business. Microsoft, by running 95% of its revenue through the channel, has no such conflict.

Detailing some of the limitation of Azure and how it integrates with on-premise Windows deployments.

If Microsoft Azure is just Windows in the cloud, is it easy to move a Windows application from your servers to Azure?

The answer is a definite “maybe”. An Azure instance is just a Windows virtual server, and you can even use a remote desktop to log in and have a look. Your ASP.NET code should run just as well on Azure as it does locally.

But there are caveats. The first is that all Azure instances are stateless. State must be stored in one of the Azure storage services.

Azure tables are a non-relational service that stores entities and properties. Azure blobs are for arbitrary binary data that can be served by a content distribution network. SQL Azure is a version of Microsoft’s SQL Server relational database.

Reassuringly expensive

While SQL Azure may seem the obvious choice, it is more expensive. Table storage currently costs $0.15 per GB per month, plus $0.01 per 10,000 transactions.

SQL Azure costs from $9.99 per month for a 1GB database, on a sliding scale up to $499.95 for 50GB. It generally pays to use table storage if you can, but since table storage is unique to Azure, that means more porting effort for your application.

What about applications that cannot run on a stateless instance? There is a solution, but it might not be what you expect.

The virtual machine (VM) role, currently in beta, lets you configure a server just as you like and run it on Azure. Surely that means you store state there if you want?

In fact it does not. Conceptually, when you deploy an instance to Azure you create a golden image. Azure keeps this safe and makes a copy which it spins up to run. If something goes wrong, Azure reverts the running instance to the golden image.

This applies to the VM role just as it does to the other instances, the difference being that the VM role runs exactly the virtual hard drive (VHD) that you uploaded, whereas the operating system for the other instant types is patched and maintained by Azure.

Therefore, the VM role is still stateless, and to update it you have to deploy a new VHD, though you can use differencing for a smaller upload.

If your application does expect access to persistent local storage, the solution is Azure Drive. This is a VHD that is stored as an Azure blob but mounted as an NTFS drive with a drive letter.

You pay only for the storage used, rather than for the size of the virtual drive, and you can use caching to minimise storage transaction and improve performance.

No fixed abode

The downside of Azure drive is that it can be mounted by only one instance, though you can have that instance share it as a network drive accessible by other instances in your service.

Another issue with Azure migrations is that the IP address of an instance cannot be fixed. While it often stays the same for the life of an instance, this is not guaranteed, and if you update the instance the IP address usually changes.

User management is another area that often needs attention. If this is self-contained and lives in SQL Server it is not a problem, but if the application needs to support your own Active Directory, you will need to set up Active Directory Federation Services (ADFS ) and use the .NET library called Windows Identity Framework to manage logins and retrieve user information. Setting up ADFS can be tricky, but it solves a big problem.

Azure applications are formed from a limited number of roles, web roles, worker roles and VM roles. This is not as restrictive as it first appears.

Conceptually, the three roles fulfill places for web applications, background processing and creating your own operating system build. In reality you can choose to run whatever you wish in those roles, such as installing Apache Tomcat and running Java-based web solution in a worker role.

Caught in the middle

Middleware is more problematic. Azure has its own middleware, called AppFabric, which offers a service bus, an access control service and a caching service. At its May TechEd conference, Microsoft announced enhanced Service Bus Queues and publish/subscribe messaging as additional AppFabric services.

As Azure matures, there will be ways to achieve an increasing proportion of middleware tasks, but migration is a substantial effort…

“While Microsoft claims you can just pick up an application and move it onto Azure, the truth is it’s not that simple,” Hines says.

“To really get the benefit, in terms of the scale-out and so on, designing it for Azure up front is probably a much better idea.

“But the same could be said for deploying an application on Amazon Web Services, to be fair to Microsoft.”