collective unconscious of the smarter planet

Oracle recently released the first post-Sun version of Solaris. Ideas International gives a long, detailed report on Solaris 11.

In early November, Oracle announced the release of Oracle Solaris 11, a major update to its UNIX operating system (OS). Solaris 11 introduces many new features, and following Oracle’s announcement of SPARC T4 systems in September, the update serves as more proof that the company is making significant investments in the Sun server platform that it acquired in 2010. Further, the release of Solaris 11 will put considerable muscle behind Oracle’s strategy to deliver a vertically integrated systems architecture that is optimized for the use of its own technology at every level. This announcement is not just of interest to Solaris users. It could strengthen Oracle’s value proposition for any users who rely on Oracle software for enterprise computing.

Operating systems remain an integral part of IT infrastructures. The rise of virtualization has shifted many of traditional OS responsibilities, such as controlling hardware, to hypervisors and equivalent mechanisms for enabling multitenancy of workloads. However, for most users, the OS still serves as the central point of integration between applications and operational processes. UNIX systems have become the preferred OS for high-end datacenter workloads in many commercial enterprises. Long before it was acquired by Oracle, Sun made a commitment to drive mainframe-grade reliability and performance functions into Solaris, and the OS has earned a notable track record for its proven ability to host a customer’s most demanding applications.

What’s New in Solaris 11

While Solaris 11 has many new features, some of the enhancements related to virtualization and cloud computing stand out in particular:

• Virtualization – Oracle Solaris Zones is a virtualization mechanism that provides workloads with a private set of OS resources that are protected from each other. Sometimes referred to as “virtual servers,” zones create the illusion that workloads are running virtually on their own machines, with a distinct set of OS resources that are visible to the applications. While virtual machines each host a complete, separate copy of an OS, zones all share a single copy of the host OS, regardless of how many are zones are deployed. As a result, zones allow workloads to be consolidated with greater efficiency than other virtualization methods. Since zones are based entirely on software, they complement other virtualization solutions available from Oracle that are implemented closer to hardware, including Oracle VM Server for SPARC (formerly called Logical Domains, or LDOMs), Dynamic Domains (hard partitions for some SPARC servers, which provide electrical isolation between separate OS instances), and on x86 servers, Oracle VM, a version of the Xen hypervisor. While zones were already available in Solaris 10, their implementation in Solaris 11 benefits from a completely redesigned networking stack, which has been virtualized and optimized for use with zones. As a result, administrators can control the network resources that are assigned to a particular zone much more precisely in Solaris 11 than before. The new virtualized network engine also helps administrators manage network resources for applications that are not hosted in zones, with the ability to define virtual network “traffic lanes” that have different priorities, depending on the importance of a particular workload.
• Scalability and Performance – Solaris has long been optimized for use on large Symmetric Multiprocessing (SMP) systems, and Solaris 11 adds new refinements for increasing its performance on servers with very large numbers of cores. The Solaris 11 scheduler has increased awareness of variants in topologies used to connect processor sockets in large servers, and it also takes I/O latencies into account when scheduling access to memory. The memory management in Solaris 11 has also been optimized specifically to improve the performance of the Oracle database on systems with large amounts of memory. The new network stack has a parallelized architecture that will improve network performance on large SMP systems, and it fully exploits hardware in network adapters to offload more network processing operations, freeing the server processors to take on more computing tasks under heavy network I/O loads. InfiniBand support in Solaris 11 has been extended with the Reliable Datagram Sockets (RDS) V3 protocol, which will provide better performance and observation for Oracle Real Application Clusters (RAC) databases.
• Security – In Solaris 11, the mechanism for auditing system behavior has become much more efficient, and it is now turned on by default (in previous releases of Solaris, the auditing mechanism could slow performance, and it was therefore disabled by default). Delegated administration, that is, the ability to give select users a limited range of administrative control, has been made more flexible to accommodate the management of virtualized workloads. Now, administrators can be assigned the rights to manage specific zones, limiting their control to that zone rather than the entire host. Immutable zones can be configured with read-only file systems, so that virtualized workloads can be hosted in an environment with a locked-down system configuration. Solaris 11 can provide transparent hardware encryption for the Oracle Database Advanced Security Option, taking advantage of the Advanced Encryption Standard (AES) cipher feedback mode in SPARC T4 processors to perform table space encryption. Solaris 11 also supports the Trusted Platform Module (TPM) specification, which can be used to give virtualized workloads a definitive ID for the hardware on which they are running…

Solidifying the Solaris Base

Solaris 11 is the first major update of the OS to be released since Oracle acquired Sun, and its new features clearly indicate that Oracle is continuing to invest in Solaris. While the new release doesn’t make any huge architectural leaps forward, it does offer a mix of practical improvements targeting key concerns that administrators have today. Many new features will be most appreciated by current Solaris users, rather than serving to attract new users to the platform. For example, the new frameworks for managing software and patches should help to reduce the burden of maintaining applications based on Solaris and will thus help build the confidence of existing Sun users to maintain their investments in the platform.

Other improvements, though, could become potent differentiators as Oracle steps up efforts to assert itself in datacenters with the server platforms that it acquired with Sun, and as it competes for new opportunities in virtualization and cloud computing. With rising interest in cloud computing, networks will play a more strategic role than ever before, and the new virtual networking functions in Solaris 11 could become a key asset as virtualization expands from servers to storage and networking. Early in its existence, Solaris was aggressively optimized for the TCP/IP protocol underlying the Internet, even as most other UNIX operating systems were still being tuned for other proprietary network protocols. Solaris 11’s developers built on this heritage in designing a new network stack that is optimized for virtual infrastructure from the ground up. As virtualization continues to mature, and users look for innovative ways to implement multitenancy beyond separate operating systems, the low overhead of Solaris Zones, coupled with the virtual networking in Solaris 11, could emerge as a compelling alternative for users seeking to host large numbers of web-centric workloads with the greatest possible efficiency.

What about Linux?

Some may question what Oracle’s renewed focus on Solaris means for its strategy to support Linux, and, given the clear optimization of Solaris 11 for SPARC-based servers, what it says about Oracle’s commitment to x86 servers. In fact, Oracle fully supports Solaris 11 on x86 servers, both from Oracle and other hardware vendors, and the prices and licensing terms for Solaris 11 are no different from those of Solaris 10. Oracle also continues to offer its own Linux distribution, called Oracle Linux, which is replicated from the same code base as Red Hat Enterprise Linux (RHEL). As an option for Oracle Linux, users can install Oracle’s own Linux kernel, called Unbreakable Enterprise Kernel (UEK). With UEK, Oracle is now also bringing some Solaris-developed features to its Oracle Linux customers. Among other things, UEK features:

• Availability functions developed by Ksplice, a company that Oracle acquired earlier this year, which allow code in the UEK kernel to be replaced “in flight,” that is, without rebooting the OS. While Ksplice is presently a feature that is unique to UEK, Oracle is investigating to see if the technology can also be incorporated into a future version of Solaris.
• Forthcoming support for container functionality (similar to Solaris Zones) in Oracle Linux.
• Forthcoming support for Dynamic Tracing (DTrace) capabilities in Oracle Linux, which will give developers a powerful tool to isolate performance bottlenecks at the kernel level. DTrace had long been a feature exclusively available on Solaris.

Oracle thus appears to be making an effort to balance the capabilities of SPARC servers running Solaris with those of x86 servers running either Solaris or Oracle Linux. However, this effort is not just a hedge against the future success or failure of either platform. In fact, each of these platforms is relevant to very different deployment scenarios, involving separate classes of users. On x86, both Solaris and Oracle Linux are suitable alternatives for users who are pursuing strategies based on commodity-style systems and who want to exploit the volume economics of x86-based servers as much as possible. These users generally install hardware and software separately, using industry-standard building blocks at both levels. While Oracle Linux will be preferable for many who prefer to install Linux on commodity systems, Oracle can also offer Solaris on x86 to those want to maintain operational consistency between their x86 and SPARC platforms. Both operating systems can be presented to customers opportunistically, offering them an OS that can inserted under Oracle software with the greatest ease of use and confidence in Oracle’s support, allowing Oracle to take control over the bare metal of x86 servers whenever possible. However, Solaris for x86 and Oracle Linux are just two of many possible levers that Oracle can exercise to help customers deploy its software on x86 platforms, and they are not instrumental to the success of its solution there.

In contrast, on SPARC servers, Solaris is a critical enabler for Oracle’s ability to match the value proposition of classic mainframe-style systems by providing:

• The ability to scale-up hardware that combines large amounts of shared memory and large numbers of processors/cores into a single hardware footprint
• Advanced reliability functions that allow a single server to continue running in the event of hardware or software failure
• Advanced optimizations that enable massive I/O throughput
• A vertically integrated software stack that takes maximum advantage of the hardware features in the platform

The original mainframes proved that this style of systems architecture was optimal for handling huge and variable workloads with the best possible economies of scale. Just as MVS (and now z/OS) represented a critical layer in the IBM mainframe stack, the value of Solaris becomes greatly magnified when deeply integrated with the layers above (Oracle database and application software) and below (SPARC server hardware).

The IDEAS Bottom Line

The assumption that Solaris will be installed as the OS, and SPARC servers will installed as the natural hardware platform for Oracle software, gives developers tremendous flexibility for deciding at which layer certain problems can best be addressed: in the database, in cluster software, in the OS, or in the hardware itself. Long before its acquisition by Oracle, Sun made a commitment to drive mainframe-grade reliability and performance functions into Solaris to complement the mainframe-like capabilities in its high-end server hardware. With Solaris 11, Oracle has followed this development direction to its next logical step. By integrating not only with Oracle hardware, but also key functions from Oracle’s database platform, Solaris becomes the centerpiece of a complete system that can be tuned and operated as a unit in order to deliver the highest possible levels of scalability and reliability for a particular workload.

A breakdown of the latest IDC and Gartner server numbers for the quarter. Two things of note - IBM continues its dominance of the UNIX space, growing its worldwide marketshare by 5% in the quarter; and stripped down “cookie-sheet” servers, modeled on the bare-bones servers that Google uses in their datacenters, are becoming a sizable part of the server business.

The cookie-sheet servers created by Google for its own use – recently commercialized by all the top-tier vendors in one form or another as hybrid rack-blade boxes – have become a sizeable and important part of the server business.

In the third quarter ended in September, the box counters at IDC reckon that end users snapped up 2.07 million machines from server makers and their channel partners, an increase of 8.7 per cent compared to the year-ago period. Revenues increased a more modest 4.2 per cent, to $12.74bn. The market is cooling a bit thanks to tough compares, and as fellow box counter Gartner pointed out earlier this week, shipment and revenue levels on a global basis have more or less recovered to the levels prevailing ahead of the server crash in the wake of the Great Recession three years ago.

Blade servers, which have been around for a little more than a decade, are in essence racks in miniature with integrated backplanes for linking servers to integrated switching and system management processors in the chassis, seemed poised to become a dominant server architecture based on the hockey-stick uptake of rack servers during the dot-com boom, but blades are a high-end product that is a tougher sell than many server-makers had expected. Still, blades are the best option for many customers, and at $2bn in sales for the quarter (16 per cent of revenues) and just under 280,000 units (13.5 per cent of machines sold) they are an important system option even if they have not knocked out rack servers, as many expected.

Blade servers are full of system management and resiliency features that most supercomputer and hyperscale cloud operators simple won’t pay for. That’s why over the past several years the cookie sheet server – which jams multiple server nodes into a rack chassis on metal trays – has become popular. These nodes are all about low-cost and ripping out any extraneous stuff in a blade box – such as service processors, integrated node management, and switching. The assumption with hyperscale servers is that compute and local storage are all that matter and the application environment itself will provide the resilience. And hence these hyperscale servers, as IDC calls them, have come into their own.

In some cases, such as those of Google and Amazon, the company is building all or some of their own hyperscale machines, and even while Facebook has designed its own servers, it still farms out the manufacturing and thus those servers get counted as commercial boxes in IDC’s numbers.

In the third quarter, the hyperscale server segment accounted for 118,888 shipments and $428.5m in revenues, which is an increase of 8.7 per cent in terms of sales and 4.3 per cent in terms of shipments. So hyperscale machines now account for 3.4 per cent of revenue and 5.7 per cent of shipments and have much lower average selling prices per node.

How much lower? If you do the math across the whole server market in the third quarter, the average server cost $6,149. There were a few thousand mainframes and high-end RISC/Itanium boxes in there to raise the class average, but x86 machines account for the lion’s share of shipments in any quarter these days. (All but about 70,000 machines in this case.) The average rack or tower server cost $6,163, and the average blade server cost $7,151. You can see now why blades have had limited appeal: they offer operational cost advantages, but you pay a premium for the hardware. The average hyperscale server cost a mere $3.604 according to IDC’s data, which shows you why you might want to build resiliency in your software stack instead of on any particular server node.

Dicing and slicing server sales

In addition to casing server sales by form factor, IDC takes a stab at estimating the shipments of servers based on the primary operating system that gets installed on the boxes as well as by price band. The System zEnterprise 196 mainframes announced a little more than a year ago gave Big Blue a big bump in sales, but that refresh cycle is starting to slow; the company only booked $970m in sales of mainframes in the third quarter, according to IDC. Windows server sales also cooled a bit, with revenues up 5.3 per cent to $6.3bn against shipment increases of only 2 per cent. That said, Windows-based machines are by far the dominant server platform in the world in terms of shipments.

Unix machines showed some life, with sales up 1.6 per cent to $2.6bn, thanks mainly to IBM pushing its AIX boxes like crazy. “IBM is really starting to dominate this market,” Jed Scaramella, research manager of enterprise servers at IDC, tells El Reg, adding that IBM accounted for 46 per cent of all Unix revenues in the third quarter and gaining five points of revenue market share.

Hewlett-Packard lost 5 points of share and Oracle lost 1.5 points, and they were in a statistical tie for second place in Unix system sales in the quarter. Linux systems saw a very nice 12.3 per cent revenue jump in Q3, to $2.3bn and now account for 18.6 per cent of total server revenues. If you want to be generous, you could say that the combined Unix and Linux markets – call it Unilinux – experienced a 6.4 per cent revenue bump to $4.9bn.

Various proprietary platforms from Unisys, Fujitsu, Bull, NEC, IBM, and HP (not including IBM System z mainframes) accounted for a mere $570m in sales, down 8.8 per cent over last year’s third quarter.

Server sales were not uniform around the world, as you might expect given the differences in regional economies and the state of IT infrastructure in different regions.

“After nearly two years of steady revenue growth, the server market began to decelerate in Q3 2011 as demand stabilized for many system categories,” explained Matt Eastwood, general manager of enterprise platforms at IDC, in a statement accompanying the server stats. “Asia/Pacific and Japan exhibited strong revenue growth while server demand in EMEA, North America, and Latin America was flat to slightly down year over year. IDC continues to believe that weakening macroeconomic conditions around the world will serve to further moderate demand for new servers in 2012.”

By segment, IDC calculates that volume servers (machines that cost under $25,000) had a 5 per cent increase in revenues as a group in the quarter. The high-end segment, which covers machines that cost more than $250,000, had a 1.1 per cent revenue bump (despite the System z decline and because of improving Unix system sales), and the midrange machines between these two bookends had a 4.7 per cent revenue increase as a group.

If you look at the server business by vendor, IBM and Hewlett-Packard were in a dead heat in Q3 as far as IDC can tell, with IBM and HP both getting $3.79bn in sales. (Technically, IDC believes Big Blue had $3m more in sales, but declares a tie when the difference between the vendors is less than one per cent.) IBM grew 3.5 per cent and HP dropped 3.8 per cent.

Dell was the number three server seller, with $1.93bn in sales, and grew at a pace that was nearly three times as fast as the market at large. Another way of saying that is this: If you take Dell out of the numbers, the other vendors only grew their sales by 2.7 per cent, so Dell accounted for two-thirds of the growth of the overall market. Oracle’s server sales declined by 3.2 per cent to $764m, and Fujitsu shrank four-tenths of a per cent to $605m. Other vendors – helped by supercomputer-makers Silicon Graphics and Cray as well as upstarts Lenovo and Cisco Systems – grew as a group by 22 per cent to $1.86bn.

Despite the doomsday scenarios, UNIX will be around for a very long time. Here’s why.

There has been a lot of ill-considered press coverage about the “death” of UNIX and coverage of the wholesale migration of UNIX workloads to LINUX, some of which (the latter, not the former) I have contributed to. But to set the record straight, the extinction of UNIX is not going to happen in our lifetimes.

While UNIX revenues are not growing at any major clip, it appears as f they have actually had a slight uptick over the past year, probably due to a surge by IBM, and seem to be nicely stuck around the $18 - 20B level annual range. But what is important is the “why”, not the exact dollar figure.

UNIX on proprietary RISC architectures will stay around for several reasons that primarily revolve around their being the only close alternative to mainframes in regards to specific high-end operational characteristics:

• Performance – If you need the biggest single-system SMP OS image, UNIX is still the only realistic commercial alternative other than mainframes.
• Isolated bulletproof partionability – If you want to run workload on dynamically scalable and electrically isolated partitions with the option to move workloads between them while running, then UNIX is your answer.
• Near-ultimate availability – If you are looking for the highest levels of reliability and availability ex mainframes and custom FT systems, UNIX is the answer. It still possesses slight availability advantages, especially if you factor in the more robust online maintenance capabilities of the leading UNIX OS variants.

So given that UNIX is not glamorous but is still a solid presence in the IT landscape, what does the future hold? I expect that the leading vendors will continue to enhance their offerings of both hardware and software, with an emphasis on making several of the operations strengths of UNIX even stronger:

• Improved scalability – With the improvement in underlying hardware and the growing demands of the IT world, I expect to see UNIX vendors concentrate on improving the scalability of the OS scheduling to accommodate massive numbers of threads and cores.
• Better online maintenance and availability – Ina nutshell, reducing the number of scenarios where the system must be rebooted for maintenance, including hardware and software.
• Improved partitioning – Continual improvements in preporetary VM and partitioning capabilities allowing I&O groups to choose between software-based abstractions such as full VMs, containers, and fully isolated hardware partitions.
• Continual improvements in systems management tools – Integrating the virtualization abstractions of the underlying hardware and OS with the management tools will give opportunities for increased differentiation as vendors strive to make their environments both easier to manage and more flexible.

My expectation is that we will see the following general direction from the major system vendors:

• Oracle will introduce new OS capabilities to go with its T4 and future M-series as a follow-on to Solaris 11 Express.
• IBM will continue to enhance AIX, with a high probability of a major new release with future POWER 8 CPU.
• HP will enhance its systems and HP-UX with the introduction of the anticipated Poulson Itanium CPU next year as it attempts to hang on to its installed base, and will invest significant resources to eventually port HP-UX to an x86 platform.

Intel presented the latest Itanium Poulson chip the other day. Not a lot of fanfare - particularly after the embarrassing hp/Oracle dispute - but apparently, it is a nice step forward, according to Forrester.

Intel Raises the Curtain on Poulson

At the Hot Chips conference last week, Intel disclosed additional details about the upcoming Poulson Itanium CPU due for shipment early next year. For Itanium loyalists (essentially committed HP-UX customers) the disclosures are a ray of sunshine among the gloomy news that has been the lot of Itanium devotees recently.

Poulson will bring several significant improvements to Itanium in both performance and reliability. On the performance side, we have significant improvements on several fronts:

• Process – Poulson will be manufactured with the same 32 nm semiconductor process that will (at least for a while) be driving the high-end Xeon processors. This is goodness all around – performance will improve and Intel now can load its latest production lines more efficiently.
• More cores and parallelism – Poulson will be an 8-core processor with a whopping 54 MB of on-chip cache, and Intel has doubled the width of the multi-issue instruction pipeline, from 6 to 12 instructions. Combined with improved hyperthreading, the combination of 2X cores and 2X the total number of potential instructions executed per clock cycle by each core hints at impressive performance gains.
• Architecture and instruction tweaks – Intel has added additional instructions based on analysis of workloads. This kind of tuning of processor architectures seldom results in major gains in performance, but every small increment helps.
• Instruction replay – Beyond performance, Intel has added the ability to re-execute a failed instruction. This is a powerful capability for enhancing reliability, and a first for Intel. Instruction replay allows a failed instruction to be retried without the overhead of refetching all of the data, and is triggered by a number of low-level failures. The attraction of this technology is that it happens at a very low level of the hardware, and is completely hidden from the OS and application software. This feature will add to the already impressive reliability of HP-UX running on Itanium-based systems.

Does Anyone Care?

With the bloody divorce of HP and Oracle, with Intel as an embarrassed spectator, does anyone still care about Itanium? Simple answer – the thousands of Itanium customers who are not running Oracle. While the Oracle breakup will definitely hurt HP’s Itanium business, it will not kill it outright, and HP-UX running on Itanium systems will still remain a highly reliable platform for other users. These users will be rewarded with an Itanium platform that will have improved reliability as well as an impressive boost in performance from today’s offerings. While HP has not made any public statements about systems or availability, there is nothing immediately evident about the current Superdome II architecture that makes it inappropriate for Poulson.

Beyond Poulson the future gets a bit fuzzy. Intel has one visible generation of Itanium beyond Poulson named Kittson. Kittson should appear in 22 nm process in approximately 2014. Beyond that our crystal ball gets a bit fuzzy. Intel is releasing no details, and my opinion is that Kittson and one successive performance bump might be the end of the line for Itanium, which would mean that its evolution may stop on or about 2016, plenty of time for HP and its users to come to terms with HP-UX on an x86 platform based on x86 CPUs that will have gone through approximately three additional product cycles by then.

It appears that there is life for OpenSolaris after Oracle dumped it — thanks to Joyent.

Joyent, a company well-known to RWW readers for its backing of Node.js is announcing a step in a different direction today: the company has ported Linux KVM to its SmartOS offering and is making SmartOS available as a ready to install distribution and putting all the source on GitHub.

SmartOS is a “highly specialized distribution of Illumos.” Illumos, as you might recall, is the fork of the OpenSolaris project that sprang up after Oracle snarfed Sun and decided to shutter OpenSolaris.

Given that Linux has a perfectly good KVM implementation, and Illumos has its Zones, one might wonder why Joyent took the time and engineering energy to port KVM over to SmartOS. Jason Hoffman, Joyent’s chief scientist and founder, says that Joyent is looking to provide hardware virtualization for the Data Intensive Real-Time (DIRTY) environments that the company works with. Thanks to the Solaris/Illumos heritage, SmartOS already had Containers and Zones – container-based virtualization that allowed users to run multiple applications sets on one server isolated from one another.

With KVM on SmartOS, Joyent can now address workloads that require running a full operating system for those customers who need Linux, Windows, or other operating systems to run in full virtualization. And, unlike Linux, SmartOS will also give customers access to Solaris technologies that many users find compelling – like DTrace and ZFS. Hoffman says that, with networking bound or I/O bound workloads, that the company actually can make “very repetitive workloads 10x faster” when running on SmartOS with KVM thanks to ZFS and not running “a filesystem on a filesystem, or networking on networking.”

The company isn’t planning to sell the OS, at least outright. Hoffman says that, like Node.js, Joyent may offer support to companies that are looking for it – but it’s not looking to sell SmartOS subscriptions and so forth.

SmartOS with KVM is immediately available for download, and Joyent is putting up the code under the required licenses on GitHub.

Will customers flock to SmartOS with KVM? A lot of developers and admins love the features that are present in the Solaris kernel, but aren’t necessarily in love with the idea of proprietary Solaris. Having a well-supported variant with KVM and a strong open source community could be compelling.

Here’s another read of that Gabriel Consulting Group Unix study. Much the same information as the previous entry, but a few good quotes from the report.

In an announcement, GCG Principal Analyst Dan Olds said that, although sales have been strong for Linux and Windows servers, Unix servers aren’t going away. “But these Unix systems fulfill a different role in most enterprises; they run mission-critical applications that are vital to the functioning of the business,” Olds says. “Just because the sales of small, fuel-efficient cars are skyrocketing worldwide doesn’t mean that the need for dump trucks has gone away.”

Only 20% of respondents said that they were reducing their reliance on Unix, and according to GCG that number has been shrinking over the past few years, indicating that the replacement of Unix with other operating system is slowing. Most (76%) believe they will still be using Unix systems five years from now.

According to a separate announcement, most respondents that run both Unix and Linux believe that commercial Unix OSes are more available and reliable and have better support than commercial Linux distributions.

The majority of respondents (78%), are using two or more Unix brands and 45% use three or more. One GCG report says that this proportion has changed little in the past five years, indicating that no single company is “winning” at converting companies to a particular brand. “We don‟t see much evidence to support the idea that customers are moving toward commercial Unix standardization,” the report says.

While Linux on Intel systems continue to chip away at the Unix market, it would appear that Unix shops are die-hard Unix fans and are not planning on phasing the operating system out anytime soon.

Unix systems may not be all the rage that they were two decades ago, but in nearly eight out of 10 data centers based on them, their use is either holding steady or increasing.

That’s the assessment of a recent survey of the HP, IBM, and Oracle Unix customer bases by Gabriel Consulting Group, which has just finished up its fifth annual slicing and dicing of Unix customer sentiments.

Unix systems have successfully colonized their neighborhoods in the data centers of the world, and are resisting the onslaught of Windows and Linux on those systems’ relatively inexpensive x64 iron. The Unix colonists are also resisting all of the marketing muscle and money that is dedicated to evicting them.

The GCG survey, which ran in late 2010 and early 2011, solicited input from places where Unix techies like to hang out; 306 shops responded to the full survey and the results were normalized for the HP, IBM, and Oracle installed bases.

If you are a Unix expert, the recession was probably a bit unnerving, as CFOs and CIOs ran around with machetes trying to cut costs anywhere they could, with Unix systems being a target in the same way that proprietary mini and mainframe systems once were – and continue to be.

According to the Unix shops polled by GCG, Unix usage was indeed on the wane in 2007, as the recession in the United States was building up a good head of steam. As you can see from the chart below, some shops slammed on the brakes while others kept on increasing.

By 2008 and 2009, GCG’s data from surveys of Unix customers showed the declines slowing, but the share of customers who were boosting their budgets also slowed. Now, in the 2010-2011 survey, 45 per cent of Unix shops say they are increasing their spending, and a third say they are holding steady.

While the Unix-system makers have always competed fiercely against each other, each are notoriously difficult to completely dislodge. The tendency to have two, three, or more Unix variants running in the data center has changed only slightly in the past five years, and in this case, the change has been toward more diversity, not less, among Unix shops.

This is all the more striking as the Unix market collapsed down to three vendors, and server consolidation reduced the number of Unix footprints, says Dan Olds, the principal at GCG and a Reg contributor. Only a fifth of the Unix customers surveyed by GCG say they have only one Unix variant running in their shops, and a third say that have two different Unixes and nearly half have three or more.

The simple fact is that companies choose a particular Unix to run specific jobs based on the technical characteristics of the platforms and the skills their employees have – the same reasons that all applications end up on a particular OS. Or don’t.

Despite the similarities between Unix and various commercial-grade Linuxes (meaning they have third-party tech support from reputable companies), Linux has not become a replacement for Unix.

“Enterprise customers who have both commercial Unix and Linux see value in both systems,” Olds explains. “While they’re increasingly relying on Linux, they don’t see it as a complete substitute for commercial Unix systems at this point. They see commercial Unix platforms as a bit better choice for mission-critical workloads, particularly those that require vertical scalability combined with high availability.

“These are also customers who see a lot of value in vendor support,” he says, “and on this score they believe that the commercial Unix vendors have more to offer than what they can get with Linux.”

About 60 per cent of the shops surveyed that have both Unix and Linux workloads say Unix is the better choice for hosting some applications and databases, and about the same proportion of shops that have both in their data centers say Linux is not as technically sophisticated as their Unixes. A little more than half of those polled who have both Linux and Unix running in their data centers said Unix is more reliable and has better availability than Linux.

Of those shops polled by GCG, 76 per cent said they are planning to have Unix systems installed five or more years from now, and only 5 per cent indicated that they would not. As a control question, CGC asked Unix shops later in its poll if they had plans to migrate away from Unix, and 65 per cent said they had no such plans. However, 17 per cent said they did have plans to do so and another 17 per cent said they were not sure.

So depending on how you ask the question, you get a slightly different answer. Perhaps the situation is more like this: There are some shops that want to migrate away from Unix for economic, skills, or technical reasons, but at least some of them don’t expect to actually do the migration.

Not surprisingly, the shops from which CGC solicited input deploy mission-critical applications on their Unix systems:

A little more than a quarter of the Unix shops polled by CGC said nearly all of their important applications were on Unix, and another quarter said the bulk of their apps were on Unix. If you do a weighted average of the distribution shown above, about two-third of the mission critical applications installed at shops that like Unix are on Unix. Again, this stands to reason.

What would be interesting to learn now, of course, is how shops that have Unix, Linux, Windows, and proprietary minis or mainframes running their applications think these different platforms stack up.

A good, honest look at the Unix market and the continuing viability of RISC platforms like POWER.

The proliferation of powerful and less costly x64-based systems that can run Solaris, Linux or Windows is making more than a few Unix shops think the unthinkable: migrating away from Unix for their mission-critical workloads.

The dot-com crash was the last hoorah for Unix systems. Unix machines accounted for nearly half of worldwide server revenues, shipments ran at about 75,000 units per quarter and revenues were about $2.5bn per quarter.

Hit for six

Shipments and revenues took a hit in the recession that ran from 2001 through 2003, and Unix also vendors suffered in the slump of 2008-9.

Unix now represents about 20 per cent of worldwide server sales. In the first quarter of 2011, according to statistics from Gartner, Risc/Itanium server sales where Unix is the primary operating system started heading back towards 50,000 units and revenues hit $2.6bn. The Unix market has rebounded to where it was after the dot-com crash.

Meanwhile, the Windows and Linux markets have surged, driving about two and a half times as much revenue and accounting for the bulk of the two million or so servers that get shipped in a quarter.

And even as Unix has recovered, the market is adopting Windows and Linux at a feverish pace, predominantly on Xeon and Opteron server platforms.

The same economic and technical pressure that Unix put on proprietary systems is not letting up on Itanium and Risc servers. Organisations look at their data centres and see the big cheques going out to IBM, Oracle and HP, the three remaining commercial Unix suppliers, and start asking a lot of questions about the alternatives.

At the start of the Unix revolution, the processors and system components in Unix machines were leaps and bounds ahead of what x86 server makers could deliver in terms of capacity and reliability. A Unix system was a safe bet.

So safe, in fact, that when the dotcom bubble started to inflate in the late 1990s a Sun Microsystems Sparc server and an Oracle database were the default platforms for Web 1.0 startups – a fact that made Sun and Oracle stinking rich and IBM and HP envious…

Salute the kernel

Perhaps more importantly, Windows and Linux, the operating systems favored on x64 platforms in the data centre, have also improved greatly over the past decade in terms of reliability and scalability.

The kernels have been tweaked to support SMP and Numa scaling, and have real-time options for applications where low-latency is the primary desirable characteristic (think hedge fund trading systems).

These latest Windows and Linux platforms also sport virtualisation hypervisors that can stand toe-to-toe with virtualisation technologies created more than a decade ago for Unix systems.

The evolution of x64 hardware and the continuing improvements in Windows and Linux mean Unix shops can contemplate moving off their Risc/Itanium iron. But the higher prices for hardware, software and support on Unix platforms is what actually makes a certain percentage of them actually go through the process.

Quick calculator

Over the past two decades, I have developed a rule of thumb for the back-office, transaction processing workloads that mainframes, proprietary minicomputer and Unix machines have tended to run.

Here’s the rule: for any given workload, if a Windows or Linux stack running on an x64 server costs a certain amount, then to drive the same workload on a Risc/Itanium Unix machine will cost roughly twice as much, and a proprietary mid-range or mainframe box with the same capacity will cost twice as much as the Unix alternative.

The relative prices depend on workloads: mainframes do better on batch serial workloads for which they have been tuned, and that is why they drive about $4bn in revenues today.

And Risc/Itanium machines have offered memory, CPU and I/O bandwidth that was not available in x64-based systems, and that is why they continue to drive another $15bn in revenues worldwide.

Not only is the hardware more expensive per unit of capacity, but so is the software. Take Oracle’s processor core factoring scheme for its database and middleware software, for example. If you go with per-processor core pricing, you count up the cores and then multiply by a scaling factor to come up with the price.

On Oracle’s own Sparc T3 processors, the scaling factor is 0.25 per core, which means you get a software licence at a quarter of list price; software support for databases and middleware is 20 per cent of that price per year on top of that…

If you think it should cost the same to run Oracle software on any machine, you are not alone. But IBM would not agree with you.

Risc assessment

IBM has its own processor value unit software pricing scheme which is used for on-premise machines as well as for Amazon’s EC2 and IBM’s own SmartCloud clouds.

IBM’s pricing scheme gives 50 per cent price break on Opteron and early Xeon processors, and a 30 per cent price break on newer Xeon 5600 and 7500 processors.

However, the fatter Xeon 7500 and E7 processors are priced the same as IBM’s own Power6 and Power7 processors, as well as Oracle/Fujitsu UltraSparc and Sparc64 processors and Intel’s Itanium chips.

And in many cases, Power6, Power7 and Xeon 7500 and E7 processors have software costing the same per core as IBM’s System z machines, which is 20 per cent over the standard price.

Why do mainframe and Unix vendors charge more for their systems? Because they can. Companies that have coded a trillion line of mainframe code are not about to save a few million or even tens of millions of dollars on a migration to Unix, Windows or Linux systems and incur billions of dollars of risk.

Organisations that have Unix skills are similarly unwilling to move to a new server architecture and operating system at the same time (although if they are using packaged software and migrating to a new version, this kind of transition can be done less painfully than actually porting home-grown applications from a Unix box to a Windows or Linux system).

Staying power

The other factor that helps Unix systems persist in the data centre is the competition from the big three system vendors – IBM, Oracle and HP – and between the two big database and middleware providers, Oracle and IBM.

Those comparisons outlined above are vendor list prices and where two or more vendors are brought in to compete, they can drop significantly. Sometimes a discounted Unix system can come down to the same price as an x64-based system of equivalent performance and capacity.

An x64-vendor pushing Windows or Linux alternatives has to push even lower to win the deal, and there just isn’t as much room for price cutting. The competition in the Unix space makes it a healthier market, much as the mainframe racket was when Amdahl and Hitachi were grinding away at Big Blue in the 80s and early 90s.

The secret is to get a collection of Intel, AMD, Oracle, IBM and HP coffee cups and make sure vendors see that theirs is missing when they come a-calling.

And the most important thing is to move your workloads off Unix systems only when, and if, it makes sense for your company, not for the vendor pushing x64 alternatives.

It is always easier to start software projects on a new platform than to try to move an older system to a new platform. If you want to save money and grief, this is probably the best way to do it.

The easiest thing to do, however, is compute on the platforms you know how to run best. More than any sticker on a shiny new server, that will determine your real costs over the long haul.

The Gartner numbers are out now. The Register provides a great piece on how they differ slightly from IDC’s numbers earlier in the week. Apparently, the Gartner numbers show a slightly larger increase for Oracle.

In the quarter ended in March, Gartner reckons that revenues from servers sold both directly by vendors and through their channel partners accounted for $12.7bn – that’s an increase of a healthy 17.3 per cent compared to the year-ago period. Unit shipment were – obviously – up as well, rising by 8.5 per cent to over 2.3 million boxes.

These numbers are a bit higher than what IDC calculated, since IDC bases its figures on vendor factory revenues, and not the server price after the reseller mark-up. The numbers sometimes get out of phase, as was the case in Q1, depending on how vendors are stuffing their channels – or not – and what kind of demand there is from end-user companies.

The most interesting bit of contrast in the two sets of numbers is that Gartner’s model shows Oracle growing a lot faster than IDC’s model. IDC said that Oracle’s system sales rose by 13.6 per cent, slightly ahead of the overall market, to $773m. Gartner thinks that Oracle’s revenues in Q1 grew by a much more impressive 33.6 per cent, to $798.6m – and that was against a shipment decline of 13.5 per cent to 36,795 boxes.

It appears that Oracle has gotten some traction with its Exadata and Exalogic appliances, and probably with upgraded Sparc machines in the T and M series, as well.

While Oracle’s growth is impressive, it bears pointing out that Oracle is growing from some easy compares from the year-ago period. Also, three years ago – before the Great Recession kicked in – then-independent Sun Microsystems, from whence Oracle’s server biz came, had a sales decline of seven-tenths of a per cent in Gartner’s server model to $1.32bn.

Sun’s server business doesn’t have anywhere near the heft that it used to, if that matters. To its credit, Oracle has pared down Sun’s unprofitable server efforts and made Sun something it should have been all along: profitable and growing. Sun’s decline is no less dramatic than the proprietary mainframe and minicomputer implosion that nearly killed IBM in the early 1990s, or the hammering that HP took in the Unix and proprietary server space the wake of the Compaq merger in the early 2000s…

With IBM’s Power Systems lineup refreshed late last summer, and its System z mainframes as well, IBM ran out of excuses – and thanks to rebounding sales of these machines, it was able to increase revenues in the first quarter by 23.3 per cent to $3.76bn.

How long growth will hold up for Big Blue is unclear. It’s reasonable to guess that the server rebound at IBM will start losing steam, probably in the third quarter and certainly by the fourth quarter, because of tough compares. You can sell only so many mainframes and big AIX boxes in North America and Europe, and Asia can’t carry the world. If the Western economies suddenly start doing better, that’s a different story entirely.

While IBM grew more than HP, according to Gartner’s reckoning, HP still managed to increase revenues by 12.9 per cent to $3.83bn, giving it the top spot. HP’s shipment growth in the first quarter was not great, rising only 2.3 percent to 687,500 boxes. Dell was the number-two shipper, with 508,650 boxes going into customer accounts, down four-tenths of a per cent, but it increased its revenues to $1.89bn, up 13.1 per cent.

IBM shipped 272,238 machines in Q1, up a mere 1.6 per cent, and Fujitsu pushed 76,648 machines, down 1.2 per cent.

So what’s going on here? Cisco Systems and a slew of whitebox vendors seem to be eating into the business of the big boys. Gartner reckons that Cisco’s server revenues nearly quintupled to $194.4m in Q1. Cisco sells only x64-based machines, and managed to get a 2.3 per cent slice of the x64 server pie and a number-five ranking – beating out Oracle and NEC…

Gartner says the Unix collective enjoyed a 20.7 per cent revenue pop, up to $2.6bn, in the first quarter against shipments of a mere 48,504 machines, up 5.2 per cent from a year ago. IBM is the clear leader in Unix server revenues, and its Q1 revenues rose by 25.6 per cent to $1.19bn. HP ranked number two, with Unix server sales up 6.7 per cent to $639.3m.

However, if HP doesn’t get the stick out soon, Oracle will be putting out press releases saying that it is now the number-two Unix vendor and is gunning for number one. Oracle’s Unix server sales rose by 34.4 per cent to $610.8m, says Gartner. Interestingly, Oracle shipped 19,097 Unix machines, compared to IBM’s 18,548. HP only pushed out 8,527 Unix boxes. Fujitsu peddled 2,105 machines and raked in $126.2m.

Gabriel Consulting Group surveyed more than 450 IT decision-makers to find out what large enterprise users believe is happening between Oracle, Intel, and HP on the Itanium front.

So, what is going on with Oracle? The end users surveyed by Gabriel don’t think Oracle is pulling the plug on software development for Itanium processors because the company wants to reduce the number of platforms it supports. Only 34 per cent of those polled agreed that this was the reason, and another 15 per cent strongly agreed.

A slightly larger share of survey respondents agreed that Oracle would use the supposed sunsetting of Itanium as a justification to raise license and support costs for Itanium platforms. This is not news. Back in December 2010, Oracle dropped its per-core pricing for its software on its own Sparc servers while at the same time doubling the price of software running on Itanium processors.

What the IT shops polled by Gabriel seem to believe is going on is exactly what most of us think is going on: Oracle is trying to undermine confidence in competitors’ platforms while shoring up its own relatively newly acquired systems business.

Of those polled, 44 per cent agreed and another 33 per cent strongly agreed that Oracle’s actions were a competitive move to kill HP’s HP-UX and NonStop platforms. And 37 per cent agreed and another 42 per cent strongly agreed that Oracle’s actions in March were but the first step in an Oracle plan that would put all competitive server platforms at a disadvantage to Oracle’s own systems.

“If these customers are right in what they’re expecting from Oracle, then we’re going to see a resumption of the server wars like we had back in the 1990s,” says Dan Olds, principal researcher at Gabriel and someone familiar to El Reg readers in the HPC area. “It’s a big gamble on Oracle’s part, and if it plays its hand too strongly, it will lock Oracle into a battle royale with the rest of the industry.”