When I first wrote this blog I didn’t realise a few things. First that it would become the reference location on the internet for HANA material, and Second that it would therefore require updating!
So here we are, some 9 months after the initial post, with a full revision of this information. Hardware Vendors – this is based on the publicly certified systems on SAP’s Product Availability Matrix. SAP update this regularly with new appliances and vendors.
SAP released the first version of their in-memory platform, SAP HANA 1.0 SP02, to the market on June 21st 2011. We are now at the release of SAP HANA 1.0 SP04 in May 2012 and things have moved on hugely. We now have High Availability, system monitoring and scale-out appliances: up to 16TB certified and 100TB in the lab.
SAP has now an open hardware platform, allowing multiple hardware vendors (currently 7), so that customers could choose who they want to procure from. This should theoretically produce a level playing field where the price becomes commoditised and customers get choice and value.
This article gets into the detail of what it looks like if you actually want to purchase an appliance and it’s based on my experience of working with the hardware vendors over the last 18 months.
Note that my high-level message is pretty clear: SAP HANA hardware is ready for the masses and stable for databases up to 16TB of HANA (equivalently 80TB of Oracle compressed data).
What is the SAP HANA Technical Architecture?
SAP HANA is pretty simple. It’s based on the following components:
- Server – based on Intel’s Nehalem EX or Westmere EX platforms – X7560 or E7-2870, 4870 or 8870 CPUs, respectively. These are big rack-mount systems that take up to 8 CPUs and 80 cores. It’s commodity hardware that you can buy off the web, but for example the Dell PowerEdge R910 with 1TB RAM is $65k list price on their website. I’ve now removed all the Nehalem hardware from this post because it’s no longer sold.
- RAM – lots of it, and matched to the CPUs. 20 cores allow 256GB RAM, leading to a maximum of 1TB of RAM with current CPUs. Think along the lines of $35k list price for 1TB RAM.
- Fast Log Storage – Sized to 1x RAM and usually the very expensive Fusion-io ioDrive Duo. These are $15-30k a pop for the 320Gb and 640GB drives, respectively. In some configurations, the log and data storage are shared. Fusion-io ioDrive2 is now released though I have yet to see certified hardware using it. It is half the price of the ioDrive for the same capacity and much faster too.
- Data Storage – 4x RAM. On all the certified single-node configurations this is cheap SAS direct storage. You need this so you can power down the appliance and do things like backups. Budget $15-20k for at 1TB storage system. For multi-node configurations it uses some form of shared storage – either a SAN or local storage replicated using IBM’s GPFS filesystem. Prices vary for scale-out.
So theoretically at least, you should be looking at $145-150k for a basic 1TB appliance, based on Dell’s website list prices. Note that this is hardware only – and all of the SAP HANA hardware partners offer a pre-built system with installation services and likely require a support contract. It may add up!
The other big difference since I first wrote this blog is that we now get scale-out appliances from Cisco, Fujitsu, HP and IBM from 1TB to 16TB. And in the lab, SAP have a 200-node 100TB test system which means about 1PB of uncompressed data. Things have moved on!
In addition, SAP have invested in a company called Violin, which uses Infiniband and SSD storage. This would be an awesome way to get compact scale-out HANA appliances when Intel’s Ivy Bridge server platform which enables 1TB HANA Blades to arrive.
IBM – remains the safe choice
There’s an adage: “no-one ever got fired for buying IBM”. I’m sure someone has been, but it’s good marketing. IBM sold by far the most of the last generation in-memory appliance from SAP called BIA/BWA and they currently have the greatest choice of SAP HANA appliances.
I have a total of 9 appliances on my list from 128GB to 16TB with various different configurations depending on the customer requirements. At the time of writing they are also the only vendor to scale out to 16TB – at least for now, IBM remain ahead of the game.
My colleagues also tell me that IBM promise any hardware configuration to be available in as little as 5 days. Certainly they provided a 200-node test system for SAP in less than 2 weeks!
That said – IBM isn’t the cheapest of the vendors and there are some hidden costs like the licence cost of their high-performance GPFS filesystem. But as one CIO told me “We priced up SAP HANA appliances and the vendors seemed very varied in price. But as we got close to negotiations, the variance evaporated”.
HP – Solid Hardware, availability concerns?
Since Meg Whitman took the help at HP, things seem to have gotten better (though they could hardly have gotten worse!) and they have consolidated their hardware.
HP now have 5 certified appliances from 128GB to 1TB; they also have a scale-out appliance with up to 8TB. The amount of disk hardware required is concerning – 12 disks per node or 192 disks for a 16-node 8TB system.
The concerns I have heard with HP is that they are now very strict on loan hardware and have extremely long lead times. With Hitachi and IBM being aggressive on delivery schedules, this could put them at a disadvantage.
Dell – Services Provider or Channel Partner?
Dell now have 3 certified single-node appliances from 128GB to 512GB. I heard rumours that they have lost interest in SAP HANA and their services page says “ERP in the Cloud”. Certainly I tried to buy a SAP HANA appliance from them and summarily failed. They said:
With regards to SAP HANA, after a significant amount of research throughout Dell, I must advise that we are not in a position to supply these solutions at this time. While we at Dell strive to offer complete solutions to our customers, we will only do so when we have the capability to do so effectively.
In any case I haven’t seen Dell in any customers. Has anyone seen a Dell SAP HANA appliance in the wild? Let me know!
Fujitsu – Dark Horse
I’ve not had too many dealings with Fujitsu but they have been quick to respond and appear to know what they are doing from a sales-enablement perspective. They have the same 5 appliances as HP and 128Gb to 1TB appliance sizes – just the same.
They also have the same scale-out as HP with the same enormous amount of disks, up to 192 for a 16-node system using either NetApp or Eternus disk fabric.
Cisco – IKEA of HANA appliances?
Cisco have expanded their portfolio and I hear their blade business based on the UCS C260 and C460 blades is doing well in the enterprise. They now have 128, 256 and 512GB appliances as well as a 16-node 512GB (8TB) scale-out appliance like HP and IBM.
Their appliance requires even more disks! Up to 300 for the 16-node system. Wow!
Cisco guarantee fast delivery but your HANA appliance will arrive as a bag of parts that need assembling by a Cisco Services Partner. And then shipping to you.
Hitachi Data Systems
Hitachi used to call their appliance the Blade System 2000 or BS2000 for short. Thankfully they had the common sense to rename it the Compute Blade 2000 and it is available as a blade chassis from 256GB to 1TB, using their AMS2000 shared storage.
Theoretically this should allow them to build out a neat scale-out solution using their HDS storage arrays and 2TB per blade chassis but this has not been released yet.
One thing that is worth noting with Hitachi is that they have SAP HANA hardware on the shelf in standard configurations and have a promised ship time of 2-3 weeks.
NEC – New kid in town
NEC have arrived with a single appliance – a 1TB system using Virident SSD instead of the Fusion-IO. It is bigger than all the other vendors at a massive 7U which can accept 2TB of RAM (which has no value for HANA). I’m guessing NEC have plans to certify more hardware but I have not seen one in the wild.
Conclusions
As I predicted, the hardware market will increase in volume and consolidate and other vendors have indeed come on board. This will continue through 2012.
Scale-out: Scale-out is now a reality and there are systems running IBM’s X5 platform up to 200x512GB nodes or 100TB. The concern I have is that without IBM’s proprietary GPFS technology, a lot of shared storage is required to make HANA work. Can HP and others prove large scale-out capability?
Blades: Let’s face it – SAP HANA was meant to run on blades. But there’s no suitable blade platform yet because you can’t get 8 CPUs and 2TB RAM in a single blade. Plus, you are even more limited from an expansion (i.e. Fusion-io cards) and network bandwidth perspective, if you use a blade chassis. It now looks like when Intel’s Ivy Bridge platform arrives in late 2012, the hardware vendors will have designed high-density systems to run SAP HANA.
But to conclude, the SAP HANA hardware business has come a long way in the last 9 months. If it continues to scale at this rate, Teradata had better be concerned.
You missed Hitachi Data Systems – newly certified AND runs on blade servers.
Didn’t so much miss it as it wasn’t available at the time of writing. You say that HDS runs on blades, which is true, but they are babies – 256GB blades, from what I understand. It’s not clear to me how Hitachi can scale to above 1TB in a single configuration, on that basis, which makes it too small for most of the customers I am talking to. What are your thoughts?
HDS uses Symmetric Multi Processing to physically connect the QPI’s of up to 4 blades together via a front panel connector to aggregate the resources of up to 4 blades together into a single instance. Each of these blades are rated to support 384GB each, for a maximum total of 1.5TB in a single instance. In the case of the Large HANA Appliance, HDS uses 256GB of memory per blade, as well as a pair of E7-8870′s on each blade. providing a total of 80 cores and 1TB of memory in a 4 Blade SMP. This consumes half of a 10U chassis. HDS also is the only vendor to offer a seamless upgrade path from S to M to L Hana Appliances. Scale up from 1 blade , to two (add a 2-blade SMP connector), to four (remove 2 blade connector, add 4 blade connector). All without losing your initial investment, or having to reload the OS or the re-install the software stack.
I get that, of course you can’t use more than 25G6B per blade in HANA because you need 20 cores per 256GB RAM, so 1TB max and not 1.5TB.
What happens if you need more than 1TB? Also, how do you scale the log volume? Presumably whilst you don’t have to wipe the OS, you do have to move the logs off, reformat the log volume and copy the logs back.
What is the recommended hardware for test and development purpose. Who is the cheapest vendor for such hardware?
Great question. You still need certified hardware, at least today. You should buy this from whoever is supplying your production equipment. Most of the vendors have a 128GB appliance.
In most cases I’m involved with the cost of T&D equipment isn’t really relevant because most organisations sit in one of three camps:
1) The technical people want to do a POC. Great, if you have discretionary budget to buy 1 HANA unit from SAP and a box to run it on.
2) There is a clear cut business case or value case and you should go straight ahead. Especially true for e.g. COPA RDS. Organisations are buying this on value alone.
3) You have a potentially clear cut business case that requires a POC, proof points and has exit criteria – in this case a hardware vendor will typically loan hardware. Mostly this only works for large organisations that are doing something slightly unusual.
To answer your question directly – all the hardware vendors are roughly the same cost. Cheapest systems I’m seeing come in at about $80k.
There were a few “misses” in this blog. In particular, the obvious underestimation of Cisco as a Server player, HANA or otherwise.
Why do you think that Cisco is a serious server player? There’s IBM (36% market share), HP (30%), Dell (13%), Oracle (5.5%), Fujitsu (4%). It’s true that Cisco has some decent market share in blade servers (10% or thereabouts, which equates to about 2% of overall server market), but most HANA systems being sold today aren’t blades, for two reasons.
First, there aren’t blades that can take the memory and CPU density that HANA requires for the high-end systems – density and power problems are getting in the way. Second, blades require scale-out and manufacturers are having problems in the real world with I/O bandwidth to shared storage.
No doubt with another technology generation these problems will be architected out, but I don’t think that makes Cisco such a serious player today.
That is okay, John. I rather like the role of underdog.
I think it is important to look not only at the market-share numbers as a specific point-in-time, but also the trajectory. It’s also important to look at the big picture going from zero customers to over 10,000 in a bit less than 3 years. I encourage this because looking at it any other way is really looking at it through the eyes of “marketing.” It’s also important to compare apples-to-apples. When you look at the overall market-share, you’re including products that not all of the players make — Dell and Cisco, for example, do not make large RISC-based servers — and the trend (remember — trajectory is just as important) has been to move from large, expensive, RISC-style compute to commodity x86-64 based compute.
That last statement is especially important when it comes to HANA. Today, and the public-facing statements from SAP is that HANA is and will only ever be an Intel architecture appliance. Without prejudice, Oracle recognized this a long time ago as well. Scale-out, on industry standard, open technology can handle the vast majority of workloads required today. You are absolutely correct about some of the problems of Scale-Out HANA, but those problems aren’t too difficult to solve. The real challenge for HANA will be the 3rd-party developers who choose HANA as a real-time back-end for business analysis and, I personally think, mobility solutions.
The earlier point about Cisco, and more specifically market-share as a point-in-time data-point, vs a trajectory point of view — If we were merely a niche player, or we “didn’t get it”, whatever “it” might be — the trajectory that we’re on couldn’t happen. If you look at other solutions, it’s not hard to draw the conclusion that blades (and the points you make about power, cooling, and equally important – management) from other vendors were “fixed by patch”, meaning — as time progressed, blade chassis, power, networking were all bolted together using old-school technology — just shrunk down to fit into a smaller package. Cisco re-thought the entire stack and I refer to this as “fixed by Design.” As customer learn more about Cisco UCS, that trajectory that we have been on will continue.
Lots of interesting points.
I think many people are interested in what’s available now rather than what the overall market trajectory is, because people want to buy today. In today’s terms, it strikes me that IBM have the simplest and strongest hardware proposition for HANA, partially because of their experience with BWA, with SAP Services and partially because their GPFS filesystem resolves a bunch of real-world problems for customers that want HA, DR and reliable operations. In the UK, IBM are great because they have a knowledgeable reseller, Centiq. And remember, no one ever got fired for buying IBM! HP and Fujitsu are not too far behind, depending on the region.
In terms of trajectory, it’s much trickier. Partially because there are macro-trends like HP’s current trend towards implosion. Plus a volatile market that can be deeply affected by things like volcanos and tsunamis in the far-east. And partially for HANA because most customers buy a single hardware vendor every 3-5 years and rarely move. I don’t have a single SAP customer that runs Cisco hardware and I deal with many of the Fortune 500. Given Cisco’s market penetration, there must be some large customers using the kit; I just haven’t encountered them.
HANA has been built to be x64-only and it is highly optimised. I don’t see that changing in the short-medium term, although long-term CPU trends mean SAP can’t bind itself to Intel. This means it is a theoretically level playing field. In reality it’s not though, and I’m seeing 2-3x performance differences between different HANA systems from different vendors with the same baseline specifications.
Cisco currently still only have one hardware configuration certified: UCS C460 M2 with 512GB RAM. Also there is the B-Series blades which should be able to support 512GB nodes; it will be interesting to see who Cisco decides to partner with on scale-out. Will it be NetApp, like HP and Fujitsu?
It strikes me that IBM and HP are both a generation of hardware away from HANA blades, but that’s not a long time in software terms and they can easily catch up to what Cisco have as a possibly slightly superior architecture. Do you really think it is so much beyond what IBM have, particularly considering their massive R&D budget?
Either way, it’s interesting times!
Great Article,
Do you know of any benchmarking status that are available to highlight the pro and cons in the selection process?
I’m afraid I do not. There is no official HANA benchmark but I have used TPC-H to compare different vendors. I can say that IBM is, for some reason, faster than other vendors. I hear they found some hidden tuning that the other vendors have not applied…
Thanks for sharing Valuable information and insight of HANA Hardware Vendors !!!
As John mentioned, CISCO has limited presence with SAP Customers when compared to IBM/HP.
However, I have heard CISCO is partnering with Large IT(SAP) service provide companies to work on setting server farms for SAP cloud services.
Generally, SAP HANA will be used only by large Enterprises. (At least in initial days ).
If CISCO wants to penetrate these accounts (assuming majority of these accounts will be using IBM/HP/Dell etc),CISCO will have to come out with better technology soon.
I have seen some penetration now of their Blade chassis in Large Enterprise is good and I suspect they plan to sell the leverage of consolidation to architects.
They are however well behind the game in scale-out appliances, which are now becoming essential as HANA goes mainstream.
Interesting, no one mentioned about reliability of the RAM… with so many TB of RAM, memory failure will cause great impact to the system…. Is mirrored RAM supported? Is Intel’s SDDC/DDDC enabled? What about memory hotplug? Surely not all vendors support all these features right?
@Ah Beng;
Yes, you are exactly right and this is an aspect that almost everyone seems to overlook!
The probability of a widget failing increases with the number of widgets you have in the system. With so many little bits of DRAM silicon making up these huge DRAM subsystems, reliability in terms of MTBF, and even worse, MTTDL goes up exponentially.
SAP and Intel have worked this out though. The Intel E7 (Westmere EX) platform has all the RAS capabilities needed to make Big Memory reliable, but as far as I know only IBM supports the entire E7 RAS feature set, including DDDC, while also implementint PFA (Predictive Failure Alerts) in every major subsystem.
In case you were wondering, this is why SAP is only certified to run on Nehalem EX and Westmere EX.
Sorry this took a while to reply to – there was a NDA in place around the EP platform. Since it has been released I can reply.
There are 20 available RAS features, of which the EX platform has them all and the EP platform has 15. These mostly relate to the self-healing functions like the MCA Recovery.
Specifically PFA and Chipkill are also available on EP. So for my money this is to some extent Intel marketing BS – claiming that only the EX is enterprise ready. Having a RAM failure on an application is just as bad…
Fujitsu does have a multi-node HANA system. You can check it out here: http://scn.sap.com/community/in-memory-business-data-management/blog/2012/05/04/get-high-availability-with-bw-on-hana-scale-out-solution
Thanks, it came after my post was written. I’ve updated the post accordingly.
John – Great blog!
1. Can all the memory in the machine be used for HANA?
2. Does SAP offer any sizing guidelines for HANA?
3. Do you get any significant compression when using HANA’s row storage?
4. What are customers seeing in the field for realistic compression when using HANA’s column storage?
5. Are we all still staying mum on the software pricing of HANA?
THANKS!!!
Good questions as always mate!
1. Yes, it can (and must) all be used for HANA. Though you need space for calculations – usually reckoned to be 50% of the RAM. So only 50% can be used for the database.
2. Yes absolutely, in the usual place: https://service.sap.com/quicksizer – you need to be a SAP customer. Also check SAP Note 1637145 for a BW on HANA sizing calculator.
3. No, and that is the intention because the row store is intended for transient data e.g. queues. You still use the column store for any permanent data store.
4. Varies massively from 3:1 to 20:1 with the average at about 5:1 when compared to an equivalent MSSQL/DB2/Oracle compressed table.
5. There are some details on Steve Lucas’ blog here: https://www.experiencesaphana.com/community/blogs/blog/2012/04/30/what-oracle-wont-tell-you-about-sap-hana – €40k for HANA Edge (64GB) and as low as €13k per 64GB unit for BW on HANA. The full price list isn’t published as far as I know!
hello, great article. do you know if it’s possible to purchase a ‘personal’ HANA system, or lease it from hardware partners for personal development purposes?
thank you,
Joseph Banegas
SAP BI HANA Solutions Architect
Absolutely, there is a SAP HANA Developer Center for this exact reason. Hope it helps.
http://scn.sap.com/community/developer-center/hana