WAN Acceleration equals Big Payoff

By David Newman Framingham | Monday, 20 August, 2007

Imagine walking into the CIO’s office tomorrow and saying, “I can cut our WAN consumption by as much as 80 times, speed file transfers as much as 45 times and make our Windows users a whole lot happier.” Think you’d get the CIO’s attention? Those aren’t just idle claims. Seven months of rigorous testing showed us why application acceleration is such a hot area: these devices really work.

We tested boxes from Blue Coat Systems, Cisco, Riverbed Technology and Silver Peak Systems in a true enterprise context, with a massive test bed pushing data over multiple T-3 and T-1 links. After pounding the systems with the most popular enterprise applications, we’re inclined to believe the hype.

Riverbed’s Steelhead appliances outperformed the field in most tests, and won our Clear Choice award.

But all these devices deserve serious consideration: Blue Coat’s SG appliances for solid HTTP optimization; Cisco’s Wide Area Application System (WAAS) for excellent compression, traffic transparency and interoperability with other devices; and Silver Peak’s NX appliances for strong scalability and intuitive traffic reporting tools.



Why is Windows so bad?

The problem statement for application acceleration is simple: Windows performance in the WAN is lousy. To begin with, Windows’ two workhorse protocols — TCP and NetBIOS — were never intended for use in low-bandwidth or high-delay networks. Windows XP Service Pack 2 compounds these problems with some spectacularly sub-optimal configuration defaults. Windows Vista is better, but it isn’t widely implemented yet.

By default, XP’s TCP stack advertises a receive window — the maximum amount of data allowed in flight without acknowledgment — of 64KB. That’s fine as far as it goes, but XP isn’t very responsive about resizing that window in response to loss or delay. A large, static receive window contributes to retransmissions, possible packet loss and poor response time.

To make matters worse, XP doesn’t use a common TCP option called window scaling that can expand a 64KB receive window by a factor of four or more. Even when network conditions let XP go much faster, it won’t.



Faster file service

As part of our research for this test, we asked vendors and several corporate IT shops to name their top five candidates for application acceleration, and every respondent named Common Internet File System (CIFS) as its top pick. This is understandable, given that Microsoft’s notoriously chatty file-handling protocol was originally intended for LAN-only operations. Given its popularity and performance issues, we made CIFS the highlight of our performance testing.

We tested application acceleration the way enterprises use it — with multiple WAN links and round-trip times. Our test bed modeled a hub-and-spoke WAN linking with a headquarters office plus four remote sites, two apiece on T-1 and T-3 links. The remote sites represented every permutation of high and low bandwidth and delay.

To measure the effects of block and/or file caching, we ran the CIFS tests three times. First was a “cold run” with all caches empty. Second was a “warm run” that repeated the same transfer as the cold run, this time with the files already in cache. Finally, we changed the contents of 10% of the files; this “10% run” forced devices to serve some but not all content from the origin server.

The two most important application-acceleration metrics are bandwidth reduction and response-time improvement. While we measured both in this test, our results show there’s not necessarily a strong correlation between the two.

Looking first at bandwidth-reduction results, all products substantially lightened the WAN load, but big differences exist across devices depending on cache contents.

For example, in the cold run (caches empty), Cisco’s Wide Area Engine (WAE) appliances were by far the most effective at compression, using nearly 28 times less bandwidth than was used in our baseline, no-device test. In contrast, the bandwidth saving for other devices seeing data for the first time was usually less than a two-times reduction in bandwidth, according to measurements taken by a ClearSight Networks’ network analyzer.

Note that we’re presenting all results in terms of relative improvement rather than absolute numbers. For example, in the CIFS cold run, Cisco’s devices consumed 130MB of WAN bandwidth, compared with 3.6GB with no acceleration device inline, which translates into using 27.82 times less bandwidth.

Given that enterprise data patterns are repetitive and subject to change, bandwidth reduction in the warm and 10% test cases can be more meaningful — and this is where these devices really shine.

Riverbed’s Steelhead appliances topped these tests, reducing bandwidth by a factor of 84 in the warm run and a factor of 32 in the 10% run. While the other devices reduced bandwidth by a lesser degree, the improvements were still dramatic. Any device that reduces bandwidth use by 20 or 30 times must be considered a boon to IT budgets.

LAN differences among products were not nearly as dramatic as WAN differences. The Blue Coat and Cisco devices, for instance, reduced LAN bandwidth consumption by factors of 1.5 to two in our warm run and in our 10% run.



Measuring CIFS response time

We used a common enterprise task to gauge CIFS response time, measuring how long it took for a client to upload or download a set of Word files to or from a server. We measured transfer times at each of our four remote sites — each representing a different permutation of high and low bandwidth and delay. We’re presenting the results for each site because users’ requirements differ depending on where they work.

As our results suggest, some appliances do a better job at accelerating CIFS in low-bandwidth settings; others are better for high-delay settings. Arguably, the most important results for enterprises are from the 10% runs, where we offered 10% new content and 90% existing content to each set of appliances. This represents an enterprise where many users might see the same documents repeatedly but where there also would be some new documents added to the mix.

In the download tests, low-bandwidth sites tended to see the biggest improvements in response time, regardless of the amount of delay present.

Riverbed’s Steelhead appliances sped up file transfers 45 times to a low-bandwidth, low-delay site and 34 times to a low-bandwidth, high-delay site. The Steelhead appliances were also tops for the high-bandwidth sites, but to a lesser degree, with speed increases of four to seven times.

The Silver Peak NX appliances were next most efficient overall, with speedups of three to 16 times (again, with the most improvement shown for low-bandwidth sites), followed by the Cisco and Blue Coat appliances.

File uploads generally don’t benefit from application acceleration as much as downloads do. Even so, big improvements in upload performance are still possible. Riverbed’s Steelhead appliance again led the pack, with speed-ups of three to 34 times compared with no acceleration.



Mail call

There was significantly less differentiation among products when accelerating MAPI traffic, compared with CIFS traffic. All products sped mail delivery, but only by factors of 1.24 to 2.39 compared with a no-device baseline. Averaging results across all sites, the Blue Coat devices provided the biggest boost for mail traffic, but by a relatively small margin over the Riverbed, Silver Peak and Cisco devices.

Doubling email performance is nothing to sneeze at, but we also wanted to understand why MAPI performance didn’t match CIFS performance. A few minutes with the ClearSight analyzer gave us the answer: the Outlook 2007 clients we used in this test encrypt email traffic by default.

To the acceleration appliances, most of the MAPI data structures weren’t visible to be optimized. Some acceleration was still possible, through TCP optimizations or because some MAPI traffic was visible. After reviewing the results, Riverbed said it encourages Outlook 2007 users to disable encryption for highest performance. That said, network managers using the new version of Outlook should consider whether the security/performance trade-off is worthwhile.



A faster web

We measured acceleration of HTTP traffic in two tests, one with 248 and one with 2,480 concurrent users. The results were a bit surprising: while the products delivered web traffic as much as seven times faster than a baseline test without acceleration, performance didn’t necessarily improve as we added more users.

To avoid overloading the sites on slower links, we put proportionately fewer users on the T-1 sites than on the T-3 sites. For example, our 2,480-user test involved 1,200 clients at each of two sites on a T-3, and 40 clients at each of two sites on a T-1. We used Spirent Communications’ Avalanche/Reflector tool to emulate web clients and servers. Because previous studies of web objects place the average size at 8KB to 13KB, we configured the clients to request an 11KB object from the servers.

As in the CIFS and MAPI tests, the Riverbed Steelhead appliances delivered web traffic the fastest web acceleration. In all three ways we measured — transactions per second, traffic rates and response time — the Steelhead appliances delivered web traffic seven times faster than tests with no device in-line. We observed the same seven-times improvement with 248 and 2,480 users. Because LAN and WAN bandwidth use was almost identical in each test, it’s likely that WAN bandwidth was the bottleneck.

Blue Coat’s SG appliances were second fastest, but that result must be stated with a caveat: the Blue Coat boxes worked better with fewer web users, not more. Compared with no acceleration, the Blue Coat appliances boosted web performance by around seven times for 248 users, but by around six times for 2,480 users (and that’s just for transactions per second and data rate; the response time improved by only a factor of three).

Silver Peak’s NX appliances were third-fastest, tripling transaction and data rates and reducing response time by around 2.5 times when handling 248 users.



QoS testing

Quality of service testing revealed some of the most interesting — and in some ways most problematic — results of all our performance testing. While three of the four products did a virtually perfect job of prioritizing traffic, the path there was anything but straightforward, involving much tuning — and in some cases external devices to protect key flows during congestion.

To measure QoS capabilities, we offered a small amount of high-priority traffic — in this case, a single VoIP call, which is sensitive to delay and jitter — while walloping the WAN with huge amounts of background traffic. We used User Datagram Protocol UDP for both high- and low-priority flows. VoIP uses UDP by default, and TCP was not suitable as background traffic, because of its built-in congestion control.

Blue Coat’s SG appliances couldn’t participate in this test, because they don’t optimize UDP traffic. The other vendors turned in excellent results but used different paths to get there.

Cisco recommends using WAN routers (in this case, the Cisco 3845 and ISR 2800 Series devices it supplied) rather than application accelerators for shaping traffic. Cisco’s WAAS-acceleration devices and routers work together using network-based application recognition (NBAR). We verified in testing that flows the acceleration devices classified using NBAR will be prioritized by the routers during congestion. The routers turned in great results. The ClearSight analyzer measured R-value, an audio-quality metric, as 92.03 out of a possible 93, and they correctly re-marked DSCPs.

This approach differs from the Riverbed and Silver Peak devices, which performed prioritization on board. Many network managers already run QoS on WAN routers, and for them handing off this function to a router isn’t a big deal. For users just getting started with QoS, it may be simpler to set it up on application-acceleration devices, and leave routers alone, at least for now.

The Riverbed and Silver Peak appliances also protected voice traffic, with R-value scores of 91.80 and 90.07, respectively, and both correctly re-marked DSCPs.



Concurrent connections

Our final performance test determined the maximum number of TCP connections each system could optimize. This is an important metric for enterprises with many remote offices and hub-and-spoke network designs, where connection counts for data-center devices can run into the tens of thousands. All the devices we tested get into that tens-of-thousands range, but there was more than a fourfold difference between the highest and lowest capacities.

Cisco’s new WAE-7371 came out tops in this test, accelerating more than 50,000 TCP connections. Silver Peak’s NX appliances were next, optimizing 43,306 concurrent connections. This is well short of the NX 7500’s rated capacity of 128,000 optimized connections, a level that Silver Peak achieved in internal testing.

We were unable to reproduce that result in our lab, and, despite extensive troubleshooting, neither we nor Silver Peak’s engineers were able to explain the difference. The Blue Coat SG appliances were next, handling about 19,500 optimized connections.

Riverbed’s Steelhead 5520 optimized more than 12,200 connections, but that result reflects the limits of the two Steelhead 3520 units through which we set up connections.

Riverbed says the higher-end 5520 model can optimize 15,000 connections. We were unable to confirm that result, but our tests did show that each 3520 slightly outperformed its rated limit of 6,000 connections to get to the 12,200 total mentioned previously.


— Newman is president of Network Test, an independent test lab in California. He can be reached at dnewman@networktest.com

Managed WAN Acceleration Service - Tredent Data Systems

Tredent Data Systems, Inc., the leader in wide are networking, today announced that they have selected Riverbed's Steelhead platform, a highly scalable WAN acceleration solution, for a new Managed WAN Acceleration Service or Solution as a Service (SaaS). Riverbed's high-performance Steelhead platform enables Tredent's professional services staff to fine-tune network performance to improve application delivery and reduce bottlenecks. Tredent's new Managed WAN Acceleration Service, which relies on Riverbed's Steelhead platform, can help distributed enterprises to accelerate application delivery over the WAN, leverage WAN investments and boost user productivity.

"We chose Riverbed Technology's high-performance Steelhead platform for our Managed WAN Acceleration Service because it enhances application visibility and improves application response time and WAN efficiency," said Charles Beck, vice president of sales at Tredent Data Systems, Inc. "Riverbed is helping us to deliver on our commitment to offer a world-class experience to our business customers. Their platform offers significant scalability which is enabling us to meet the needs of our multinational customers."

The Steelhead platform improves application performance over the WAN by recognizing and eliminating redundant transmissions; accelerating TCP and application-specific protocols; prioritizing and allocating access to bandwidth; and ensuring high application availability at locations with various WAN connections. The Steelhead application acceleration platforms work end-to-end with a multitude of applications, including Web-based (HTTP,HTTPS,FTP,SSL, etc), client-server (CIFS, MAPI, SQL, Oracle, Email), custom software, voice and video. The platforms also include a broad set of management tools for monitoring and reporting on WAN optimization and application acceleration performance.

The new Managed WAN Acceleration Service (SaaS) offering will allow Tredent's customers take advantage of the business enhancements that WAN optimization and application acceleration bring to corporate networks without having the huge capital outlay that normally comes with deploying an enterprise wide WAN acceleration solution. This approach is allowing more businesses to take advantage of WAN optimization technology in their environment while only needing to make low monthly payments during the term of the service. "Most customers are choosing to make the term of the new Managed WAN Acceleration Service (SaaS) offering match that of their current WAN circuit contract so that they can make a decision on new technology or upgrades to existing technology when they are considering the renewal of their circuit contract" said Justin Lofton, senior systems engineer at Tredent Data Systems, Inc.

About Tredent Data Systems, Inc.

Tredent Data Systems, Inc. is a leading provider of systems integration solutions and services to medium and large business customers worldwide. Combining unsurpassed customer service with advanced technology and professional service capabilities, Tredent Data Systems, Inc. delivers innovative and unique business solutions to customers around the world. For more information, visit www.tredent.com.

About Riverbed's Steelhead Appliances

Riverbed's WDS solutions enable organizations with more than one office to overcome a host of severe problems, including poor application performance and insufficient bandwidth at remote sites. By speeding the performance of applications between distributed sites by five to 50 times and in some cases up to 100 times between enterprise data centers and remote offices, Riverbed's award-winning Steelhead WDS appliances enable companies to consolidate IT, improve backup and replication processes to ensure data integrity, and improve staff productivity and collaboration. Steelhead appliances have been deployed in organizations ranging from the world's largest corporations with offices around the globe to small companies with a couple of sites that are just miles apart.

About Riverbed

Riverbed Technology is the performance leader in wide-area data services (WDS) solutions for companies worldwide. By enabling application performance over the wide area network (WAN) that is orders of magnitude faster than what users experience today, Riverbed is changing the way people work, and enabling a distributed workforce that can collaborate as if they were local. Additional information about Riverbed (Nasdaq:RVBD) is available at www.riverbed.com.

Riverbed Technology, Riverbed, Steelhead, RiOS, Interceptor, and the Riverbed logo are trademarks or registered trademarks of Riverbed Technology, Inc. All other trademarks used or mentioned herein belong to their respective owners.