Download accelerators boost transfer speeds, but are they more harm than good?


Download accelerators can sometimes make quick work of file transfers. The programs attempt to squeeze all bandwidth available to make downloads go faster by manipulating how the server sends files. While software such as this may be attractive, Mac users may be inadvertently causing problems for the hosting sites.

Grant Bayley maintains the AusMac Archive, located in Sydney Australia. The site hosts software from around the world intended to be downloaded by Mac users in the Asia/Pacific region. The mirror site has been operating since 1995 and is primarily funded through private donations. AusMac aims to make downloads quicker for users in the region by providing links through his servers that are more local. While Asia/Pacific users may find AusMac a quicker route to get their Mac software, Mac users around the globe also use the site, perhaps many times without even knowing it.

Recently Bayley has been working to eliminate the use of download accelerators at AusMac. According to Bayley, use of this software is detrimental to his service. Bayley has warned that if usage of download accelerators continue, they will force sites like AusMac to clamp down on access, if not go away all together.

Accelerating Downloads

Download accelerators work by splitting a download up over multiple parts. Each part is then transferred simultaneously to be then reassembled at the requesting computer. Downloading a file in multiple parts allows an Internet connection to more efficiently use all available bandwidth. With a typical download, you get one connection to a server, which may not be be using all of the user's maximum bandwidth. Furthermore, each connection could be limited to the maximum amount of bandwidth available.

The potential problems of using download accelerators are twofold: First, by using multiple connections, one user could very well take up all available download connections on an FTP server, effectively shutting out any other users. Second, one user could monopolize all available bandwidth at once for all services. Even if all available bandwidth isn't used, it's still a concern that a single user could take more than his/her share of resources.

"On a server that is limited to 150 simultaneous users, having one user take up a fifth of that capacity is both outrageous and a slap in the face to everyone else that plays by the rules," Bayley told IGM.

The problems from a single user at one time may not make a big impact, however, if many users connect at once, even the beefiest of servers will feel the strain. Developers of download accelerators disagree that problems arise from use of their software. Used correctly and not excessively, most download accelerators could probably keep under the radar of large mirror sites such as AusMac. The developers view their software as tools to best use what network resources are currently available. Furthermore, in theory if one user used two connections instead of one to download a file, the file could be finished much quicker than if only the single connection was used, therefore, in the end leveling things out by not being connected for as long as a time.

"Speed Download is designed to download files more efficiently and offers tons of other features that benefit those who use it," said Steven Mullin, of Yazsoft, developers of Speed Download. "The way [Speed Download] is designed right now, has absolutely no negative impact on any server. We test it rigorously against our own servers and everything still runs solid."

For the most part, software developers are placing the problems back onto the server administrators. Each developer we spoke with stressed that if download sites are experiencing problems with users opening excessively multiple connections, the servers should be configured to limit or all together eliminate this ability.

"It's quite simple for system administrators to limit the number of simultaneous connections each user can make to the server," said Mal Paine, of Ilesa Software, the developers of Download Deputy. "By limiting the user to 1 connection, you've eliminated the download accelerator 'problem' in one fell swoop."

Bayley has since started blocking the simultaneous connections during heavy load times. According to Bayley, the problem, however, doesn't end there. Bayley notes that even if the server is blocking additional connections, problems still persist. If no additional connections are available, currently all download accelerator software continuously attempt to open multiple new connections, even if all attempts are rejected. These requests themselves in large numbers pose their own problem in terms of traffic and CPU usage, essentially resembling a denial of service attack. Bayley has suggested perhaps software should have measures to wait a period of time before retrying a connection after it has been rejected. That, he claims, would be less of an assault on file servers.

Jason Andrade of Planet Mirror, another download mirror site, notes that the problem of multiple connections that do succeed can become quite severe when usage of download accelerators gets out of hand. Andrade notes that some software available on the PC side open hundreds, even thousands, of connections, which can put a large strain on servers in terms of just CPU load. This doesn't appear to be the case, however, with available Mac download accelerators, as most limit the maximum number of simulations connections to more reasonable numbers. Even so, Andrade notes that general usage of download accelerators are not always practical.

"Download accelerators are far too easily prone to misconfiguration and are more likely to end up causing more problems than any benefits to end users," said Andrade. "Mirror admins tune their systems to achieve the best possible service to end users and something like a download accelerator generally upsets that balance."

That balance comes from both a bit of science and art in managing heavy traffic loads. AusMac can reach upwards to 100 GB per day of combined FTP and HTTP traffic, according to Bayley. In general, to manage large traffic, limits can be placed on the numbers of available download slots, maximum total bandwidth and maximum bandwidth per connection. Even with the most conservative calculations, server load issue gets complicated with a wide variety of connecting users from all over the globe using everything from a dial up modem over old ratty copper to Ethernet a couple hops off of fiber. If bandwidth is managed through averages and assumptions, a download accelerator could very well tip the balance. The problem escalates as more and more download accelerators are used at once.

The developers I spoke with stand strongly by their software. They firmly believe their software is not damaging, and realistically they not, however as more people use the software and more abuse their abilities, problems such as at AusMac may appear to surface . The software developers insist their programs simply make better use of available network resources, however, the two download mirror sites I spoke to disagree strongly.

"The critical mistake that download accelerator developers make is assuming that both network and server resources are unlimited, said Bayley That is, that networks are underutilized, and that existing transfer/transport protocols do not operate at peak efficiency. Both are obviously untrue to network and server administrators."

User Solution

In the end, common courtesy seems to be the heart of the matter. Free services such as AusMac that provide downloads to so many people feel taken advantage of. Bayley is providing a service to the Mac community and he feels a few individuals are causing problems for everyone. Unless behavior changes, download accelerators will become less practical as services such as AusMac restrict usage, even from people using the software in moderation. Already Bayley started limited simultaneous connections, removing a major ability out of these download utilities. If administrators feel download accelerators are causing havoc with their servers and developers are unwilling or unable to modify the behavior, the only eventuality may be that servers simply won't cooperate with download accelerators. While there are other useful features found in these utilities, the biggest is the ability to fragment downloads for faster transfers. If that function is nullified through server restrictions, these utilities will be far less useful.

For now best the option may be to educate users of download accelerators on the effects of using the software. In particular, the number of connections used when linking up to a file server. Speed Download, for example uses a default of 6 open slots, with a maximum of 32, while Download Deputy defaults to 4 and allows a max of 8 connections. Users need to be aware that the more connections that are used, the less there is available for others. Also, that more download slots don't necessarily mean a faster overall transfer. In my tests, after two or three simultaneous connections, virtually no gain in speed was realized. Paine notes that he feels his default of 4 connections is more than anyone could realistically use to gain any worthwhile speed benefits. Worth noting, here we are talking about broadband connections, and the problem obviously gets worse with slower connections. Imagine one user taking up 30 download slots on a dial up modem, just to save a minute or so of time, while inconveniencing many others waiting several times that to download their own files.

Perhaps common ground could be reached between administrators and developers to find a reasonable method to utilize multiple connections to a server. Specifically something to relax the repeated connection attempts as noted above. Until then, it will likely be up to the users to keep the files flowing by not abusing servers by not excessively using the download accelerators beyond reasonable bounds.