Cybertelecom Federal Internet Law & Policy An Educational Project

Backbones, Background Info

Derived From: Characteristics and Competitiveness of the Internet Backbone Market, GAO-02-16 (Oct 2001)

To access the Internet, most residential users gain Internet access over DSL, cable, wireless or dial-up lines. For a residential customer, the ISP sends the user's Internet traffic on to the backbone network. To perform this function, ISPs obtain direct connections to one or more Internet backbone providers (connections to multiple backbone paths is known as multihoming and creates redundancy in order to protect against network outages). Small business users may also connect to a backbone network through an ISP, however, large businesses often purchase dedicated lines that connect directly to Internet backbone networks.

An ISP's traffic connects to a backbone provider's network at a facility known as a "point of presence." Backbone providers have points of presence in varied locations, although they concentrate these facilities in more densely-populated areas where Internet end users' demands for access are greatest. If an ISP or end user is far from a point of presence, it is able to reach distant points of presence over telecommunications lines. Figure 1 depicts two hypothetical Internet backbone networks that link at interconnection points and take traffic to and from residential users through ISPs and directly from large business users.

Once on an Internet backbone network, data packets that were created by the end user's computer are routed over the most best known available route and reassembled at their destination point; its possible for different packets to take different routes between end points. Internet packets are created and routed pursuant to the IETF standard the Internet Protocol (IP).

There are many Internet backbone providers offering service in the United States. A few of these are considered to be "Tier 1" backbone providers. See Wikipedia Tier 1 Providers

Regulation: Unlike telecommunications services, the provision of Internet backbone service is only lightly regulated by governmental communications agencies, usually only through ex post antitrust considerations. See discussion of backbone regulation.

NAPs and Private Interconnection Points

Two types of facilities are used for the exchange of data traffic by interconnected Internet backbone providers. The first type of facility, known as a "network access point" (NAP), enables numerous backbone providers to interconnect with each other at a common facility for the exchange of data traffic. Internet data traffic is also exchanged by backbone providers at "private" interconnections.

A NAP facilitates the interconnection of multiple backbone providers. In the early to mid-1990s, the National Science Foundation designed and partially funded four NAPs, each of which was managed by a different company. Since that time, other interconnection points have been constructed, and for purposes of this report, the term NAPs refers to approximately major traffic exchange points that host backbone providers. Managed by different companies, NAPs are not uniform facilities; differences exist in terms of equipment, software, and data transmission rates.

Although most backbone providers we interviewed use the NAPs, a few providers voiced concerns about them. In the first years of their existence, NAPs became congested with the rapid rate of growth in Internet traffic. Two of the providers with whom we spoke said that some NAPs were not well managed. Also, originally some NAP technology was not "scalable"- that is, beyond some level, it was very costly to increase the amount of traffic that could be exchanged at a NAP. If traffic exchange at a NAP became congested, service quality could be compromised. Two typical problems that congestion causes include latency (delay in the transmission of traffic) and packet loss (when transmitted data are actually lost and never reach their destination). For example, one backbone provider told us that the loss of packets at some NAPs had sometimes reached 50 percent.

The congestion and poor quality of connections at the NAPs led backbone providers to engage in another type of traffic exchange known as "private interconnection." Private interconnection refers to the exchange of traffic at a place other than a NAP. Usually, these private interconnections involve two companies entering into a bilateral agreement to exchange traffic; no third party manages the traffic exchange. The parties interconnect their networks at any feasible location, such as a facility of one of the providers. Because of the private nature of these agreements, the number of private interconnections that currently exist across the United States, according to one company representative, is not known.

Despite a variety of technological developments that have improved traffic flow at NAPs, we found that for the providers we interviewed, the majority of Internet traffic exchange occurs at private interconnection points. Of 17 backbone providers with whom we spoke, 15 used both NAPs and private interconnections; the remaining 2 used only private interconnections, avoiding the NAPs entirely. Slightly more than half of the 15 providers using both NAPs and private interconnection said they exchanged more than 80 percent of their traffic at private exchange points. Of the 17 companies that we met with, 10 provided estimates of how their mix of private interconnection and NAP use would likely change in the future. Nine of the 10 stated that they either plan less use of NAPs in the next few years or do not see their mix of NAPs and private interconnection changing; only one company said that it was likely to make greater use of NAPs in the future.

We found that some Internet backbone providers value several features of NAPs. For example, when a company interconnects at a NAP, it saves on equipment costs and administrative overhead. Representatives of two companies with whom we spoke noted that the NAPs play an important role in helping to keep the market for backbone service open for entry, and thus more competitive, because NAPs provide new backbone firms an efficient, low-cost method for exchanging traffic with numerous other providers.

A third alternative has emerged that is a hybrid of NAPs and private peering. A perceived problem with NAPs is that they tend to be hosted by a particular network which is also a party to the interconnection agreement. Companies such as Equinix have proposed a different model of a network neutral hosting center. The hosting center provides all necessary services. Networks come to the centers and create points of presence by collocating their equipment. Interconnection is achieved simply by running a new cable across the floor. It's like a NAP in that multiple networks come together at one center, but interconnection is private and it is not controlled by one of the parties to the agreement.

Peering and Transit

Independent of the type of facility at which backbone providers exchange traffic, two different types of financial arrangements exist among backbone providers for traffic exchanges. In a "peering" relationship, backbone providers exchange data destined only for each other's network generally without the imposition of a fee. Transit payments, which involve the payment by one backbone provider to another for the mutual exchange of traffic and for the delivery of traffic to other providers, have become more common with time.

When the commercial Internet began, only a few major backbone providers of relatively similar size existed, each of which sent and received roughly equal amounts of traffic. The similarities among these backbone firms led them to view each other as "peers." These providers elected to exchange traffic for free, rather than trying to measure the actual traffic exchanged and developing a payment method. In a peering arrangement, two backbone providers agree to exchange traffic destined only for each others' networks. As depicted in figure 2, the peering agreement between backbone provider A and backbone provider B only covers traffic going from A's network to B's network and vice versa. For backbone A to move traffic to backbone C's network under peering, it must have a peering agreement directly with backbone C.

By the mid to late-1990s, another financial arrangement known as "transit" emerged. Transit and peering are distinctive in two key respects. First, while peering generally entails traffic exchange between two providers without payment, transit entails payment by one provider to another for carrying traffic. Transit agreements thus constitute a supplier-customer relationship between some backbone providers, much like the relationship between a backbone provider and a nonbackbone customer (such as an ISP). Second, when a backbone provider buys transit from another provider, it obtains not only access to the "supplier's" backbone network, but also access to any other backbone network with which its supplier peers. Regarding physical locations, however, both transit and peering take place at NAPs as well as at private interconnection points.

There is a segregation of backbone providers into "tiers." The top tier or "Tier 1" providers generally peer with each other and sell transit to smaller backbone providers. However, we found that smaller providers often peered with each other and were able, in some cases, to peer with larger providers.

The illustration in figure 3 shows backbone provider C as a transit customer of backbone provider B and backbone providers B and A as peers. In this case, traffic originating on backbone C can get to backbone B's network as well as to that of backbone A (with which backbone C does not have an independent relationship) because B will pass C's traffic off to A as part of its delivery of transit service to C. Thus, a smaller backbone provider generally need only buy transit from one or two large providers to achieve universal connectivity.

We found that it is generally not viewed as economical for a backbone provider to peer with a less geographically dispersed backbone provider. Thus, even if there were equal traffic flows, the larger provider will tend to carry traffic a further distance-which, according to a larger backbone provider we spoke with, ultimately means more costs are imposed on its infrastructure-when it peers with a provider with a smaller or less widely dispersed network.

Figures 4 and 5 illustrate this paradigm. In figure 4, backbone providers A and B are of similar size, and traffic between the two could be carried mostly by one backbone provider in one direction, but mostly by the other in the opposite direction.

In figure 5, backbone provider D is smaller than backbone provider C, with more limited points at which traffic can be brought onto the network. When backbones C and D exchange traffic, C must carry the traffic much farther on the return path before it can hand off the data packets to D. Therefore, C might consider D to be benefiting from C's network investment and thus, C would be more likely to view D as a customer purchasing access to its network than as a peer in traffic exchange.

The "tiering" of Internet backbone providers and the dual system of peering and transit agreements have caused controversies. Several of the non-Tier 1 backbone providers with whom we spoke expressed concerns about their inability to peer with the largest providers. In particular, we were told that the inability of non-Tier 1 providers to peer with Tier 1 providers puts smaller companies-which must therefore purchase transit service-at a competitive disadvantage. We were also told that peering policies should be made public.

To some extent, market forces may be relieving some of these problems. First, despite the view that smaller providers have no choice but to buy transit, some backbone providers with whom we spoke stated that the market is competitive, and transit rates have been decreasing. Second, eight of the backbone providers with whom we spoke (some of which were Tier 1 providers and some of which were not) said they already had posted or soon would be posting their peering policies on their Web sites or otherwise making them publicly available.

Perhaps most interesting, we found that some non-Tier 1 backbone providers do not want to peer with the largest backbone providers. For example, one provider spoke critically of the quality of peering connections and the quality of service provided between peers. Some stated that it is difficult to guarantee their own clients a certain level of service if they receive few guarantees themselves-a common occurrence under peering. Transit customers, however, do contract for a specified level of service for such items as "uptime"-the functioning of a network without impairment or failure.

Networks

• Above.net Peering

Papers

• Scott Marcus, Global Traffic Exchange among Internet Service Providers, OECD Briefing (2001)
• Geoff Huston, Interconnection Peering and Settlements, INET’99 presentation
• William B. Norton
  • The Evolution of the U.S. Internet Peering Ecosystem, Draft 1.1 (Nov. 19, 2003)
  • Interconnection Strategies for ISPs, Document v.2.0
  • A Business Case for ISP Peering, Draft 1.3
• Bill Woodcock, White Paper on Transactions and Valuation Associated with Inter-Carrier
  Routing of Internet Protocol Traffic or BGP for Bankers, Version 0.2;
• Daniel C.H. Mah, Explaining Internet Connectivity: Voluntary Interconnection Among
  Commercial Internet Service Providers (March 26, 2003)
• Steve Gibbard, Economics of Peering (Oct. 2004)