TCP for the Uninitiated - Part I (Introduction and Background)

By: Erik Iverson
http://www.dragonmount.net/

Introduction

The number of users on the Internet has grown exponentially since its inception. The Internet as we know it today had a rather humble beginning. It was an experimental network invented by the Department Of Defense, and more specifically, the Advanced Research Projects Agency (ARPA). At the time, there were only four nodes (communication endpoints) connected to the Internet (ARPANET). Although that sounds miniscule by today's standards, it was an incredible accomplishment for the time. This all happened in the late sixties and early seventies. Unfortunately, the way that the ARPANET communicated was prone to crashing and therefore not very reliable. In the mid seventies, Vinton Cerf and Robert Kahn suggested a new set of protocols. This was the birth of TCP/IP and by the mid eighties it was fairly standard. It is still the protocol of choice for the Internet and networks worldwide.

Not amazingly, few understand even the most elementary of concepts of how the Internet functions. Most believe it is just a combination of Internet Explorer, Netscape Navigator, and Microsoft Outlook. By reading this, you should gain an understanding of some common terms you hear when discussing the Internet, the big picture of how the Internet works, and even some details about how specific applications function over the Internet. Lastly, and most importantly, I hope to show you how you can start to learn things about computer networks and the Internet on your own, and to provide you with links and programs that will help you in that cause.

First, we should define the Internet. If you're reading this you probably know what a computer network is. The Internet is the largest computer network, composed of all the clients, servers, and routers that communicate with each other across the globe.
You'll find that a lot of people confuse the terms Internet and World Wide Web (WWW). The WWW, along with e-mail, are the two most popular services the Internet offers to the common person. So when you hear "I am on the Internet", the large majority of people are referring to the WWW. We shall see that there are many more parts to the Internet than the WWW and e-mail; they are just the ones that get all the glory in pop culture. The reason for this is ease of use. These services offer a familiar and easy to learn point and click interface which the common person is able to grasp.

Protocols

So, since we are all familiar with the WWW, let us examine it to see how a typical Internet service works. We will now define a new term, protocol. A protocol is a set of rules that allow communication between hosts. By host I mean either a server or a client. Think of it this way: there are many different operating systems. Windows is the most popular, but there is also Linux, Solaris, OpenBSD, and Macintosh to name a few. All these machines must be able to "talk" to each other on the Internet. A Windows machine must be able to connect to an OpenBSD machine and fetch a web page for instance. Or vice-versa. This is where protocols come to the rescue. They standardize, or attempt to, how machines communicate to each other on the Internet.

To put it another way, if I know only English, I can't communicate to someone who is speaking Chinese. We need to speak the same language, and in the case of the Internet, the "language" is called TCP/IP.

The Internet uses many layers of protocols to get the job done. The main layers are the physical layer, link layer, network layer, transport layer, and the application layer.

Let's start with the physical layer, or hardware layer. This is the "lowest level" that deals with electronics and transfer of data across lines. It transmits the "1's" and "0's" that you are probably familiar with. For a decent look at this aspect, check out the "Voice and Data Communications Handbook."

The link layer won't be touched on much here either. It puts data into units called frames. The frames contain a header with certain information. On a LAN, the frames header would contain the source and destination MAC addresses on the network cards. Link layer protocols can carry any type of data across the link; it doesn't matter what protocols the higher layers use.

The network layer has to do with routing information across the Internet. You have probably heard the term "packet" thrown around fairly liberally. When talking about the Internet Protocol (IP), the correct term to use would be datagram. Routers are responsible to get datagrams where they should be. The decision is, for the most part, based on the destination IP address of the datagram. There are many different routing protocols one can choose from, each with their own strengths and weaknesses. This is another topic, however. What you should know is that the network layer uses IP, and is responsible for routing packets from one place to another.

The next level is the transport layer, the meat of the discussion. Two main protocols are available for this layer, the User Datagram Protocol (UDP) and the Transmission Control Protocol (TCP). UDP is an unreliable protocol; it does not guarantee message delivery. UDP gives its best effort of getting data somewhere. The good part about UDP is the overhead on the network is lower. When most people hear "unreliable", they think of negative things. UDP is not a bad thing per se; it just doesn't do any checking to see if a particular piece of data ever gets to a host. As it stands, applications should do their own checking to see if a piece of data arrives using UDP.

TCP connections are reliable; if data gets lost or garbled, TCP will retransmit it until it is received properly. We will see how this works in more detail shortly. TCP makes sure data gets to the host in the right order, with the correct message, and without duplicating the message.

The "highest" layer in computer networks is the application layer. Different Internet services, or programs, have different protocols. Some examples of application layer protocols are Telnet, File Transfer Protocol, Internet Relay Chat, and E-mail. The WWW has its own protocol too; it is the Hyper Text Transfer Protocol, better known as http. This is why you type http before most web addresses. It lets the browser know what protocol to use.

As you are seeing, protocols play a vital role in the Internet functioning. Let's take a look at some highlights of the Transmission Control Protocol, or TCP.

Ports

TCP makes sure that two computers have a reliable connection. With the millions of packets moving across the Internet, things get lost sometimes. TCP makes sure that when this happens, data gets to the intended host in an orderly, reliable manner. How does this work? Well, we are not writing a TCP stack today so we'll go over the basics and you can learn more from there. I want to hit the main points without overwhelming you with details. When you understand the concepts, the details are learned rather easily.

First, we'll tackle the issue of ports. You might have heard or ports before. "Ftp to my port 21, telnet to my port 23, use port 6667 to get on IRC," your Internet savvy friend says. Then again, perhaps you've never heard of ports before. Ports are analogous to doors on a house. Machines can have open ports and closed ports. Machines on the Internet can open up ports numbered 0 to 65535, which for you binary number folks means a port is a 2-byte integer. By and large, open ports will offer services to someone. For example, a machine running a WWW server has port 80 open. Port 80 was chosen for the standard WWW port. Unless you direct it otherwise, your browser will connect to port 80 on www.altavista.com or port 80 on www.zdnet.com when you try and visit these sites. Having a standard port helps so that everyone is on the same page. But don't think that the WWW wouldn't work if they had chosen port 45, or 199. It's just a number used so we, and programs, know how to find it. Below are some common services and their port numbers:

21-File Transfer Protocol (FTP)
22-Secure Shell (SSH)
23-Telnet
25-Simple Mail Transfer Protocol (SMTP)
79-Finger
80-World Wide Web (WWW)
110-Post Office Protocol (POP)
26000-Quake

These are not set in stone. It would be one and the same to have Telnet running on port 21, and FTP on port 23. The port numbers are there for standardization; there is nothing magic about them. I often find people who think there is something special about the numbers; there isn't! If you are running a server of some type, more likely than not, you will be able to tell it what port you want it running on through an options menu or file. More often than not, though, you will want to leave it on the default port, so it is easily found.
So somebody thinks they're clever and wants to run a "hidden" web server, one that only certain people can find. He gets a web server and configures it to listen on port 75 instead of the common port 80. Has he hidden the web server? Maybe from his little brother, but probably not even him! There are tools widely available for every operating system called port scanners. These programs will, among other things, search a target machine for every open port and report back to the user what ports are open on that machine. If it is a more advanced port scanner, it might also tell them what service is running on that port, and even the operating system you are running. So if you want to set up a "private" web server, use some sort of password authentication method. Simply hiding the server on a different port is not going to help much. Security through obscurity is a bad idea when dealing with computers; get it out of your head as soon as possible!

So now you hopefully understand that servers will have ports open with services running on them. One machine can have as many ports open as it feels like, up to the maximum limit of 65535 of course. The point is that you don't need separate machines for an FTP and a WWW server for instance; they can be running on the same computer. You can't however, have two things running on the same port on the same machine. This isn't entirely true, I suppose, if the machine had two IP addresses, but by and large it is the case.

Connection Process

So let's see ports in action and learn a new concept, how connections are established using TCP. There is no better tool to see TCP in action than a network sniffer. There are a great many sniffers available for any operating system. Since I am on a Windows machine right now, and it is a popular OS, we'll use a sniffer made for Windows. The one we will be using for our demonstrations is called ButtSniff, and was written by L0pht Heavy Industries (www.l0pht.com). They released ButtSniff as a plugin to the popular Back Orifice remote administration tool (www.cultdeadcow.com), but it is a great standalone application too! Without further ado...

We are going to view my computer connecting to www.l0pht.com, port 80. We will see what is called the 3-way handshake in action. We'll show you the output and then describe it. Pay attention! This is where a lot of people get confused, because I'll be talking about things like SYN and ACK flags. Don't get scared, it's really an intuitive concept. The scientists who thought of TCP were no fools!

Since TCP is supposed to be reliable, it needs a way of making sure data arrives in the right order. Naturally, numbers are used to keep track of where the data belongs. Think of each TCP segment (packet) as being labeled with a number, called a "sequence number". Each segment also has an ACK number, which is the segment number it is expecting from the other host. TCP is a two-way protocol. There is an "out-going" data stream and an "incoming" data stream. Confused? Let's look at this in action and I guarantee it will become clear in time.

Buttsniff Dump of directing browser to www.l0pht.com

Packet #1:

Source IP: 160.94.182.175 Target IP: 199.232.79.50
TCP Length: 0 Source Port: 3038 Target Port: 80 Seq: 08A236AC Ack: 00000000
Flags: S Window: 8192 TCP ChkSum: 43386 UrgPtr: 0

Packet #2:

Source IP: 199.232.79.50 Target IP: 160.94.182.175
TCP Length: 0 Source Port: 80 Target Port: 3038 Seq: BE51108E Ack: 08A236AD
Flags: SA Window: 16384 TCP ChkSum: 54084 UrgPtr: 0

Packet #3:

Source IP: 160.94.182.175 Target IP: 199.232.79.50
TCP Length: 0 Source Port: 3038 Target Port: 80 Seq: 08A236AD Ack: BE51108F
Flags: A Window: 8192 TCP ChkSum: 1870 UrgPtr: 0

Step 1 of 3-way handshake

To the uninitiated, this mess might just confuse you more. Here comes the explanation. Note the items in bold, as I will concentrate on them. The IP is the 4-byte address that each computer on the Internet has. Each computer must have a unique IP address if they wish to receive traffic on the Internet. So you'll see here in the 3-way handshake demo, that there are two IP addresses. Mine, which is currently a computer at the University of Minnesota - Twin Cities, and www.l0pht.com, which I assume is in Boston. You'll see my machine sends the first packet, which only makes sense. I am the one who wants to start a connection. L0pht has no idea that I want to connect to their machine, until their machine receives my first packet. It knows I want to initiate a connection because the TCP packet has a special flag set, which shows up as "S" on the sniffer output. "S" stands for the SYN, or synchronization flag. It is the first step of the 3-way handshake.

Other things to note that are interesting about this packet are the source port and the destination port, particularly the destination port, 80. It is 80 because we are trying to access the WWW service on www.l0pht.com. The source port is picked by the operating system we are using, in this case Windows. It will find a port that is unused on your machine and use that. So when you access a web site, you're port 80 is not open. And if it is open, it probably does not have anything with you having your web browser open. It would most likely mean you are running a web server yourself. So could the source port by 80 by random chance? Most likely not! In almost all cases, the OS won't pick a port below 1024; those are supposed to be reserved for things like services. They are called the "well known ports."

So in this case our machine at Minnesota has IP address 160.94.182.175 and is using port number 3038 for this connection. The L0pht's machine has IP address 199.232.79.50 and is using port number 80.

Step 2 of the 3-way handshake

Now L0pht knows we want to establish a connection with them. They have received our packet with the SYN flag set, they respond by sending us a packet with two flags set. The SYN (synchronization) and ACK (acknowledge) flags are both set on the reply packet. This is the second part of the 3-way handshake. More on the rest of the info in that packet in a moment.

Step 3 of the 3-way handshake

The third and final piece to the puzzle is our packet that we send to L0pht with only the ACK flag set. So go over it in your head until you remember: SYN, SYN/ACK, ACK - SYN, SYN/ACK, ACK. You've now got the order of the 3-way handshake down, but what does it mean?

SEQ/ACK Numbers

Remember how I said before that for TCP to be reliable it had to number the data it sent? We're going to learn about that now. The two numbers in bold above are the Sequence and Acknowledgement numbers, also known as SEQ and ACK. Notice when we send the first packet of the 3-way handshake that there is a SEQ number but not an ACK number. This sequence number, the one on the initial SYN packet sent to establish the connection, is called the initial sequence number. The TCP stack decides what the initial sequence number should be. Why not just start with the number 1 you may ask? We will touch this in a future tutorial. Now notice the SEQ and ACK numbers and how they correspond with each other. What is an ACK number then, and why is it 0 in the initial sequence packet? A machine sends an ACK to acknowledge that it has received a packet. The ACK number is one greater than the last SEQ number received from the other host. The ACK number can be thought of as the number that that particular end of the connection wants to receive next. TCP is full duplex; data can travel both ways. There is an outgoing data stream and an incoming one, too. So each end of the connection has it's own Sequence number and keeps track of the sequence number it wants to receive from the other end in the form of an ACK number.

So using our example, our initial sequence number is: 08A236AC. This is a hex number, which uses the standard 0-9, as well as A-F to extend the number system to base 16 (for example, 08A236AC + 1 = 08A236AD). Our initial packet has no ACK number, since we don't know what the L0pht's initial sequence number is (the ACK number shows up as 0, but this just means it does not really have one). The L0pht's machine receives our SYN packet, with the initial sequence number of 08A236AC. They reply with their own initial sequence number (decided by the operating system, in this case BE51108E), and the ACK number is our initial sequence number + 1 (08A236AD)! Notice the addition of one since the ACK number is the number they want to receive next, not the number they have already received. That is a vital concept. Notice the L0pht's packet has both the SYN and ACK flags set, since it contains both the initial sequence number and an ACK number, letting our machine know what to send next.

Our machine receives that packet and peeks at the ACK number, and sees that it is 08A236AD. So our machine sends the L0pht a packet with sequence number 08A236AD, and also has an ACK number. The ACK number is the L0pht's initial sequence number + 1, or BE51108F. Notice that only the ACK flag is seta the SYN is only used when giving an initial sequence number (that is, when first connecting).

Hopefully you can see how this works now. It should be clear how TCP retains a reliable connection. Let's take one end of the connection, ours. If the L0pht's machine sent us a packet with a sequence number lower than what we are expecting, we can discard it because it is old data. If it is higher than what we were expecting, then we must have missed some data. When this happens, we do not send an ACK for the new packet until we have received all the packets before it. When the L0pht's machine realizes we have not acknowledged receiving some of the data it sent (that is, when it times out), it will send it again. We receive it this time, and continue data transmission. All this occurs at immense speed and is transparent to the end user.

This wraps up section one. I don't want to lay too much information on you at once. There is plenty to know about TCP and the Internet Protocols, and I myself am learning more about it each day. Let me list the things that I hope you can take away from this tutorial. Lastly, there are some references which have been invaluable to me and that I highly recommend to further your knowledge.

Key Concepts

• Ports
• IP addresses
• 3-way handshake
• Layered protocols
• Function of Internet Protocol (IP)
• Simple differences between TCP and UDP
• Basic truths about TCP
• Basic concept of roles of Sequence (SEQ) and Acknowledgment (ACK) numbers

References

Attrition. Presentation on TCP/IP. http://www.attrition.org/news/content/tcpip_defcon.zip

Bates, Regis, et al. "Voice and Data Communications Handbook." McGraw-Hill, 1998.

L0pht. Buttsniff.  www.l0pht.com. File in packetstorm.securify.com archives.

Feit, Sidnie. "TCP/IP: Architecture, Protocols, and Implementation with Ipv6 and IP Security." McGraw-Hill, 1999.

Stevens, Richard. "TCP/IP Illustrated Volume 1." Addison-Wesley, 1994.