v0.1 Server
About Select
The v0.1 server is built on a special socket function called "select". Select is an asynchronous operation, meaning that it doesn't wait(or block) until data has been sent or received. Instead it is capable of multiplexing over a large number of sockets and only receiving or sending data on those sockets when they are ready to be written to or read from.
Select is a good alternative to a multithreaded server because:
- It is easier to program and conceptualize
- Unless you're dealing with long requests, spawning threads for each request is overkill
Code
Here is the complete v0.1 server code listing. I'll explain pieces next.
"""
BobServer.py v0.1
---
This is the first version of the simple Bob Chat Server
"""
import sys, time
from select import select
from socket import socket, AF_INET, SOCK_STREAM
def now():
return time.strftime("%I:%M:%S%p",time.localtime())
# Server Constants
HOSTNAME = "" # "" == localhost
PORT = 7777
# Make main sockets for accepting new client requests.
mainsocks, readsocks, writesocks = [],[],[]
# Create an reading socket.
portsock = socket(AF_INET, SOCK_STREAM)
portsock.bind((HOSTNAME, PORT))
portsock.listen(5)
# A more serious application might open several ports for listening. (like 7777 - 7780)
# These lists are used by select. For now we just have incoming sockets
mainsocks.append(portsock)
readsocks.append(portsock)
# Message Queue
Q = []
def ServeIt():
# Starting Select Server:
done = 0
while not done:
# The one and only call to select - There is one missing optional argument
# in the select call which can be used to set a timeout or timeout behavior.
readables, writeables, exceptions = select(readsocks,writesocks,[])
for sockobj in readables:
if sockobj in mainsocks:
# port socket: accept new client
newsock, address = sockobj.accept()
print 'Connect:', address, id(newsock)
readsocks.append(newsock)
else:
# This is already an open connection, handle it
try:
data = sockobj.recv(1024)
print '\tgot', data, 'on', id(sockobj)
if not data:
sockobj.close()
readsocks.remove(sockobj)
if sockobj in writesocks:
writesocks.remove(sockobj)
else:
# Drop message in the Q
Q.append(data)
# Make the socket writable
if not (sockobj in writesocks):
writesocks.append(sockobj)
except:
pass
# Get ready to write
while 1:
if (len(writeables)==0) or (len(Q) == 0):
break
message = Q[0]
# Actually Write Messages
for sockobj in writeables:
# Name(Time): Message
try:
sockobj.send( "%s(%s): %s" % (id(sockobj), now(), message))
print "\tsent",message,"to", id(sockobj)
except:
pass
Q.remove(message)
if __name__=="__main__":
try:
ServeIt()
finally:
portsock.close()
syntax highlighted by Code2HTML, v. 0.9.1
Servers need to listen
The following code is a basic part of what makes a server a server.
# Create an reading socket.
portsock = socket(AF_INET, SOCK_STREAM)
portsock.bind((HOSTNAME, PORT))
portsock.listen(5)
You create a socket object and "bind" yourself to it, which is kind of like listing your address or phone number so that others know how to contact you. (i.e. You can reach me on HOSTNAME street, house number PORT). The "listen" function puts this socket officially into 'server' mode, and the argument (5) is the backlog. (The number of connections the server will queue before denying clients. This number really only comes into play when your server is getting hit with new connections faster then it can handle.)
Socket Types
The constants AF_INET and SOCK_STREAM specify the type of socket. These are common values that give us an IPv4 TCP/IP socket. (AF_INET specifies IPv4, SOCK_STREAM specifies TCP/IP as the protocol). Most applications on the web today use IPv4 and TCP/IP.
The alternative to TCP/IP is UDP, which uses the constant SOCK_DGRAM. TCP/IP provides error checking and guarantees that packets of data arrive at their destination. TCP/IP also guarantees that packets arrive in order. UDP on the other hand simple throws packets of data out onto the network and does not guarantee that they will arrive, much less arrive in order.
UDP is useful for specifying your own low level protocols (like TCP/IP), and it's also used in some common applications and games where reliability isn't as important as speed. Some data transfer applications use UDP rather then TCP/IP because the order the data arrives is not as important as the speed with which the data is sent. TCP/IP's error checking tends to make it slower. It is important to remember that there is a tradeoff between speed and reliability in these applications.
Socket Lists
# Make main sockets for accepting new client requests.
mainsocks, readsocks, writesocks = [],[],[]
# SNIP #
# These lists are used by select. For now we just have incoming sockets
mainsocks.append(portsock)
readsocks.append(portsock)
# Message Queue
Q = []
The server uses a set of lists to manage its sockets.
Mainsocks is simply the list of sockets that the server listens to. In this example (and most of BobChat) this will only be a single socket. The list is used incase I were to open multiple sockets on multiple ports or something similar.
Readsocks is the list of sockets that select will monitor for incoming messages. Each client will have one entry in the readsocks list.
Writesocks is the list of sockets that the server will write to to send messages to its clients. Select will monitor this list to see if these sends have completed.
Q is a qute name... haha...
Q is a cute name for a Queue or FIFO(first in, first out) datastructure. It's extremely easy to implement a queue in Python. You can simply use a list and always append to add elements and always remove elements in order from the beginning of the list.
In this case Q is a list of messages that are waiting to be sent to the connected Client(s).
The Main Loop
The server itself is simply the "ServeIt" method, which consists of a single main "select" loop. each iteration of the loop calls
readables, writeables, exceptions = select(readsocks,writesocks,[])
Which selects the sockets from readsocks that are readable and puts them in the readables list as well as selects the sockets from writesocks that are writable and puts them in the writeables list.
Loop over readable sockets
for sockobj in readables:
Accepting New Connections
if sockobj in mainsocks:
# port socket: accept new client
newsock, address = sockobj.accept()
print 'Connect:', address, id(newsock)
readsocks.append(newsock)
Reading Data
If it's not a new socket we should attempt to read data from it (since it's readable). If the data is None / "" ( no message ) that indicates that the socket was closed or disconnected. The "if not data:" case handles this by closing the socket on the server and removing it from the servers socket lists.
Otherwise the message is simply placed in the queue and the socket is now considered writable
else:
# This is already an open connection, handle it
try:
data = sockobj.recv(1024)
print '\tgot', data, 'on', id(sockobj)
if not data:
sockobj.close()
readsocks.remove(sockobj)
if sockobj in writesocks:
writesocks.remove(sockobj)
else:
# Drop message in the Q
Q.append(data)
# Make the socket writable
if not (sockobj in writesocks):
writesocks.append(sockobj)
except:
pass
The try statement above is overly general. It's there simply to catch EOF errors from sockets when they run out of data.
Loop over Q/writables
while 1:
if (len(writeables)==0) or (len(Q) == 0):
break
message = Q[0]
# Actually Write Messages
for sockobj in writeables:
# Name(Time): Message
try:
sockobj.send( "%s(%s): %s" % (id(sockobj), now(), message))
print "\tsent",message,"to", id(sockobj)
except:
pass
Q.remove(message)
Because writing is more unpredictable we use a while loop here and simply break out when we've emptied either of writables or Q lists.
