An Introduction to Python: Python Scripting

Introductary Python Scripting

Table of Contents

Python as Tool

Python has a generally been used for System Scripting tasks. Although it can certainly be used to generate more serious applications, scripts are something that Python is particularly well suited for. This is mostly because Python has a large library of utilities for various systems.

By using scripting languages the average person can gain more control over their system. A high level language that offers interfaces to lower level components makes it possible for people to write backup routines, strip content from the web, or do some computations that wouldn't be much fun by hand.

I figure here is as good a place as any to have some system command lists. Check it out(If you have good links for any of these please send it to me):
Microsoft Technet Command Line Reference for Windows NT+
Command line reference for Linux
Command line reference for Mac OS X
Command line reference for Mac OS Classic

Let's start with a simple example of downloading files from the popular web-comic, Penny Arcade. You can check out this game related humor/news site here. I wrote this script because myself and some of my friends are big fans, and we wanted to have access to this comics all the time. Poking around on the PA webserver a bit, I figured out where the comics were stored and decided to write this script:

Please note that this code relies on a type written by Jeff Bauer or Starship Python. The code is included here in the downloadedable example, and it is good code worth taking a look at. However, Mr. Bauer did a very nice job so I won't bother butchering his work here.

In the above code I'm using a very blunt instrument. Not a lot of finesse in this application, but then I just wanted a simple hack to download a lot of files from this page. So a bunch of data is hardcoded when it should be automagically generated, and I could try to not try every single date for the last 6 years. As it goes, this code will bombard the Penny Arcade server with about 2000 requests. I don't really recommend using this code too much, but I think it's a decent example.

Notice that there are two versions of the same code, one that works solely on linux/unix(where I originally wrote the script), and another that should work on any platform that can run Python. The key difference is a shift from reliance on a system application like "wget", to a Python library, namely "urllib". Both work fairly similarly, each takes a url and that's about it.

The Pound-Bang Hack

You've probably noticed from playing with examples that you run these scripts in the following way. 


$ python download_pa_images.py
This isn't like running an executable at the command prompt, and can be a little confusing or just inelegant. A nifty little hack for Unix based systems is the addition of a single line at the beginning of each script you want to use as if it were a program. The line looks like this: 

#!/usr/bin/python
# The above should always be the path of your python
Additionally you'll have to make this file executable, and then run it like you normally would: 

$ chmod +x download_pa_images.py
$ ./download_pa_images.py

Alternatively you can use the env program to invoke Python regardless of where it's located on the users system, assuming you can find env. 


#!/usr/bin/env python

This hack is unavailable on Windows. However, doubleclicking on python files in windows will cause them to be executed, which is a behavior that doesn't show up on the Unix side.

Defining a "__main__" method

Another thing thats useful in scripting is defining what happens when the script file is run. Normally typing: 


$ python myfile.py
executes all the code in "myfile.py", and prints output if there is any, does computations, makes files, whatever. However, sometimes you want to define a set of functions, and execute specific ones when you call the file as a script. By defining a main method, you could specify that when you run "download_pa_images.py" you use the "download_lin()" function, or the "download_cross_platform()" script.

Saving Disk Space/Avoiding WinZip

Note: I'm playing with Zipfiles here because they are used on a pretty wide variety of systems. However, Python also has support for Gzip, Bzip, Tar, and possibly others. So don't think that you don't have options. These libraries are generally similar in usage, so I won't cover them here (unless someone asks).

Let's say that you really hate WinZip, but you like Python. You could do the following:

And of course we can unzip things too. You'll notice that unzipping is quite different. It's similiar enough to file reading with respect to just getting data out of the ZipFile that that isn't too confusing I hope. On the other hand, what is a little tricky is recreating all the directories that need to exist so that you can write out the appropriate file paths. A simple check with the os.path.dirname call takes care of it though as you can see. I can't stress enough how useful the online help for Python is.

Now let's say we have a directory for of Movies, large text files or word documents, and music in the form of mp3's. Now in an effort to save disk space, and knowing that it's not uncommon for a 20 Megabyte textfile to take up only 1 Megabyte after compression, we figure we can save some space on our 300 Gigabyte Harddrive. (Ha!) Using our knowledge from the Zip scripts above, let's write a more specific script to handle the compression of a specific type of file on an entire tree of directories.

I'm currently using something like this to produce archives of all the sourcecode and examples in this tutorial. I could have used a program like tar or gzip, or even winzip, but doing it with Python is just more fun I think. /home/steder/Projects/Tutorials/python/BobChat-v0.3/BobClient.py 

# Simple Application/Script to Compress files of a
# specific type in a Directory.
# I'm using this to build the packages of
# example codes for download that are
# on the first page of this tutorial.
# Yes:  You could use Zip, Winzip, Tar, or whatever
# to pick out just the code files and so on, but this
# is just a little more fun.  
"""
Path can be a file or directory
Archname is the name of the to be created archive
"""
from zipfile import ZipFile,ZIP_DEFLATED
import os
import sys 
def zipty(path, archive, type):
    paths = os.listdir(path)
    for p in paths:
        p = os.path.join(path, p) 
        if os.path.isdir(p): 
            zipty(p, archive,type)
        elif os.path.splitext(p)[1] == type: #Just a little change here
            archive.write(p) 
    return

def ziptype(path, archname, type):
    # Create a ZipFile Object primed to write
    archive = ZipFile(archname, "w",ZIP_DEFLATED) # "a" to append, "r" to read
    # Recurse or not, depending on what path is
    if os.path.isdir(path):
        zipty(path, archive, type)
    elif os.path.splitext(path)[1] == type: #This is all we're changing
        archive.write(path)
    else:
        "Compression of \""+path+"\" was failed!" # And a failure message
    archive.close()
    return "Compression of \""+path+"\" was successful!"

instructions = "ziptype.py:  Simple zipfile creation script." + \
               "recursively zips files in a directory that match" +\
               "a specific type."+\
               "e.g.:  python zipit.py myfiles .py myfiles.zip"

# Notice the __name__=="__main__"
# this is used to control what Python does when it is called from the
# command line.  I'm sure you've seen this in some of my other examples.
if __name__=="__main__":
    if len(sys.argv) == 4:
        # ziptype("directory/path", "filetype", "archivename")
        result = ziptype(sys.argv[1], sys.argv[3], sys.argv[2])
        print result
    else:
        print instructions

That's about it for some scripting examples. I'd love to do some more diverse things, I kind of got sidetracked on Zipfiles here, but I found that I actually needed a simple zip utility, and now I've got one, and a pretty simple example. I'd like to apologize then to people who are interested in some other area that they'd like to script. In an attempt to make it up to those of you who feel slighted, please let me know what you think of these examples, and/or what you would rather see, and I'll add some new examples as soon as possible.

Download the source for this example here.

linkage
Table of Contents