Documentation
LINK SORTER:  V2.0

System Requirements

  • Unix
  • Perl 5 with sockets library (standard)
  • Telnet access
  • Simple to install

Installation

Preliminaries

  • Determine the path to PERL 5 on your web server host.  Note that some web hosting companies run both PERL 4 and PERL 5.  Make ABSOLUTELY sure you are not setting this up under PERL 4.  Ask your administrator if you are not sure.
  • DOWNLOAD the tarfile for this program and save it to your desktop.
  • Unpack the tar archive on your desktop using a program that unpacks UNIX TAR ARCHIVES. If you don't have such a program then download WINZIP FREE from SHAREWARE.COM
  • After you have unpacked the TAR archive you will have a collection of folders and files on your desktop.  Now you have to do some basic editing of each of these files (or at least some of them).  Use a text editor such as wordpad, notepad, BBEdit, simpletext, or teachtext to edit the files.  These are NOT WORD PROCESSOR DOCUMENTS they are just simple TEXT files so don't save them as word processor documents or save them with extentions such as .txt or they will NOT WORK.   Note that there may be a some files inside of folders which are "blank".   This is normal.

Preparing the CGI scripts

Define Path To PERL 5

The first step is to open up each and every file that has a .cgi extention and edit line number one of each script.  Each of the cgi scripts is written in perl 5. For your scripts to run they must know where perl 5 is installed on your web server. The path to perl 5 is defined to a cgi script in the first line of the file. In each of the cgi scripts the first line of code looks something like this:

#!/usr/bin/perl

If the path to perl 5 on your web server is different from /usr/bin/perl you must edit the first line of each cgi script to reflect the correct path. If the path to perl 5 is the same no changes are necessary. If you do not know the path to perl 5 ask the webmaster or system administrator at your server site.  

Configure the cgi files

only one thing to configure here so this is really simple

  • $nslookup = "/path/to/nslookup"; path to nslookup on your system (usually /usr/bin/nslookup or /usr/sbin/nslookup)

Get ready to rock

Create a directory somewhere and upload linksort.cgi (chmod this to 755).  Now upload your url list and call it "input.txt".  Make sure the url list is ONE URL PER LINE and upload as ascii.  You can leave the http://'s and other garbage in the list - it will parse all of this anyway.  Just make sure there are no BLANK lines.   Now chmod the directory containing linksort.cgi and input.txt to 777.

Running the software

This is really simple.  Login via telnet and navigate to the linksort directory and type

perl linksort.cgi

The script will then start going through the urls inside of "input.txt" and sorting them into various categories.  Take a look as this is running and you might see:

  • 200.txt    these are good urls
  • 302.txt    these are redirecting to another url
  • 404.txt    these are file not founds
  • dead.txt    these sites are offline
  • unknown.txt    these sites are not classified
  • sorted.txt    these are the urls that the program has already sorted

Sometimes the program will hang up on what is called a "broken pipe".  This is a bug in the sockets protocols that can't be fixed apparently.  When this happens (if you see the program hang for a very long time) hit CTRL-Z to break the program then restart it

perl linksort.cgi

linksort will begin where it left off.   The broken pipe url will get thrown into the "dead.txt" file.