Installation

To install Shepherd:

1. Download Shepherd:

   wget http://www.whuffy.com/shepherd/shepherd
   

2. Run it:

   perl shepherd
   

3. If Shepherd won't run because a "mandatory module" is not found, install it (see "Perl Dependencies") and try again. You may have to install several mandatory Perl modules before Shepherd can run.
   
   When Shepherd runs for the first time, it will attempt to download and install components (probably into ~/.shepherd/). Most likely, one or more components will fail to install due to missing Perl modules, but ignore this for now.
4. Answer the configuration questions to select your region, desired channels, etc. Make sure your channel XMLTV IDs match those expected by your PVR: for example, if you're a MythTV user and in the mythtv-setup Channel Editor you have "ABC Melbourne" set to an XMLTV ID of "abc.free.au", that's what it needs to be in Shepherd, too.
   
   When asked if you want to create the configuration file, say yes.
5. [optional] Shepherd will now test all its components. If any require configuration (e.g. the oztivo grabber), you will be prompted to do this. You do not have to configure components, and you can do it later: just press enter to skip any questions asked of you.
6. [optional] Shepherd will ask if you want to grab data now. You probably don't, because you haven't installed the optional Perl modules, so say no. Shepherd will exit.
7. [optional] For maximum functionality, install any missing Perl modules required by components. See the detailed instructions and why this is worth doing. Essentially, you want to install any missing Perl modules Shepherd complains about when you do this:

   ~/.shepherd/shepherd --check
   

8. [optional] If you want mythfilldatabase to be able to call Shepherd automatically, symlink it to /usr/bin/tv_grab_au:

   sudo ln -s ~/.shepherd/shepherd /usr/bin/tv_grab_au
   

9. [optional] You may delete the original Shepherd file you downloaded (leaving the newly installed version in ~/.shepherd/).
10. [optional] If you'd like Shepherd to provide you with some channel icons, set these up:

     ~/.shepherd/shepherd --set-icons
    

11. [optional] Install Tor if you wish to increase the speed of some grabbers. (Note: most grabbers don't use Tor, so for most users at this time the benefit is marginal.)

Integration with MythTV

Shepherd's job is to create a file of TV guide data for the next 7 days: by default, ~/.shepherd/output.xmltv. (It also creates a log file at ~/.shepherd/logs/shepherd.log.)

Most users want this guide data to end up inside MythTV. This task is handled by MythTV's mythfilldatabase program. You have two ways of managing it:

1. Specify in mythtv-setup that you want to use tv_grab_au for guide data. (This requires that you have set up the tv_grab_au symlink as described above.) You can also specify here when and how often you want mythfilldatabase to run.
2. Create a cron job to run Shepherd, and feed the output to mythfilldatabase manually with the --file option.

Note that Shepherd should be installed as whichever user is going to run it. If you run mythfilldatabase as root (which is not a good idea) and use option 1 above, then in order for Shepherd to find its config files you need to install it as root, too (despite this also being not a good idea).

Some Useful Commands

A full list of command-line options:

~/.shepherd/shepherd --help

The status screen:

~/.shepherd/shepherd --status

Re-configure Shepherd (preserving old settings except where overridden):

~/.shepherd/shepherd --configure

Check whether any components are missing Perl modules or need to be configured:

~/.shepherd/shepherd --check

Perl Dependencies

Shepherd and its components rely on various third-party Perl modules. Some are mandatory, meaning that Shepherd won't run without them. Most, however, are optional, meaning that Shepherd can still operate without them, but some of its components (including grabbers) won't.

You should install all the optional modules if possible, as this increases Shepherd's efficiency, reliability, and possibly the quality of the guide data it delivers. For more information, see the FAQ.

To see which Perl modules you're missing, do this:

~/.shepherd/shepherd --check

If applicable, follow the distribution-specific instructions for Debian-based, Fedora Core 6, and Gentoo. Alternately you can use the general method.

Debian-Based Distributions

The following command should install the necessary packages:

sudo apt-get install xmltv libxml-simple-perl libjavascript-perl libalgorithm-diff-perl \
                     libgetopt-mixed-perl libcompress-zlib-perl libdata-dumper-simple-perl \
                     libdate-manip-perl liblist-compare-perl libdatetime-format-strptime-perl \
                     libhtml-parser-perl libxml-dom-perl libgd-gd2-perl libdigest-sha1-perl \
                     libarchive-zip-perl libio-string-perl

Fedora Core 6 (FC6)

Providing you have your yum/smart system setup for ATrpms.net (see http://www.atrpms.net/install.html), all of the required modules can be installed with:

sudo yum -y install perl-GD perl-Algorithm-Diff perl-XML-Parser perl-TermReadKey \
  js js-devel perl-XML-XQL perl-IO-String perl-XML-Simple perl-DateTime-Format-Builder \
  perl-List-Compare perl-DateTime-Format-Strptime tor xmltv

cd /tmp
wget http://ftp.mozilla.org/pub/mozilla.org/js/js-1.60.tar.gz
tar xvzf js-1.60.tar.gz
cd js/src
make -f Makefile.ref BUILD_OPT=1
cp jsopcode.tbl /usr/include

cpan JavaScript
cpan
cpan> force install JavaScript

Gentoo

Some (but not all) of the mandatory modules can be installed with:

sudo emerge -av xmltv Algorithm-Diff Digest-SHA1

For the rest, you can use g-cpan (e.g. "sudo g-cpan -i List::Compare"). You may wish to useeix to search for new ebuilds before installing (e.g. "eix -s List::Compare").

Alternately, you can use the non-distribution specific method.

Non-Distribution Specific

The general method to install Perl modules is to do this (as root):

cpan <module>

For example:

cpan List::Compare

This will install the mandatory Perl modules:

cpan XMLTV::Ask Algorithm::Diff Compress::Zlib Cwd Data::Dumper Date::Manip Getopt::Long \
     List::Compare LWP::UserAgent POSIX Digest::SHA1

And this will install the optional ones:

cpan DateTime::Format::Strptime File::Basename File::Path GD HTML::Entities \
     HTML::TokeParser HTML::TreeBuilder IO::File Storable Time::HiRes XML::DOM \
     XML::DOM::NodeList XML::Simple Storable HTTP::Cookies File::Basename \
     LWP::ConnCache GD Digest::MD5 Archive::Zip IO::String \
     DateTime::Format::Strptime JavaScript

(Note: this list may not be up to date. If Shepherd complains about more missing modules, install those with cpan <module> as described above.)

It's safe to run these commands even if you have some modules installed; it will leave any previously-installed modules alone, so long as they're the most recent versions.

Note: XMLTV

XMLTV can also be downloaded from http://files.xmltv.org/.

If you are unsure about whether you have XMLTV installed, run the command perldoc XMLTV and see if there is any documentation on your system. If there is, then it is already installed.

XMLTV version 0.5.44 or later is recommended as it supports HDTV flags.

Note: JavaScript

The JavaScript module enables grabbers to read web pages with embedded Javascript. It can be a difficult module to install on some (non-Debian-based) distributions, but provides access to higher-quality TV guide data.

If you're running a RedHat Fedora system, you can verify whether you have the required Javascript perl module bindings js and js-devel through RPM:

# rpm -q js js-devel
js-1.5-2.fc3
js-devel-1.5-2.fc3

Providing rpm responds with something similar to above, the javascript library files are installed and you can proceed with installing the perl javascript bindings. If rpm indicates the packages are not installed, you will need to install them from a RPM repository. [rpmfind.net] can be useful here:

• http://rpmfind.net/linux/rpm2html/search.php?query=js-devel
• http://rpmfind.net/linux/rpm2html/search.php?query=js

Once you have installed the javascript library files, you can install the Javascript perl bindings from CPAN:

cpan Javascript

Optional Software: Tor

Some grabbers work faster/better if they can operate using The Onion Router (tor) from http://tor.eff.org/. Once Tor is installed, shepherd will automatically find it and start using it. No configuration of Tor is necessary.

Tor is available in RPM form from as http://tor.eff.org/dist/rpm/tor-0.1.1.25-tor.0.rh4_4.i386.rpm and http://dag.wieers.com/packages/libevent/libevent-1.2-1.fc3.rf.i386.rpm

For Debian-based installations, a simple "sudo apt-get install tor" will get things working.

Security

Shepherd automatically checks for new versions of itself and its components, downloads, and executes them. See Security for details and implications.

Troubleshooting

Shepherd works from command-line but not automatically via mythfilldatabase

Make sure you have created the symbolic link from tv_grab_au to Shepherd, and that the following command points to your Shepherd executable:

ls -l `which tv_grab_au`

If it points somewhere else--for example, if you have an old tv_grab_au script from a previous grabber in /usr/local/bin/--you should delete/replace this with the symlink to Shepherd.

Further information may be gleaned by running mythfilldatabase from the command-line and looking for any error messages between the "-----Start/End of XMLTV output----" lines.

Guide data doesn't match right channels inside MythTV

It's important that the XMLTV IDs you assigned to channels in Shepherd match those you assigned in MythTV. It doesn't matter what each XMLTV ID is, just that they match. For example, if in the mythtv-setup Channel Editor you have "ABC Melbourne" set to an XMLTV ID of "abc.free.au", you should also specify this as the XMLTV ID for the ABC channel in Shepherd:

tv_grab_au --configure

mythfilldatabase keeps adding unwanted channels

If you are upgrading from the older channel 9 grabber, and you use mythfilldatabase to fetch the data, you may have to add the '--update' option to ensure mythfilldatabase does not try and add missing channels to your video sources. This can be a problem if you have video sources that receive different sets of channels.

Auto-update fails

If you're manually upgrading from a previous version because we broke auto-update, it may be necessary to delete your ~/.shepherd/ directory.

JavaScript with Suse 10.1 and 10.2

After installing libjs and libjs-devel you will still be missing some files to get 'cpan JavaScript' working please be careful with the following it is more of a hack than a fix.

wget http://ftp.mozilla.org/pub/mozilla.org/js/js-1.60.tar.gz
tar zxvf js-1.60.tar.gz
cd js/src
make -f Makefile.ref BUILD_OPT=1
cp js*.h /usr/include/
cp jsopcode.tbl /usr/include/
cd Linux_All_OPT.OBJ
cp js*.h /usr/include/
cp libjs.* /usr/include/
ldconfig

‘cpan JavaScript’ will still have issues but should be about 86% successful, you can now do a force install by

pc:/ # cpan
cpan> force install JavaScript