2009-11-20

MythTV: Transcode to XviD and Add to Library

Before starting with MythTV, I already had a library of several TV series that I've been serving to an Xbox running XBMC over Samba, so I wanted the ability to configure certain shows to automatically be transcoded to XviD and moved into my library directory structure (below) and named with season number, episode number, and episode name. MythTV doesn't get the season and episode number from TV listings (at least not through Schedules Direct), so I found a Perl module to query The TVDB and cache results for me. I have some ideas on ways to improve it, but for now it works well enough.
Series Name/Season 00/Series Name - s00e00 - Episode Name.avi

Update

nuvexport seems to have been removed recently, so I'm looking into using mythnuv2mkv to do the transcoding.

Older versions

The script requires lots of modules, so use the cpan command-line tool on linux to install the following if you don't already have them.
  • Getopt::Std
  • File::Path
  • File::Copy
  • TVDB::API
Make sure to read the source up to # no more changes needed beyond, because the script depends on the nuvexport script for transcoding, and some settings matching those in your nuvexportrc and MythTV settings. There are a couple of command-line options to retry TVDB lookups, so I have the following User Jobs configured in MythTV:
  • Transcode:
    /usr/bin/mythtv_export.pl "%FILE%" "%TITLE%"
  • Move to Archive:
    /usr/bin/mythtv_export.pl -r "%FILE%" "%TITLE%"
The first job I configure to run for all recording schedules that I want stored permanently in the library. The second I just have for error recovery or when The TVDB is down and I need to rerun the info retrieval and move. If The TVDB is available but it's unable to uniquely identify the season and episode, it will still move the XviD into the TV series root directory.

NOTE: Updated to check for errors in failed encodes and better episode matching. It has not been tested with the latest version of MythTV.
#!/usr/bin/perl -W
#
# Runs nuvexport, looks up season and episode number for TV series
# from http://thetvdb.com
#
# TheTVDB calls themselves "An open database for television fans"
# and they request that all users of their data
# "help contribute information and artwork if possible."
#
# This script was written against TVDB::API v0.33, a perl module
# written by Behan Webster (behanw AT websterwood DOT com)
# which caches to minimize connections to TheTVDB
#
# Requires nuvexport
# Edit nuvexportrc to define settings for transcoding
#
# Requires mplayer
# For checking encode output for errors
#
# !!!
# DEPENDS ON filename=%T - %S - %oY-%om-%od in nuvexportrc
# !!!
#
# Invoke from mythtv user job with the following line:
# /path/to/mythtv_export.pl "%FILE%" "%TITLE%"
#
# * Loosely based on the mythexport shell script written by Kyle Hill *
#

# Location of nuvexport
# This is the default for the Ubuntu nuvexport package
my $nuvexport = "/usr/bin/nuvexport";

# Output directory as configured in nuvexportrc: "path"
my $tempdir = "YOUR SETTING";

# Final destination base directory
my $destdir = "YOUR LIBRARY";

# TVDB Cache directory
my $tvdbCache = "/tmp/mythtv_export";

# TVDB API key
# Go to http://thetvdb.com/?tab=apiregister to get a key
my $tvdbAPIKey = "YOUR API KEY";

# Series name map
# Some series that share the name of an older series
#  can't be identified by name alone, so specify the
#  string to use for lookups here
# The first element is the series name as MythTV knows it;
#  the second is the series name at The TVDB
my %seriesNames = ( 
  "The Office", "The Office (US)" 
);

#
# no more changes needed beyond
#

use strict;
use Getopt::Std;
use File::Path;
use File::Copy qw(move);
use TVDB::API;

my %args = ();
getopts( "hdrn", \%args );
my $help  = defined $args{h} ? $args{h} : "";
my $retry = defined $args{r} ? $args{r} : "";
my $noop  = defined $args{n} ? $args{n} : "";
my $filename = shift(@ARGV);
my $title    = shift(@ARGV);

my $usage = <<USAGE;
 Usage: $0 [-h] [-r] [-n] RECORDING_FILENAME SHOW_TITLE  
  -h: print this message and exit  
  -r: retry naming (skip transcode)  
  -n: no-op (skip transcode, move)  
 USAGE  
   
 die $usage if $help || !$filename || !$title;  
   
 #  
 # Transcode the file with nuvexport  
 #  
 my $transcodeCmd = "$nuvexport --infile \"$filename\" --noprogress";  
   
my $transcodeFailed = 0;

if (!$noop && !$retry) {

    print "Executing $transcodeCmd\n";

    if (system($transcodeCmd) != 0) {
 print "Transcode failed.\n";
 $transcodeFailed = 1;
    }
    else {
 print "Transcode complete\n";
    }

}
else {
    print "Skipping transcode\n";
}

my $outputFile = $filename;
if (!$outputFile) {

    #
    # identify the newly transcoded file in the temp dir
    # - just get the newest file
    #

    my @files = `ls -1tr \"$tempdir/$title\"*.avi`
      or die "Can't find the transcoded file!\n";
    $outputFile = $files[$#files];
    chomp($outputFile);

}

print "Found $outputFile\n";

#
# first look for errors in the avi on failure
#

if ($transcodeFailed) {
    print "Scanning file for errors.\n";

    my $scanOutputFile = $outputFile;
    $scanOutputFile =~ s/\.avi$/\.out/;

    my $mplayerCmd = "nice mplayer -nojoystick -nolirc -nomouseinput -vo null -ao null -speed 10 \"$outputFile\" 2>&1 | tr '\r' '\n' >\"$scanOutputFile\" 2>&1";

    print "Executing $mplayerCmd\n";

    system($mplayerCmd) == 0
      or die "mplayer scan failed: $?";

    my $lineCount = `wc -l \"$scanOutputFile\" 2>/dev/null`;
    $lineCount =~ s/^(\d+)\w.*$/$1/;

    if ($lineCount < 1000) {
 die "mplayer output is fewer than 1k lines.";
    }

    my $errorCount = `egrep -ic 'error|skip|damaged|overflow' \"$scanOutputFile\"`;
    if ($errorCount > 15) {
 die "Too many errors: $errorCount";
    }

    print "No errors found. Cleaning up scan output.\n";

    unlink($scanOutputFile)
      or die "Failed! $!\n";

}

#
# Look up season & episode number
#

# parse out the episode name and air date
$outputFile =~ /$title - (.*) - (\d{4}-\d{2}-\d{2})\.(.*)$/
  or die "Couldn't parse $outputFile! "
  . "The following setting is required in nuvexportrc:\n"
  . "filename=%T - %S - %oY-%om-%od\n";
my $episodeName   = $1;
my $airDate       = $2;
my $ext           = $3;
my $seasonNumber  = "";
my $episodeNumber = "";
my $matchName     = $episodeName;

print "Series=$title\nEpisode name=\"$episodeName\"\nAir date $airDate\n"
  if $retry or $noop;  

# make sure cache dir exists
if (!-e $tvdbCache) {
    print "Making $tvdbCache\n";
    mkpath($tvdbCache)
      or die "Failed! $!\n";
}

# TVDB::API init
my $tvdb = TVDB::API::new($tvdbAPIKey, "en");
$tvdb->setCacheDB("$tvdbCache/tvdb.db");
my $mirrors = $tvdb->getAvailableMirrors();
$tvdb->chooseMirrors();

my $match = 0;

#
# TVDB::API doesn't handle two episodes in one day; it just returns the first, so:
# 1. lookup episode by air date to get season
# 2. get episode ids for the entire season
# 3. iterate over episodes, match on name
# 4. if name match fails, fall back to air date match
#
# TODO: improve TVDB::API's treatment of multiple episodes for one day
#

if ($seriesNames{$title}) {
    $title = $seriesNames{$title};
    print "Switching to series name \"$title\"\n";
}

print "looking up $title\n";

# look up episode by air date
my @episodesByDate = $tvdb->getEpisodeByAirDate($title, $airDate);

if (@episodesByDate) {

    # get season
    foreach my $element (@episodesByDate) {

        # TODO: research reason for arrays, better data retrieval than loops?
        foreach my $episode (@$element) {
            $match         = 1;
            $seasonNumber  = $episode->{SeasonNumber};
            $episodeNumber = $episode->{EpisodeNumber};
            $matchName     = $episode->{EpisodeName};
            last;
        }

    }

}

# we identified the season
if ($seasonNumber) {

    my $nameMatch = 0;

    # get all episode ids for the season
    my @episodesBySeason = $tvdb->getSeason($title, $seasonNumber);

    # look for the episode by name
    foreach my $element (@episodesBySeason) {

        # TODO: research reason for arrays, better data retrieval than loops?
        foreach my $episodeId (@$element) {

            if ($episodeId) {

                my $episode = $tvdb->getEpisodeId($episodeId);
                my $epname  = $episode->{EpisodeName};
                $epname =~ s/^\s+//;
                $epname =~ s/\s+$//;

                print "$episode->{EpisodeNumber} - \"$epname\"\n"
                  if $retry
                      or $noop;

                # compare the name, both upper cased
                if (uc($epname) eq uc($episodeName)) {
                    print "Matched!\n";
                    $nameMatch     = 1;
                    $match         = 1;
                    $episodeNumber = $episode->{EpisodeNumber};
      $matchName     = $episode->{EpisodeName};
                    last;
                }

            }

        }

    }

    if (!$nameMatch) {
        print
"Unable to identify episode within season $seasonNumber! Falling back to air date match.\n";
    }

}
else {
    die "Unable to identify episode by air date!\n";
}

# build the final filename
my $destPath = "$destdir/$title";
$destPath .= "/Season " . sprintf("%02d", $seasonNumber) if $match;

# make sure the dest path exists
if (!-e $destPath) {
    print "Making $destPath\n";
    my $mask = umask();
    my $retval = mkpath($destPath, {mode => 0775});
    umask($mask);
    die "Failed! $!\n" unless $retval;
}

my $finalFilename .= "$title - ";
$finalFilename .= sprintf("s%02de%02d", $seasonNumber, $episodeNumber) . " - "
  if $match;
$finalFilename .= "$matchName.$ext";

print "Moving $outputFile to $destPath/$finalFilename\n";

if (!$noop) {

    # move the file to the destination or delete if it already exists
    if (!-e "$destPath/$finalFilename") {

        move("$outputFile", "$destPath/$finalFilename")
          or die "Move failed! $!\n";

    }
    else {
        print "This episode already exists! Deleting $outputFile.\n";

        unlink($outputFile)
          or die "Failed! $!\n";
    }

}
else {
    print "Skipping due to no-op\n";
}

2 comments:

  1. Hey,

    Cool script, have you done any work to it lately?

    ReplyDelete
  2. I have, and I'll update the post soon. Thanks for the reminder!

    ReplyDelete