Readme: epid

Problem

You record digital television programs and want to archive them, but the broadcaster multicasts, and you're only interested in saving some of the programs (instead of all of them).  The extra programs are just wasted storage.

Solution

This program, "epid" (erase PIDs), will read ATSC transport streams and erase the elementary streams for the PIDs that you specify.  epid doesn't delete the packets, but rather replaces them with null packets.  The output is no smaller than the input, but it will compress better.  (i.e., with some not-lossy compression like ZIP, NTFS compression, tape backup compression, etc.)  This is appropriate for certain hardware decoders like the HiPix, and perhaps some D-VHS players, because they expect packets to be delivered to the decoder at just the right time.  Removing the null packets completely would destroy information about the original spacing in time of the video & audio packets that you care about. 

Some players, like most software players and the MyHD, are happy playing streams with the null packets removed.  These players look at the PCR time stamps in the stream to know when to deliver packets to the decoder.  PCRs, however, are typically spaced 450-1200 packets apart, so it is not possible to reconstruct the original packet spacing perfectly by using the PCRs.  See NullPacketStripper.

Usage

Usage: epid [-s] <input file/pattern> [0xPID] [0xPID] ...
  -s: save input files instead of overwriting them

Specify PIDs in hex, without a leading '0x'.  You can specify as many PIDs as you want.

How to pick PIDs

You should specify the PMT, video, and audio PIDs of the program that you want to remove.

TSReader Lite:  http://www.coolstf.com/tsreader/

bbDMUX:  part of bbTOOLS (try Google)

The convention for most ATSC multiplexers is that program 1 has PMT 0x10, video 0x11, audio 0x14;  program 2 has PMT 0x20 video 0x21, audio 0x24, etc.

Notes and Limitations

For each input file, epid writes output to a temporary file in the same directory as the input. After processing each input file, the input file is replaced with the output file. So files are effectively processed in-place.  This means that you must have at least as much extra storage as the largest file that you will process.

Because epid is reading and writing from the same drive, it uses a large buffer (69 MB) in order
to minimize processing time.  Using an even larger buffer will probably not help performance.  If you don't have 69 MB of free physical RAM, then you should buy more RAM, or use a smaller buffer (at the expense of performance).  The buffer size is defined in the source code.

epid expects all packets to be 188 bytes long, to begin with the sync byte, and to be aligned to 188-byte boundaries.  So if your ATSC recorder splits streams into multiple files, it should start each new file at the beginning of a packet.  The HiPix does this.

epid does not handle large files (>4 GB).

epid does not touch the PAT, so your playback mechanism may still think the deleted programs are there even though the actual data is not.  It is usually not a problem if you remember to remove the PMT of the program that you're removing.

Performance

On my system, a minute's worth of ATSC stream (137 MB) is processed in about 15 seconds.  The bulk of that time is spent reading and writing to/from the disk.  The only way to improve this would be to read and write to different physical disks.  But that would mean that epid would no longer be processing files in-place.

Warning

This code is not guaranteed bug-free.  If you don't want to lose your data, save the source files (-s option) and test the output before deleting the source files.

Contact

e-mail

License

epid is licensed free for personal use.  You may not distribute the source or executable.