FTimes' dig output can be thought of as an index that links DigStrings
(i.e. specific sequences of bytes) to one or more offsets in a given
set of files.  This output is sufficient to answer the questions:
"Which files contain a given DigString?" and "How many times does
a particular DigString appear in a given file?"  At a deeper level,
however, it is often useful to extract some of the context surrounding
a given DigString.  This can help investigators determine whether
or not a particular match is relevant to their line of analysis.
This recipe covers two techniques for extracting dig context.  The
first technique is a manual approach that uses dd to extract context.
The second technique uses ftimes-dig2ctx.pl, included in this recipe,
which is more flexible and built for bulk processing.

This recipe assumes that:

   - all local commands are in the search path and will be executed
     from a Bourne shell;

   - the local operating system is a supported flavor of UNIX;

   - working copies of FTimes and Perl are properly loaded and
     functional on the host where commands will be executed; and

   - the $TMP environment variable exists and is set to '/tmp'.

1. The first approach to extracting dig context assumes that you
   intend to operate on one DigString match at a time.  To produce
   results that are consistent those depicted in this recipe, set
   the $TMP environment variable to '/tmp'.  Using another value
   for this variable will cause variation in the actual filenames
   reported by ftimes.  This is due to the fact that ftimes always
   specifies filenames as full paths.  Set $TMP as follows:

   export TMP=/tmp

   Create a file in $TMP called subject-dig-file-1 having the
   following content.

--- subject-dig-file-1 ---
=987654321hacker123456789=
--- subject-dig-file-1 ---

   The command provided here may be used to extract the preceding
   content into a separate file.  Note that the filename argument
   must match the actual name of this recipe for the sed command
   to work.

   sed -e '1,/^--- subject-dig-file-1 ---$/d; /^--- subject-dig-file-1 ---$/,$d' ftimes-dig-context.txt > $TMP/subject-dig-file-1

2. Use ftimes in dig mode to search $TMP/subject-dig-file-1 for
   the following string: 'hacker'.

   echo "DigString=hacker" | ftimes --digauto - -l 6 $TMP/subject-dig-file-1

   This should produce output similar to the following:

--- dig-output ---
name|offset|string
"/tmp/subject-dig-file-1"|10|hacker
--- dig-output ---

3. Now, extract 8 bytes of context on either side of the DigString.
   To do this, a total 22 bytes must be read from the file starting
   at byte 2.  Extract the dig context with the following command:

   dd if=$TMP/subject-dig-file-1 of=$TMP/dig-context bs=1 skip=2 count=22

   This should produce output similar to the following:

--- dd-output ---
22+0 records in
22+0 records out
22 bytes transferred in 0.001173 secs (18757 bytes/sec)
--- dd-output ---

   At this point, $TMP/dig-context should contain the following
   sequence of bytes: '87654321hacker12345678'.  Use the following
   command or an equivalent technique to inspect the contents of
   this file.

   cat $TMP/dig-context; echo ""

4. The second approach to collecting dig context requires you to
   make use of the following Perl script:

   http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/ftimes/ftimes/tools/dig/ftimes-dig2ctx.pl

   Once you have downloaded the script, make sure that it is
   executable.  If in doubt, run the following command.

   chmod 755 ftimes-dig2ctx.pl

5. Use ftimes-dig2ctx.pl to extract the same dig context as was
   done in step three.  This can be accomplished with the following
   command:

   echo "\"$TMP/subject-dig-file-1\"|10|hacker" | ./ftimes-dig2ctx.pl -h -p 8 -c 22 -f -

   Here, the '-p' argument sets the desired prefix length, and the
   '-c' argument sets the desired context length.  By default, these
   values are 0 and 64 respectively.  The '-h' option causes the
   script to generate a header line.  Executing this command should
   produce output similar to the following:

--- ftimes-dig-context-output-1 ---
dig_name|dig_offset|dig_string|ctx_offset|lh_length|mh_length|rh_length|ctx_string
"/tmp/subject-dig-file-1"|10|hacker|2|8|6|8|87654321hacker12345678
--- ftimes-dig-context-output-1 ---

   The output produced by ftimes-dig2ctx.pl has eight fields.
   These fields are described here:

         dig_name = File from which the context will be extracted
       dig_offset = Offset of DigString in file
       dig_string = DigString (URL encoded)
       ctx_offset = Offset of context in file
        lh_length = Left-hand length of the context
        mh_length = Middle-hand length of the context
        rh_length = Right-hand length of the context
       ctx_string = Context (URL or Hex encoded)
      
6. Two options that make this script more flexible than the manual
   approach are '-l' and '-r'.  These options take Perl regular
   expressions as input.  The specified expressions act as left-
   and right-hand boundaries that limit the amount of context
   returned.  One example of how this might be applied is the case
   where you only want to extract context bounded to the left and
   right by newlines (i.e. '\n').

   To see how this works, first create a file in $TMP called
   subject-dig-file-2 having the following content.

--- subject-dig-file-2 ---
=98765432
1hacker1
23456789=
--- subject-dig-file-2 ---

   The command provided here may be used to extract the preceding
   content into a separate file.  Note that the filename argument
   must match the actual name of this recipe for the sed command
   to work.

   sed -e '1,/^--- subject-dig-file-2 ---$/d; /^--- subject-dig-file-2 ---$/,$d' ftimes-dig-context.txt > $TMP/subject-dig-file-2

   Next, combine steps 2 and 5 into a single command.  Since ftimes
   always generates a header line, the script will need to ignore
   the first line of input.  This can be achieved with the '-i'
   argument.

   echo "DigString=hacker" | ftimes --digauto - -l 6 $TMP/subject-dig-file-2 | ./ftimes-dig2ctx.pl -h -i 1 -p 8 -c 22 -f -
   
   This should produce output similar to the following:

--- ftimes-dig-context-output-2 ---
dig_name|dig_offset|dig_string|ctx_offset|lh_length|mh_length|rh_length|ctx_string
"/tmp/subject-dig-file-2"|11|hacker|3|8|6|8|765432%0a1hacker1%0a234567
--- ftimes-dig-context-output-2 ---

   Observe how the '\n' characters have been URL encoded -- look
   for '%0a'.  This is the script's default encoding scheme.

   Finally, modify the previous command to include right- and
   left-hand boundaries.

   echo "DigString=hacker" | ftimes --digauto - -l 6 $TMP/subject-dig-file-2 | ./ftimes-dig2ctx.pl -h -i 1 -p 8 -c 22 -l "\n" -r "\n" -f -

   This should produce output similar to the following:

--- ftimes-dig-context-output-3 ---
dig_name|dig_offset|dig_string|ctx_offset|lh_length|mh_length|rh_length|ctx_string
"/tmp/subject-dig-file-2"|11|hacker|10|8|6|8|1hacker1
--- ftimes-dig-context-output-3 ---