Digital Video Authenticity: Auditing Frame-Rate Drops and Container Metadata

A digital video file records time in at least three independent layers, and an authentic camera keeps them aligned. Video metadata analysis for an expert witness is largely the discipline of pulling those layers apart and checking that they still agree.

• The container timeline. In MP4 and MOV files the ISO Base Media File Format (ISO/IEC 14496-12) stores per-sample durations in timing tables. This is the file's own claim about how long each frame was shown.
• The codec timeline. The H.264 elementary stream (ITU-T H.264, also MPEG-4 AVC) carries its own presentation and decode timestamps and its own declared frame rate in the sequence headers.
• The visual timeline. The pixels themselves have a cadence. Motion across frames either flows continuously or it stutters, repeats, or jumps.

Original camera output tends to keep these three synchronized because a single firmware pipeline wrote all of them at once. Editing, transcoding, and splicing desynchronize them, because most tools rewrite one layer and reconstruct the others imperfectly. The audit points below are the places where that desynchronization becomes visible and, more importantly, documentable.

A dropped frame is not automatically evidence of tampering, and treating it that way is a liability. Many surveillance DVRs and IP cameras record at a variable frame rate by design. They key capture to detected motion or to available bandwidth, so the raw stream legitimately contains uneven frame spacing. The forensic question is not whether spacing varies, but whether it varies in a pattern consistent with the stated recording system.

The mechanism to examine is the timestamp series. Every coded picture carries a presentation timestamp (PTS) and a decode timestamp (DTS). Extract the per-packet PTS values and compute the delta between consecutive frames. That delta series is the actual cadence, independent of whatever frame rate the file advertises.

• Uniform deltas indicate constant frame rate (CFR) capture. A DVR that claims motion-based recording but shows perfectly uniform deltas is worth questioning.
• Irregular deltas indicate variable frame rate (VFR). This is normal for many camera systems, but the gaps should correlate with quiet scene content, not with points of evidentiary interest.
• Duplicate frames padded to a fixed rate are a re-encode signature. When a tool converts VFR to CFR it inserts repeated frames. A stream that is visually frozen for several frames while the timestamps keep advancing steadily has usually passed through a transcoder.

Cross-check the measured cadence against the frame rate the container declares and against the frame rate the codec headers declare. Three different numbers for the same footage is a finding. Tools such as ffprobe (packet and frame timestamps) and MediaInfo (declared frame-rate mode, CFR or VFR) produce this data in a form you can attach to a report.

The container is a box structure, and each box records a fact you can audit. Do not rely on the operating system's file date. Read the atoms directly with a tool such as ExifTool, MediaInfo, or a hex viewer.

• ftyp box. The major brand and compatible brands identify the muxer. Purpose-built recorders often write manufacturer-specific brands. Generic brands like isom or mp42, or a handler name of Lavf, indicate FFmpeg-family muxing, which is what many editing and conversion tools use. Camera-original footage carrying an editor's muxer signature is a re-processing flag.
• mvhd and mdhd boxes. These hold creation and modification times plus the media timescale. QuickTime-derived timestamps count seconds from a 1904 epoch, so verify your tool is interpreting the epoch correctly before drawing conclusions. A modification time later than creation time, or timestamps that disagree with the chain-of-custody log, is a lead to run down.
• stts (time-to-sample). This table encodes the duration of every frame. It is the container's own record of CFR versus VFR. Compare it against the codec-level cadence from the prior step.
• stss (sync sample table). This lists the keyframe positions. Irregular keyframe placement, or a keyframe forced at a suspicious cut point, is worth correlating with the visual content.

None of these atoms alone proves editing. Together they establish whether the file's structural story matches the device that supposedly produced it.

H.264 divides each frame into 16x16 macroblocks and compresses them in a repeating group-of-pictures (GOP) structure: an intra-coded I-frame (a full keyframe), followed by predicted P-frames and bidirectional B-frames that only encode changes. Understanding this structure is what lets you distinguish ordinary compression from an edit.

• Normal compression artifacts. Blocking along the macroblock grid, ringing near sharp edges, and mosquito noise around moving objects are expected in any compressed video. Their presence is not tampering. Their sudden change in character mid-clip can be.
• Double-compression signatures. Re-encoding footage that was already H.264 leaves periodic artifacts and can shift the macroblock grid relative to the original. A block grid that does not align to a consistent 16-pixel lattice suggests the frame was cropped or resized and re-encoded.
• GOP discontinuities. Splicing two clips almost always forces an unscheduled keyframe at the join, breaking the regular GOP cadence. A keyframe interval that is uniform across the recording except at one point of interest is a classic splice signature.
• Error-concealment carryover. When a decoder hits a corrupted region it copies macroblocks from a reference frame. Regions that visually lag or smear from a prior frame can reflect transmission loss rather than editing, and distinguishing the two protects against a false authenticity claim.

Report these as consistent or inconsistent with a single-encode origin. Avoid stating a conclusion the artifact class cannot support on its own.

Below the container sits the raw elementary stream, organized into NAL units. Two of them define how the whole stream must be decoded, and they are where re-encoding leaves its cleanest fingerprints.

• Sequence Parameter Set (SPS). This carries the profile and level, the coded resolution, and the video usability information (VUI). Inside the VUI, the timing fields num_units_in_tick and time_scale, together with fixed_frame_rate_flag, express the codec's own declared frame rate. Compare this declared rate to the container's stts durations and to the measured PTS cadence. Divergence across those three sources is a structured finding, not an opinion.
• Picture Parameter Set (PPS). This governs entropy coding and slice behavior. SPS or PPS values that change partway through a stream that is supposed to be one continuous recording point to concatenation of differently encoded segments.
• Encoder signatures. Many encoders write an identifying string into SEI user-data NAL units. An x264-family settings string, for example, records the encoder version and configuration. If a segment is offered as raw recorder output but carries a desktop encoder or editing-suite signature, that is a strong lead. Treat it as a lead, not proof, because some recorders embed the same encoder cores. The finding is the mismatch between the claimed source and the embedded signature.

Extract NAL units and parse SPS/PPS with a stream analyzer so the exact field values, not a paraphrase, go into the record.

All of the analysis above is only defensible if it was performed on a bit-for-bit copy of what was received. Establish the cryptographic baseline before you touch the file.

1. Hash on receipt. Compute a SHA-256 value for the file the moment it enters your custody and record it in the log with date, time, and handler. SHA-256 is preferred for court-facing work. MD5 is still encountered and is adequate for detecting accidental corruption, but its known collision weakness makes it a poor sole basis for an adversarial authenticity claim.
2. Work on copies. Verify that your working copy's hash matches the acquisition hash before every analysis session, and record that it matched.
3. Reconcile with the producing party. If the party that produced the footage supplied a hash, compare it to yours. A mismatch means the file you are examining is not the file they attested to, which is itself a finding worth raising early.
4. Preserve the original container. Do not remux or transcode the evidence file to make it play in a preferred viewer. Remuxing rewrites the atoms you need to audit. Analyze the original and document its exact structure first.

This is procurement and buyer education, not legal advice, and no technical finding guarantees admissibility or an outcome. What the analysis does is make an authenticity opinion survivable under the standards that govern expert testimony.

• Daubert v. Merrell Dow Pharmaceuticals asks whether a method is testable, has a known error rate, has been peer reviewed, and is generally accepted. Timestamp extraction, atom parsing, and NAL-unit analysis are reproducible procedures a second examiner can rerun on the same file, which speaks directly to those factors. Federal Rule of Evidence 702 governs the qualification and reliability of the testimony itself, and some jurisdictions still apply the Frye general-acceptance standard.
• Authentication under Federal Rule of Evidence 901 requires evidence sufficient to support a finding that the item is what its proponent claims. A documented match between the acquisition hash, the container structure, the codec headers, and the visual cadence is the kind of showing that supports authentication of a recording.
• Self-authentication provisions at Federal Rule of Evidence 902(13) and 902(14) address electronic records and data copied from a device identified by a reliable digital process, which is where the hash discipline in the prior section carries weight.
• SWGDE (Scientific Working Group on Digital Evidence) publishes best-practice documents for forensic video and image analysis. Aligning your procedure to a recognized practice document strengthens the general-acceptance and reliability elements.

Frame the report as a set of consistencies and inconsistencies with a single-source origin, each traceable to a named field or measurement. That structure is what lets counsel and the retained expert defend it on cross-examination without overclaiming.