Metadata

Metadata that travels through your OptiView Live stream — whether pushed via the API or embedded as SEI — surfaces on the player as cues on a text track. The general approach is the same in every case:

1. Listen for addtrack on player.textTracks so you find out when a new metadata track shows up.
2. Subscribe to a cue event on that track.
3. Inspect the cue to make sure it carries the metadata you care about.
4. Read the payload from cue.content.

The properties you match on, the cue event you subscribe to, and the shape of cue.content all depend on which kind of metadata is being delivered.

User-data-unregistered SEI and API push

UDU SEI and API-pushed payloads surface as ID3 PRIV frames inside an ID3 v2.4 tag embedded in an emsg box, carried by the CMAF segments of the stream.

• The owner identifier of the PRIV frame will be set to: optiview.live:meta:<your-uuid>
• The frame's private data will contain your metadata as raw binary data.

(See section 4.27 of the ID3 v2.4 Native Frames specification and ID3 in CMAF for more details.)

The OptiView Player iOS SDK exposes these ID3 frames on a text track with TextTrack.type equal to "id3". Listen for the cue event that matches when you want to react:

• TextTrackEventTypes.ADD_CUE fires as soon as a cue becomes available on the track, before playback reaches it — useful when you want to prefetch data or update application state ahead of time.
• TextTrackEventTypes.ENTER_CUE fires when playback enters the cue — useful when you want to act in sync with the video, for example to render an overlay at the moment the cue was inserted.

The ID3 frame will be contained in cue.contentDictionary with id equal to "PRIV" and ownerIdentifier equal to optiview.live:meta:<your-uuid>.

important

Make sure to always check the frame's id and ownerIdentifier before using it, to avoid processing unrelated ID3 frames.

Suppose your UUID is 11111111-1111-1111-1111-111111111111:

_ = theoplayer.textTracks.addEventListener(type: TextTrackListEventTypes.ADD_TRACK, listener: { (addTrackEvent) in
  guard let track = addTrackEvent.track as? TextTrack,
        track.type == "id3"
        else { return }
  // ID3 track was added. Listen for incoming ID3 cues.
  _ = track.addEventListener(type: TextTrackEventTypes.ADD_CUE, listener: { (addCueEvent) in
      guard let cue = addCueEvent.cue as? Id3Cue,
            let frame = cue.contentDictionary else { return }
      guard frame["id"] == "PRIV",
            frame["ownerIdentifier"] == "optiview.live:meta:11111111-1111-1111-1111-111111111111"
            else { return }
      print("Metadata: \(frame["data"])")
  })
})

Here the frame["data"] carries the binary payload exactly as it was sent — your application is responsible for parsing it (for example UTF-8 JSON, protobuf, or your own format).

Source timecode

When the video you ingest carries SMPTE timecode in its picture timing SEI, the Low-Latency HLS and HLS (CMAF) output passes it on as ID3 timed metadata, so players can read the timecode of every frame without parsing the video bitstream. No channel configuration is needed. Channels whose source carries no timecode get no timecode events, and HLS (MPEG-TS) output does not carry them.

Timecode events travel the same way as the metadata above: an ID3 v2.4 tag in an emsg box with scheme https://aomedia.org/emsg/ID3, carried by the CMAF segments of the stream. The tag holds a TXXX (user defined text) frame:

• The description of the TXXX frame is optiview.live:timecode.
• Its value is the timecode as HH:MM:SS:FF, for example 10:00:00:12. A drop-frame timecode has ; before the frames instead: 10:00:00;12.

(See section 4.2.6 of the ID3 v2.4 Native Frames specification for the TXXX frame.)

Where the events sit

There is one timecode event per segment and per partial segment. The event sits on the frame of that segment or partial segment that is presented first, and its emsg box declares a duration of exactly one frame. OptiView Player starts the cue at that frame's presentation time and keeps it active until the next timecode cue starts.

A segment and its first partial segment open on the same frame, so they carry the same event, with the same emsg id. If your player reports it twice, the two cues are identical.

Timecode of any other frame

Only the first-presented frame of each segment or partial segment carries an event. To find the timecode of any other frame:

1. Take the latest timecode event that starts at or before the presentation time of the frame.
2. Count the frames presented since that event: multiply the time between the start of the event and the presentation time of the frame by the frame rate, which the multivariant playlist declares for each rendition as FRAME-RATE.
3. Add that many frames to the timecode of the event, counting with the frame rate rounded to a whole number (30 for 29.97 fps). For a drop-frame timecode, skip the frame numbers that drop-frame timecode leaves out: frames 00 and 01 at the start of every minute except every tenth minute at 29.97 fps, and frames 00 to 03 at 59.94 fps.

This assumes the rendition has the frame rate of the source. Every frame carries the timecode of the source frame it was encoded from, so on a rendition with a lower frame rate than the source, such as a 30 fps rendition of a 60 fps source, count at the frame rate of the source instead.

A jump in the source timecode, for example when the source switches to another timecode generator, only shows from the next segment or partial segment on. Until then, counting on from the previous event continues the old sequence.

Reading the timecode with the iOS SDK

Timecode events surface on the same "id3" text track as the metadata above, as an Id3Cue whose contentDictionary has id equal to "TXXX" and description equal to "optiview.live:timecode". The frame's text holds the timecode.

TextTrackEventTypes.ENTER_CUE fires when the frame carrying the timecode is presented, and TextTrackEventTypes.EXIT_CUE when the next timecode cue starts:

_ = theoplayer.textTracks.addEventListener(type: TextTrackListEventTypes.ADD_TRACK, listener: { (addTrackEvent) in
  guard let track = addTrackEvent.track as? TextTrack,
        track.type == "id3"
        else { return }
  _ = track.addEventListener(type: TextTrackEventTypes.ENTER_CUE, listener: { (enterCueEvent) in
      guard let frame = (enterCueEvent.cue as? Id3Cue)?.contentDictionary,
            frame["id"] == "TXXX",
            frame["description"] == "optiview.live:timecode"
            else { return }
      print("Timecode: \(frame["text"] ?? "")")
  })
  _ = track.addEventListener(type: TextTrackEventTypes.EXIT_CUE, listener: { (exitCueEvent) in
      guard let frame = (exitCueEvent.cue as? Id3Cue)?.contentDictionary,
            frame["id"] == "TXXX",
            frame["description"] == "optiview.live:timecode"
            else { return }
      print("Timecode ended: \(frame["text"] ?? "")")
  })
})