This document defines a set of ECMAScript APIs in WebIDL to allow media to be sent and received from another browser or device implementing the appropriate set of real-time protocols. However, unlike the WebRTC 1.0 API, Object Real-Time Communications (ORTC) API does not mandate a media signaling protocol or format. As a result, ORTC does not utilize Session Description Protocol (SDP), nor does it mandate support for the Offer/Answer state machine. Instead, ORTC focuses on "sender", "receiver" and "transport" objects, which have "capabilities" describing what they are capable of doing, as well as "parameters" which define what they are configured to do. "Tracks" and "data channels" are sent over the transports, between senders and receivers.
Object Real-Time Communications (ORTC) provides a powerful API for the development of WebRTC based applications. ORTC does not mandate a media signaling protocol or format (as the current WebRTC 1.0 does by mandating SDP Offer/Answer). Instead, ORTC focuses on "sender", "receiver" and "transport" objects, which have "capabilities" describing what they are capable of doing, as well as "parameters" which define what they are configured to do. "Tracks" and "data channels" are sent over the transports, between senders and receivers.
This specification defines several objects: RTCDtlsTransport (Section 2), RTCIceTransport (Section 3), RTCIceTransportController (Section 4), RTCIceListener (Section 5), RTCRtpSender (Section 6), RTCRtpReceiver (Section 7), RTCRtpListener (Section 8), RTCDtmfSender (Section 10), RTCDataChannel (Section 11), and RTCSctpTransport (Section 12). RTP dictionaries are described in Section 9, the Statistics API is described in Section 13, the Identity API is described in Section 14, an event summary is provided in Section 15, and WebRTC 1.0 compatibility issues are discussed in Section 16.
In a Javascript application utilizing the ORTC API, the relationship between the application and the objects, as well as between the objects themselves is shown below. Horizontal or slanted arrows denote the flow of media or data, whereas vertical arrows denote interactions via methods and events.
The EventHandler
interface represents a callback used for event handlers as defined in
[[!HTML5]].
The concepts queue a task and fires a simple event are defined in [[!HTML5]].
The terms event, event handlers and event handler event types are defined in [[!HTML5]].
The terms MediaStream and MediaStreamTrack are defined in [[!GETUSERMEDIA]].
For Scalable Video Coding (SVC), the terms single-session transmission (SST) and multi-session transmission (MST) are defined in [[RFC6190]]. This specification only supports SST but not MST. The term SST-SS refers to an SVC implementation that transmits all layers within a single RTP session, using a single SSRC. The term SST-MS refers to an SVC implementation that transmits all layers within a single RTP session, using a distinct SSRC for each layer.
The RTCDtlsTransport object includes information relating to Datagram Transport Layer Security (DTLS) transport.
An RTCDtlsTransport
instance is associated to an RTCRtpSender
,
an RTCRtpReceiver
, or an RTCSctpTransport
.
A RTCDtlsTransport
instance is constructed with an RTCIceTransport
object.
The associated RTCIceTransport
instance.
The current state of the DTLS transport.
Obtain the DTLS parameters of the local RTCDtlsTransport
.
Obtain the current DTLS parameters of the remote RTCDtlsTransport
.
Obtain the certificates used by the remote peer.
Start DTLS transport negotiation with the parameters of the remote DTLS transport;
then once the DTLS transport session is established, negotiate a DTLS-SRTP [[!RFC5764]] session to protect media.
Since symmetric RTP [[!RFC4961]] is utilized, the DTLS-SRTP session is bidirectional.
If remoteParameters
is invalid, throw an InvalidParameter
exception.
Stops and closes the DTLS transport object. stop()
. is final - calling start()
afterwards
will throw an InvalidState
exception.
This event handler, of event handler type dtlsstatechange
uses the RTCDtlsStateChangedEvent
interface.
It MUST be supported by
all objects implementing the RTCDtlsTransport
interface.
It is called any time the RTCDtlsTransportState changes.
This event handler, of event handler type error
,
MUST be supported by all objects implementing the RTCDtlsTransport
interface.
This event MUST be fired on reception of a DTLS alert.
The RTCDtlsParameters object includes information relating to DTLS configuration.
The DTLS role, with a default of auto.
Sequence of fingerprints.
The RTCDtlsFingerprint object includes the hash function algorithm and certificate fingerprint as described in [[!RFC4572]].
One of the the hash function algorithms defined in the 'Hash function Textual Names' registry, initially specified in [[!RFC4572]] Section 8.
The value of the certificate fingerprint in lowercase hex string as expressed utilizing the syntax of 'fingerprint' in [[!RFC4572]] Section 5.
RTCDtlsRole indicates the role of the DTLS transport.
The DTLS role is determined based on the resolved ICE role: the 'Controlled' role acts as the DTLS client, the 'Controlling' role acts as the DTLS server.
The DTLS client role.
The DTLS server role.
RTCDtlsTransportState indicates the state of the DTLS transport.
The RTCDtlsTransport
object has been created and has not started negotiating yet.
DTLS is in the process of negotiating a secure connection.
DTLS has completed negotiation of a secure connection (including DTLS-SRTP).
The DTLS connection has been closed intentionally or as the result of an error.
The dtlsstatechange
event of the RTCDtlsTransport
object uses
the RTCDtlsTransportStateChangedEvent
interface.
Firing an
RTCDtlsTransportStateChangedEvent
event named
e with an RTCDtlsTransportState
state means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCDtlsTransportStateChangedEvent
interface with the
state
attribute set to the new RTCDtlsTransportState,
MUST be
created and dispatched at the given target.
The state
attribute is the new
RTCDtlsTransportState
that caused the event.
The state
attribute is the new
RTCDtlsTransportState
that caused the event.
The RTCIceTransport includes information relating to Interactive Connectivity Establishment (ICE).
An RTCIceTransport
instance is associated to a transport object (such as RTCDtlsTransport
),
and provides RTC related methods to it.
An RTCIceTransport
instance is constructed without any arguments.
The iceListener is set to the value of listener passed in the latest call to start()
.
The current role of the ICE transport.
The component-id of the RTCIceTransport
.
The current state of the ICE transport.
Retrieve the sequence of candidates associated with the remote
RTCIceTransport
. Only returns the candidates previously
added using setRemoteCandidates
or addRemoteCandidate
.
Retrieves the nominated candidate pair. If there is no nominated pair yet, NULL is returned.
The first time start
is called candidate connectivity checks are started and the ICE transport attempts to connect to the remote
RTCIceTransport
.
If start
is called with invalid parameters,
throw an InvalidParameters
exception.
If start
is called again, an ICE restart occurs, all remote candidates are flushed, and addRemoteCandidates
or
setRemoteCandidates
must be called to add the remote candidates back or replace them.
Stops and closes the current object. Also removes the object from the RTCIceTransportController
.
Obtain the current ICE parameters of the remote RTCIceTransport
.
Create an associated RTCIceTransport
for RTCP.
If called more than once for the same
component, throw an InvalidStateError
exception. If called when
component is "RTCP",
throw a SyntaxError
exception.
Add remote candidate associated with remote RTCIceTransport
.
Set the sequence of candidates associated with the remote RTCIceTransport
.
This event handler, of event handler type icestatechange
,
uses the RTCIceTransportStateChangedEvent
interface.
It MUST be supported by
all objects implementing the RTCIceTransport
interface.
It is called any time the RTCIceTransportState
changes.
This event handler, of event handler type icecandidatepairchange
,
uses the RTCIceCandidatePairChangedEvent
interface.
It MUST be supported by
all objects implementing the RTCIceTransport
interface.
It is called any time the nominated RTCIceCandidatePair
changes.
RTCIceComponent contains the component-id of the RTCIceTransport
, which will be "RTP" unless RTP and RTCP are not multiplexed and
the RTCIceTransport
object was returned by createAssociatedTransport()
.
The RTP component ID, defined (as '1') in [[!RFC5245]] Section 4.1.1.1. Protocols multiplexed with RTP (e.g. SCTP data channel) share its component ID.
The RTCP component ID, defined (as '2') in [[!RFC5245]] Section 4.1.1.1.
The RTCIceParameters object includes the ICE username and password.
ICE username.
ICE password.
RTCIceRole contains the current role of the ICE transport.
controlling state
controlled state
RTCIceTransportState represents the current state of the ICE transport.
The ICE Transport is gathering addresses and/or waiting for remote candidates to be supplied.
The ICE Transport has received at least one remote candidate, and is checking candidate pairs but has not yet found a connection. In addition to checking, it may also still be gathering.
The ICE Transport has found a usable connection, but is still checking other candidate pairs to see if there is a better connection. It may also still be gathering.
The ICE Transport has finished gathering and checking and found a connection.
Liveness checks have failed. This may trigger intermittently (and resolve itself without action).
The ICE Transport has shut down and is no longer responding to STUN requests.
The non-normative ICE state transitions are:
The icestatechange
event of the RTCIceTransport
object uses
the RTCIceTransportStateChangedEvent
interface.
Firing an
RTCIceTransportStateChangedEvent
event named
e with an RTCIceTransportState
state means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCIceTransportStateChangedEvent
interface with the
state
attribute set to the new RTCIceTransportState,
MUST be
created and dispatched at the given target.
The state
attribute is the new
RTCIceTransportState
that caused the event.
The state
attribute is the new
RTCIceTransportState
that caused the event.
The icecandidatepairchange
event of the RTCIceTransport
object uses
the RTCIceCandidatePairChangedEvent
interface.
Firing an
RTCIceCandidatePairChangedEvent
event named
e with an RTCIceCandidatePair
pair means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCIceCandidatePairChangedEvent
interface with the
pair
attribute set to the nominated RTCIceCandidatePair,
MUST be
created and dispatched at the given target.
The pair
attribute is the nominated
RTCIceCandidatePair
that caused the event.
The pair
attribute is the nominated
RTCIceCandidatePair
that caused the event.
The RTCIceServer is used to provide STUN or TURN server configuration. In network topologies with multiple layers of NATs, it is desirable to have a STUN server between every layer of NATs in addition to the TURN servers to minimize the peer to peer network latency.
An example of an array of
RTCIceServer
objects:
[ { urls: "stun:stun1.example.net } , { urls:"turn:turn.example.org", username: "user", credential:"myPassword"} ]
STUN or TURN URI(s) as defined in [[!RFC7064]] and [[!RFC7065]]
If this RTCIceServer
object represents a TURN server, then this attribute specifies
the username to use with that TURN server.
If the uri element is a TURN URI, then this is the credential to use with that TURN server.
The RTCIceCandidate object includes information relating to an ICE candidate.
{
foundation: "abcd1234",
priority: 1694498815,
ip: "192.0.2.33",
protocol: "udp",
port: 10000,
type: "host"
};
A unique identifier that allows ICE to correlate candidates that appear on multiple RTCIceTransport
s.
The assigned priority of the candidate. This is automatically populated by the browser.
The IP address of the candidate.
The protocol of the candidate (UDP/TCP).
The port for the candidate.
The type of candidate.
The type of TCP candidate.
For candidates that are derived from others, such as relay or reflexive candidates, the relatedAddress refers to the host candidate that these are derived from. For host candidates, the relatedAddress is set to the empty string.
For candidates that are derived from others, such as relay or reflexive candidates, the relatedPort refers to the host candidate that these are derived from. For host candidates, the relatedPort is null.
The RTCIceProtocol includes the protocol of the ICE candidate.
A UDP candidate, as described in [[!RFC5245]].
A TCP candidate, as described in [[!RFC6544]].
The RTCIceTcpCandidateType includes the type of the ICE TCP candidate, as described in [[!RFC6544]].
An active TCP candidate is one for which the transport will attempt to open an outbound connection but will not receive incoming connection requests.
A passive TCP candidate is one for which the transport will receive incoming connection attempts but not attempt a connection.
An so candidate is one for which the transport will attempt to open a connection simultaneously with its peer.
The RTCIceCandidateType includes the type of the ICE candidate.
A host candidate.
A server reflexive candidate.
A peer reflexive candidate.
A relay candidate.
The RTCIceCandidateComplete is an empty dictionary signifying that all RTCIceCandidate
s are gathered.
The RTCIceCandidatePair contains the currently selected ICE candidate pair.
The local ICE candidate.
The remote ICE candidate.
The icecandidate
event of the RTCIceTransport
and RTCIceListener
objects uses
the RTCIceTransportEvent
interface.
Firing an
RTCIceTransportEvent
event named
e with an RTCIceGatherCandidate
candidate means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCIceTransportEvent
interface with the
candidate
attribute set to the new ICE candidate,
MUST be
created and dispatched at the given target.
The candidate
attribute is the
RTCIceGatherCandidate
object with the new ICE
candidate that caused the event.
If candidate
is of type RTCIceCandidateComplete
,
there are no additional candidates.
The ICE candidate that caused the event.
// Assume we already have a way to signal. This is an example
// of how to offer ICE and DTLS parameters and ICE candidates and
// get back ICE and DTLS parameters and ICE candidates, and start
// both ICE and DTLS, assuming that RTP and RTCP are multiplexed.
function initiate(mySignaller) {
var gatherOptions = new RTCIceGatherOptions;
gatherOptions.gatherPolicy = RTCIceGatherPolicy.all;
gatherOptions.iceservers = ... ;
var iceListener = new RTCIceListener(gatherOptions);
var ice = new RTCIceTransport();
var dtls = new RTCDtlsTransport(ice);
// ... get tracks and RTP objects from other example
mySignaller.mySendInitiate({
"ice": iceListener.getLocalParameters(),
"dtls": dtls.getLocalParameters(),
// ... include RTP info from other example
}, function(remote) {
// Start the ICE transport
ice.start(iceListener, remote.ice, RTCIceRole.controlling);
dtls.start(remote.dtls);
// ... start RTP senders and receivers from other example
});
iceListener.onlocalcandidate = function(candidate) {
mySignaller.mySendLocalCandidate(candidate);
}
mySignaller.onRemoteCandidate = function(candidate) {
ice.addRemoteCandidate(candidate);
}
}
// Assume we already have a way to signal and remote info is
// signalled to us. This is an example of how to answer with ICE and DTLS
// and DTLS parameters and ICE candidates and start both ICE and DTLS,
// assuming that RTP and RTCP are multiplexed.
//
function accept(mySignaller, remote) {
var gatherOptions = new RTCIceGatherOptions;
gatherOptions.gatherPolicy = RTCIceGatherPolicy.all;
gatherOptions.iceservers = ... ;
var iceListener = new RTCIceListener(gatherOptions);
var ice = new RTCIceTransport();
var dtls = new RTCDtlsTransport(ice);
// ... get tracks and RTP objects from other example
iceListener.onlocalcandidate = function(candidate) {
mySignaller.mySendLocalCandidate(candidate);
}
mySignaller.onRemoteCandidate = function(candidate) {
ice.addRemoteCandidate(candidate);
}
mySignaller.mySendAccept({
"ice": iceListener.getLocalParameters(),
"dtls": dtls.getLocalParameters()
// ... include RTP info from other example
});
// Start the ICE transport with an implicit gather policy of "all"
ice.start(iceListener,remote.ice, RTCIceRole.controlled);
dtls.start(remote.dtls);
// ... start RTP senders and receivers from other example
}
The RTCIceTransportController object assists in the managing of ICE freezing and bandwidth estimation.
An RTCIceTransportController
object provides methods to add and retrieve RTCIceTransport
objects with a component of "RTP".
An RTCIceTransportController
instance is automatically constructed.
Returns the RTCIceTransport
objects with a component
of "RTP".
Adds transport to the RTCIceTransportController
object for the purposes of managing
ICE freezing and sharing bandwidth estimation.
Since addTransport
manages ICE freezing, candidate pairs that are not in the
frozen state maintain their state when addTransport(transport)
is called.
RTCIceTransport
objects will be unfrozen
according to their index. transport is inserted at index,
or at the end if index is not specified.
If index is greater than the current number of RTCIceTransport
s
with a component of "RTP",
throw an InvalidParameter
exception.
If transport has already been added to another
RTCIceTransportController
object, or if
the component is "RTCP", throw an InvalidStateError
exception.
// This is an example of how to utilize distinct ICE transports for Audio and Video
// As well as for RTP and RTCP. If both sides can multiplex audio/video
// and/or RTP/RTCP then the multiplexing will occur.
//
// Assume we have an audioTrack and a videoTrack to send.
//
//create the RTP and RTCP ICE listeners for audio and video
var gatherOptions = new RTCIceGatherOptions;
gatherOptions.gatherPolicy = RTCIceGatherPolicy.all;
gatherOptions.iceservers = ... ;
var audioRtpIceListener = new RTCIceListener(gatherOptions);
var audioRtcpIceListener = new RTCIceListener(gatherOptions);
var videoRtpIceListener = new RTCIceListener(gatherOptions);
var videoRtcpIceListener = new RTCIceListener(gatherOptions);
//create the RTP and RTCP ICE transports for audio and video
var audioRtpIceTransport = new RTCIceTransport();
var audioRtcpIceTransport = audioRtpIceTransport.createAssociatedTransport();
var videoRtpIceTransport = new RTCIceTransport();
var videoRtcpIceTransport = audioRtpIceTransport.createAssociatedTransport();
//
// Prepare the audio and video ICE transports
audioRtpIceListener.onlocalcandidate = function (event) {sendLocalCandidate(audioRtpIceTransport, event.candidate,"audio")};
audioRtcpIceListener.onlocalcandidate = function (event) {sendLocalCandidate(audioRtcpIceTransport, event.candidate,"audio")};
videoRtpIceListener.onlocalcandidate = function (event) {sendLocalCandidate(videoRtpIceTransport, event.candidate,"video")};
videoRtcpIceListener.onlocalcandidate = function (event) {sendLocalCandidate(videoRtcpIceTransport, event.candidate,"video")};
audioRtpIceTransport.onicestatechange = ... ;
audioRtpIceListener.onerror = errorHandler;
audioRtcpIceTransport.onicestatechange = ... ;
audioRtcpIceListener.onerror = errorHandler;
videoRtpIceTransport.onicestatechange = ... ;
videoRtpIceListener.onerror = errorHandler;
videoRctpIceTransport.onicestatechange = ... ;
videoRctpIceListener.onerror = errorHandler;
//Prepare the remote candidate handler
mySignaller.onRemoteCandidate = function(remote) {
switch (remote.kind) {
case "audio":
if (remote.component === RTCIceComponent.RTP){
audioRtpIceTransport.addRemoteCandidate(remote.candidate);
} else {
audioRtcpIceTransport.addRemoteCandidate(remote.candidate);
};
break;
case "video":
if (remote.component === RTCIceComponent.RTP){
videoRtpIceTransport.addRemoteCandidate(remote.candidate);
} else {
videoRtcpIceTransport.addRemoteCandidate(remote.candidate);
};
break;
default:
log('Invalid media type received');
};
}
// Create the DTLS transports
var audioRtpDtlsTransport = new RTCDtlsTransport(audioRtpIceTransport);
var audioRtcpDtlsTransport = new RTCDtlsTransport(audioRtcpIceTransport);
var videoRtpDtlsTransport = new RTCDtlsTransport(videoRtpIceTransport);
var videoRtcpDtlsTransport = new RTCDtlsTransport(videoRtcpIceTransport);
//
// Create the sender and receiver objects
var audioSender = new RtpSender(audioTrack, audioRtpDtlsTransport, audioRtcpDtlsTransport);
var videoSender = new RtpSender(videoTrack, videoRtpDtlsTransport, videoRtcpDtlsTransport);
var audioReceiver = new RtpReceiver(audioRtpDtlsTransport, audioRtcpDtlsTransport);
var videoReceiver = new RtpReceiver(videoRtpDtlsTransport, videoRtcpDtlsTransport);
//
// Retrieve the receiver and sender capabilities
var recvAudioCaps = RTCRtpReceiver.getCapabilities("audio");
var recvVideoCaps = RTCRtpReceiver.getCapabilities("video");
var sendAudioCaps = RTCRtpSender.getCapabilities("audio");
var recvVideoCaps = RTCRtpSender.getCapabilities("video");
//
// At this point, ICE/DTLS parameters and Send/Receive capabilities can be exchanged.
mySignaller.myOfferTracks({
// Indicate that the initiator would prefer to multiplex both A/V and RTP/RTCP
"bundle": true,
// Indicate that the initiator is willing to multiplex RTP/RTCP without A/V mux
"rtcpMux": true,
// Offer the ICE parameters
"audioRtpIce": audioRtpIceListener.getLocalParameters(),
"audioRtcpIce": audioRtcpIceListener.getLocalParameters(),
"videoRtpIce": videoRtpIceListener.getLocalParameters(),
"videoRtcpIce": videoRtcpIceListener.getLocalParameters(),
// Offer the DTLS parameters
"audioRtpDtls": audioRtpDtlsTransport.getLocalParameters(),
"audioRtcpDtls": audioRtcpDtlsTransport.getLocalParameters(),
"videoRtpDtls": videoRtpDtlsTransport.getLocalParameters(),
"videoRtcpDtls": videoRtcpDtlsTransport.getLocalParameters(),
// Offer the receiver and sender audio and video capabilities.
"recvAudioCaps": recvAudioCaps,
"recvVideoCaps": recvVideoCaps,
"sendAudioCaps": sendAudioCaps,
"sendVideoCaps": sendVideoCaps
}, function(answer) {
// The responder answers with its preferences, parameters and capabilities
//
// Derive the send and receive parameters, assuming that RTP/RTCP mux will be enabled.
var audioSendParams = myCapsToSendParams(sendAudioCaps, answer.recvAudioCaps);
var videoSendParams = myCapsToSendParams(sendVideoCaps, answer.recvVideoCaps);
var audioRecvParams = myCapsToRecvParams(recvAudioCaps, answer.sendAudioCaps);
var videoRecvParams = myCapsToRecvParams(recvVideoCaps, answer.sendVideoCaps);
//
// If the responder wishes to enable bundle, we will enable it
if (answer.bundle) {
// Only start the single ICE and DTLS transport that is needed.
// No need for the ICE Transport Controller.
audioRtpIceTransport.start(audioRtpIceListener,answer.audioRtpIce, RTCIceRole.controlling);
audioRtpDtlsTransport.start(remote.audioRtpDtls);
//
// Replace the transport on the Sender and Receiver objects
//
audioSender.setTransport(audioRtpDtlsTransport);
videoSender.setTransport(audioRtpDtlsTransport);
audioReceiver.setTransport(audioRtpDtlsTransport);
videoReceiver.setTransport(audioRtpDtlsTransport);
// If BUNDLE was requested, then also assume RTP/RTCP mux
answer.rtcpMux = true;
} else {
if (answer.rtcpMux){
// The peer doesn't want BUNDLE, but it does want to multiplex RTP/RTCP
// Create the ICE Transport Controller object
var controller = new RTCIceTransportController();
controller.addTransport(audioRtpIceTransport);
controller.addTransport(videoRtpIceTransport);
// Start the audio and video ICE transports, with an implicit gather policy of "all"
audioRtpIceTransport.start(audioRtpIceListener,answer.audioRtpIce, RTCIceRole.controlling);
videoRtpIceTransport.start(videoRtpIceListener,answer.videoRtpIce, RTCIceRole.controlling);
// Start the audio and video DTLS transports
audioRtpDtlsTransport.onerror = errorHandler;
audioRtpDtlsTransport.start(answer.audioRtpDtls);
videoRtpDtlsTransport.onerror = errorHandler;
videoRtpDtlsTransport.start(answer.videoRtpDtls);
// Replace the transport on the Sender and Receiver objects
//
audioSender.setTransport(audioRtpDtlsTransport);
videoSender.setTransport(videoRtpDtlsTransport);
audioReceiver.setTransport(audioRtpDtlsTransport);
videoReceiver.setTransport(videoRtpDtlsTransport);
};
// Check if the responder does not want BUNDLE
// and does not want RTP/RTCP multiplexing
if (!answer.rtcpMux) {
// Create the ICE Transport Controller object
var controller = new RTCIceTransportController();
controller.addTransport(audioRtpIceTransport);
controller.addTransport(videoRtpIceTransport);
controller.addTransport(audioRtcpIceTransport);
controller.addTransport(videoRtcpIceTransport);
// Start the ICE transports
audioRtpIceTransport.start(audioRtpIceListener,answer.audioRtpIce, RTCIceRole.controlling);
audioRtcpIceTransport.start(audioRtcpIceListener,answer.audioRtcpIce, RTCIceRole.controlling);
videoRtpIceTransport.start(videoRtpIceListener,answer.videoRtpIce, RTCIceRole.controlling);
videoRtcpIceTransport.start(videoRtcpIceListener,answer.videoRtcpIce, RTCIceRole.controlling);
// Start the DTLS transports that are needed
audioRtpDtlsTransport.start(answer.audioRtpDtls);
audioRtcpDtlsTransport.start(answer.audioRtcpDtls);
videoRtpDtlsTransport.start(answer.videoRtpDtls);
videoRtcpDtlsTransport.start(answer.videoRtcpDtls);
// Disable RTP/RTCP multiplexing
audioSendParams.rtcp.mux = false;
videoSendParams.rtcp.mux = false;
audioRecvParams.rtcp.mux = false;
videoRecvParams.rtcp.mux = false;
};
//
// Set the audio and video send and receive parameters.
audioSender.send(audioSendParams);
videoSender.send(videoSendParams);
audioReceiver.receive(audioRecvParams);
videoReceiver.receive(videoRecvParams);
});
// Now we can render/play
// audioReceiver.track and videoReceiver.track.
// Helper functions
function log(text) {
console.log('Time: ' + (performance.now() / 1000).toFixed(3) + ' ' + text);
}
function errorHandler (error) {
log('Error encountered: ' + error.name);
}
function sendLocalCandidate(transport, candidate, kind){
mySignaller.mySendLocalCandidate({
"candidate": candidate,
"kind": kind,
"component": transport.component
});
}
The RTCIceListener enables an endpoint to construct multiple RTCIceTransport
objects from a set of local ICE parameters,
enabling usage scenarios such as parallel forking.
An RTCIceListener
instance is associated to an RTCIceTransport
.
An RTCIceListener
instance is constructed from an
RTCIceGatherOptions
object.
Obtain the ICE parameters of the RTCIceListener
.
Retrieve the sequence of valid local candidates associated with the
RTCIceListener
.
This retrieves all local candidates currently known (except for peer reflexive candidates),
even if an onlocalcandidate
event hasn't been processed yet.
This event handler, of event handler type error
,
MUST be supported by all objects
implementing the RTCIceListener
interface.
If TURN credentials are invalid, then this event MUST
be fired.
This event handler, of event handler event type icecandidate
,
MUST be supported by all objects implementing the
RTCIceListener
interface.
It receives events when a new local ICE candidate is available.
RTCIceGatherOptions provides options relating to the gathering of ICE candidates.
The ICE gather policy.
The ICE servers to be configured.
RTCIceGatherPolicy denotes the policy relating to the gathering of ICE candidates.
The ICE transport gathers all types of candidates when this value is specified.
The ICE transport gathers all ICE candidate types except for host candidates.
The ICE transport MUST only gather media relay candidates such as candidates passing through a TURN server. This can be used to reduce leakage of IP addresses in certain use cases.
// Example to demonstrate forking when RTP and RTCP are not multiplexed.
var iceGatherOptions = new RTCIceGatherOptions();
gatherOptions.gatherPolicy = RTCIceGatherPolicy.relay;
gatherOptions.iceservers = ... ;
var iceRtpListener = new RTCIceListener(gatherOptions);
var iceRtcpListener = new RTCIceListener(gatherOptions);
var iceBaseRtpTransport = new RTCIceTransport();
//create the RTCP ICE transport, which inherits the iceGatherOptions
var iceBaseRtcpTransport = iceBaseRtpTransport.createAssociatedTransport();
mySendInitiate(
{
"icertp": iceRtpListener.getLocalParameters(),
"icertcp": iceRtcpListener.getLocalParameters()
},
function(response) {
// We may get N responses
// Create the ICE Transport Controller object
var controller = new RTCIceTransportController();
controller.addTransport(iceRtpTransport);
controller.addTransport(iceRtcpTransport);
// Start the ICE transports (using the "relay" gather policy inherited from the iceListener)
iceRtpTransport.start(iceRtpListener,response.icertp, RTCIceRole.controlling);
iceRtcpTransport.start(iceRtcpListener,response.icertcp, RTCIceRole.controlling);
// ... setup DTLS, RTP, SCTP, etc.
});
iceRtpListener.onlocalcandidate = mySendLocalRtpCandidate;
iceRtcpListener.onlocalcandidate = mySendLocalRtcpCandidate;
The RTCRtpSender includes information relating to the RTP sender.
An RTCRtpSender
instance is associated to a sending MediaStreamTrack
and provides RTC related methods to it.
A RTCRtpSender
instance is constructed from an MediaStreamTrack object and
associated to an RTCDtlsTransport
.
The associated MediaStreamTrack
instance.
The associated RTP RTCDtlsTransport
instance.
The associated RTCP RTCDtlsTransport
instance.
Set the RTP RTCDtlsTransport
(and if used) RTCP RTCDtlsTransport
.
Set the MediaStreamTrack
.
Obtain the sender capabilities, based on kind. If kind is omitted or is set to "", then all capabilities are returned.
Media to be sent is controlled by parameters.
If send()
is called with invalid RTCRtpParameters
,
throw an InvalidParameters
exception.
The send()
method does not update
parameters based on what is currently being sent, so that the value of parameters remains that
last passed to the send()
method.
The RTCRtpSender
object starts sending when send()
is called for the first time, and changes the sending parameters
when send()
is called again.
The RTCRtpSender
object stops sending when stop()
is called.
Stops sending the track on the wire, and sends an RTCP BYE.
Stop is final as in MediaStreamTrack.stop()
This event handler, of event handler type error
,
MUST be supported by all objects implementing the RTCRtpSender
interface.
This event MUST be fired if an issue is found
with the RTCRtpParameters
object passed to
send()
, that is not immediately detected.
This event handler, of event handler type RTCSsrcConflictEvent
,
MUST be supported by all objects implementing the RTCRtpSender
interface.
This event MUST be fired if an SSRC conflict is detected.
On an SSRC conflict, the RTCRtpSender
automatically sends an RTCP BYE on the conflicted SSRC.
The ssrcconflict
event of the RTCRtpSender
object uses
the RTCSsrcConflictEvent
interface.
Firing an
RTCSsrcConflictEvent event named
e with an
ssrc means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCSsrcConflictEvent
interface with the
ssrc
attribute set to the conflicting SSRC
MUST be
created and dispatched at the given target.
The ssrc attribute represents the conflicting SSRC that caused the event.
The ssrc attribute represents the conflicting SSRC that caused the event.
The RTCRtpReceiver includes information relating to the RTP receiver.
An RTCRtpReceiver
instance is associated to a receiving
MediaStreamTrack and provides RTC related methods to it.
A RTCRtpReceiver
instance is constructed from an RTCDtlsTransport
object.
The associated MediaStreamTrack
instance.
The associated RTP RTCDtlsTransport
instance.
The associated RTCP RTCDtlsTransport
instance.
Set the RTP RTCDtlsTransport
(and if used) RTCP RTCDtlsTransport
.
Obtain the receiver capabilities, based on kind. If kind is omitted or set to "", then all capabilities are returned.
Media to be received is controlled by parameters.
If receive()
is called with invalid RTCRtpParameters
,
throw an InvalidParameters
exception.
The receive()
method does not update
parameters based on what is currently being received, so that the value of parameters remains that
last passed to the receive()
method.
The RTCRtpReceiver
object starts receiving when receive()
is called for the first time, and changes the receiving parameters
when receive()
is called again.
The RTCRtpReceiver
object stops receiving when stop()
is called.
After receive
returns, track is set, and the value of track.kind is determined
based on the kind of the codecs provided in parameters.codecs.
If parameters.codecs are all of a single kind then track.kind is set to that kind.
If parameters.codecs are of mixed kind,
throw an InvalidParameters
exception.
Stops receiving the track on the wire. Stop is final like MediaStreamTrack.
This event handler, of event handler type error
,
MUST be supported by all objects implementing the RTCRtpReceiver
interface.
This event MUST be fired if an issue is found with the RTCRtpParameters
object passed to
receive()
, that is not immediately detected.
// Assume we already have a way to signal, a transport
// (RTCDtlsTransport), and audio and video tracks. This is an example
// of how to offer them and get back an answer with audio and
// video tracks, and begin sending and receiving them.
// The example assumes that RTP and RTCP are multiplexed.
function myInitiate(mySignaller, transport, audioTrack, videoTrack) {
var audioSender = new RTCRtpSender(audioTrack, transport);
var videoSender = new RTCRtpSender(videoTrack, transport);
var audioReceiver = new RTCRtpReceiver(transport);
var videoReceiver = new RTCRtpReceiver(transport);
// Retrieve the audio and video receiver capabilities
var recvAudioCaps = RTCRtpReceiver.getCapabilities("audio");
var recvVideoCaps = RTCRtpReceiver.getCapabilities("video");
// Retrieve the audio and video sender capabilities
var sendAudioCaps = RTCRtpSender.getCapabilities("audio");
var sendVideoCaps = RTCRtpSender.getCapabilities("video");
mySignaller.myOfferTracks({
// The initiator offers its receiver and sender capabilities.
"recvAudioCaps": recvAudioCaps,
"recvVideoCaps": recvVideoCaps,
"sendAudioCaps": sendAudioCaps,
"sendVideoCaps": sendVideoCaps
}, function(answer) {
// The responder answers with its receiver capabilities
// Derive the send and receive parameters
var audioSendParams = myCapsToSendParams(sendAudioCaps, answer.recvAudioCaps);
var videoSendParams = myCapsToSendParams(sendVideoCaps, answer.recvVideoCaps);
var audioRecvParams = myCapsToRecvParams(recvAudioCaps, answer.sendAudioCaps);
var videoRecvParams = myCapsToRecvParams(recvVideoCaps, answer.sendVideoCaps);
audioSender.send(audioSendParams);
videoSender.send(videoSendParams);
audioReceiver.receive(audioRecvParams);
videoReceiver.receive(videoRecvParams);
// Now we can render/play
// audioReceiver.track and videoReceiver.track.
});
}
// Assume we already have a way to signal, a transport (RTCDtlsTransport)
// and audio and video tracks. This is an example of how to answer an
// offer with audio and video tracks, and begin sending and receiving them.
// The example assumes that RTP and RTCP are multiplexed.
function myAccept(
mySignaller, remote, transport, audioTrack, videoTrack) {
var audioSender = new RTCRtpSender(audioTrack, transport);
var videoSender = new RTCRtpSender(videoTrack, transport);
var audioReceiver = new RTCRtpReceiver(transport);
var videoReceiver = new RTCRtpReceiver(transport);
// Retrieve the send and receive capabilities
var recvAudioCaps = RTCRtpReceiver.getCapabilities("audio");
var recvVideoCaps = RTCRtpReceiver.getCapabilities("video");
var sendAudioCaps = RTCRtpSender.getCapabilities("audio");
var sendVideoCaps = RTCRtpSender.getCapabilities("video");
mySignaller.myAnswerTracks({
"recvAudioCaps": recvAudioCaps,
"recvVideoCaps": recvVideoCaps,
"sendAudioCaps": sendAudioCaps,
"sendVideoCaps": sendVideoCaps
});
// Derive the send and receive parameters using Javascript functions defined in Section 15.2.
var audioSendParams = myCapsToSendParams(sendAudioCaps, remote.recvAudioCaps);
var videoSendParams = myCapsToSendParams(sendVideoCaps, remote.recvVideoCaps);
var audioRecvParams = myCapsToRecvParams(recvAudioCaps, remote.sendAudioCaps);
var videoRecvParams = myCapsToRecvParams(recvVideoCaps, remote.sendVideoCaps);
audioSender.send(audioSendParams);
videoSender.send(videoSendParams);
audioReceiver.receive(audioRecvParams);
videoReceiver.receive(videoRecvParams);
// Now we can render/play
// audioReceiver.track and videoReceiver.track.
}
The RTCRtpListener listens to RTP packets received from the DTLS transport.
An RTCRtpListener
instance is associated to an RTCDtlsTransport
.
An RTCRtpListener instance is constructed from an RTCDtlsTransport object.
The RTP RTCDtlsTransport
instance.
The event handler which handles the RTCRtpUnhandledRtpEvent.
An unhandledrtp
event is fired if the RTCRtpListener
detects an
RTP stream that is not configured to be processed by an
existing RTCRtpReceiver
object. The amount of buffering to be provided for unhandled
RTP streams is recommended to be strictly limited to protect against denial of service attacks.
To determine whether an RTP stream is configured to be processed by an existing RTCRtpReceiver
object,
the RTCRtpListener
attempts to match the values of an incoming RTP packet's
Payload Type and SSRC fields as well as the value of the muxId (if present in the MID RTP header extension) against the
RTCRtpReceiver.RTCRtpParameters.RTCRtpCodecParameters.payLoadType
,
RTCRtpReceiver.RTCRtpParameters.RTCRtpEncodingParameters.ssrc
,
and RTCRtpReceiver.RTCRtpParameters.muxId
attributes of configured RTCRtpReceiver
objects.
TODO: provide details of matching behavior, along with examples.
The unhandledrtp
event of the RTCRtpListener
object uses
the RTCRtpUnhandledEvent
interface.
Firing an
unhandledrtp
event named
e with an RTCRtpUnhandled
stream means that an event with the name e,
which does not bubble (except where otherwise stated) and is not
cancelable (except where otherwise stated), and which uses the
RTCRtpUnhandledEvent
interface with the
stream
attribute set to an RTCRtpUnhandled
object,
MUST be
created and dispatched at the given target.
The stream
attribute is the
RTCRtpUnhandled
object with the characteristics of the RTP stream
that caused the event.
The characteristics of the RTP stream that caused the event.
RTCRtpUnhandled provides information on the RTP packet that caused the RTCRtpUnhandled
event.
The SSRC in the RTP stream triggering the unhandledrtp
event.
The Payload Type value in the RTP stream triggering the unhandledrtp
event.
If present, the value of the MID RTP header extension [[!BUNDLE]] in the RTP stream
triggering the unhandledrtp
event.
The RTCRtpCapabilities object expresses the capabilities of RTCRtpSender
and
RTCRtpReceiver
objects.
Features which are mandatory to implement in [[!RTP-USAGE]], such as RTP/RTCP multiplexing [[!RFC5761]]
and reduced size RTCP [[!RFC5506]] are assumed to be available and are therefore
not included in RTCRtpCapabilities
,
although these features can be set via RTCRtpParameters
.
Supported codecs.
Supported RTP header extensions.
Supported FEC mechanisms. Currently [[!RTP-USAGE]] makes no recommendations relating to FEC mechanisms.
RTCRtcpFeedback provides information on RTCP feedback messages.
Valid values for type are the "RTCP Feedback" Attribute Values enumerated in [[!IANA-SDP-14]] ("ack", "ccm", "nack", etc.).
For a type of "ack" or "nack", valid values for parameters are the "ack" and "nack" Attribute Values enumerated in [[!IANA-SDP-15]] ("sli", "rpsi", etc.). For a type of "ccm", valid values for parameters are the "Codec Control Messages" enumerated in [[!IANA-SDP-19]] ("fir", "tmmbr" (includes "tmmbn"), etc.).
RTCRtpCodecCapability provides information on the capabilities of a codec.
The MIME media type. Valid types are listed in [[!IANA-RTP-2]].
The media supported by the codec: "audio", "video" or "" for both.
Codec clock rate expressed in Hertz, null if unset.
Added to make it possible for the sender and receiver to pick a matching payload type when creating sender and receiver parameters.
The number of channels supported (e.g. stereo). For video, this will be unset.
Transport layer and codec-specific feedback messages for this codec.
Codec-specific parameters that must be signaled to the remote party.
Codec-specific parameters available for signaling.
Maximum number of temporal layer extensions supported by this codec (e.g. a value of 1 indicates support for up to 2 temporal layers). A value of 0 indicates no support for temporal scalability.
Maximum number of spatial layer extensions supported by this codec (e.g. a value of 1 indicates support for up to 2 spatial layers). A value of 0 indicates no support for spatial scalability.
Whether the implementation can send SVC layers utilizing distinct SSRCs. Unset for audio codecs. For video codecs, only set if the codec supports scalable video coding with multiple streams.
The capability parameters for commonly implemented codecs are provided below.
The following capabilities are defined for Opus, as noted in [[!OPUS-RTP]] Section 6.1:
Property Name | Values | Notes |
---|---|---|
maxplaybackrate |
unsigned long
|
A hint about the maximum output sampling rate that the receiver is capable of rendering in Hz. |
stereo |
boolean
|
Specifies whether the decoder prefers receiving stereo (if true) or mono signals (if false). |
The following receiver capabilities are defined for VP8, as noted in [[VP8-RTP]] Section 6.1:
Property Name | Values | Notes |
---|---|---|
max-fr |
unsigned long
|
This capability indicates the maximum frame rate in frames per second that the decoder is capable of decoding. |
max-fs |
unsigned long long
|
This capability indicates the maximum frame size in macroblocks that the decoder is capable of decoding. |
The following capabilities are defined for H.264, as noted in [[RFC6184]] Section 8.1, and [[!RTCWEB-VIDEO]].
Property Name | Values | Notes |
---|---|---|
max-recv-level |
unsigned long
|
Indicates the highest level a receiver supports. |
packetization-mode | sequence<unsigned short> |
A sequence of unsigned shorts, each ranging from 0 to 2, indicating supported packetization-mode values. Support for packetization-mode 1 is mandatory in [[!RTCWEB-VIDEO]]. |
RTCRtpParameters contains the RTP stack settings.
The muxId assigned to the RTP stream, if any, empty string if unset.
In an RTCRtpReceiver
or RTCRtpSender
object, this corresponds to
MID RTP header extension defined in [[!BUNDLE]].
This is a stable identifier that permits the track corresponding to an RTP stream to be identified, rather than relying on an SSRC.
An SSRC is randomly generated and can change arbitrarily due to conflicts with other SSRCs, whereas
the muxId has a value
whose meaning can be defined in advance between RTP
sender and receiver, assisting in RTP demultiplexing.
Note that including muxId in RTCRtpParameters
rather than in RTCRtpEncodingParameters
implies that if it is desired to send simulcast streams with different muxId values for each stream, then multiple
RTCRtpSender
objects are needed.
The codecs to send or receive (could include RED [[RFC2198]], RTX [[!RFC4588]] and CN [[RFC3389]]).
Configured header extensions.
The "encodings" or "layers" to be used for things like simulcast, Scalable Video Coding, RTX, FEC, etc.
Parameters to configure RTCP.
RTCRtcpParameters provides information on RTCP settings.
The SSRC to be used in the Receiver Report "SSRC of packet sender" field as defined in [[!RFC3550]] Section 6.4.2.
If unset, ssrc is chosen by the browser, and the chosen value is not reflected in RTCRtcpParameters.ssrc
.
If the browser chooses the ssrc it may change it in event of a collision, as described in [[!RFC3550]].
The Canonical Name (CNAME) used by RTCP (e.g. in SDES messages). Guidelines for CNAME generation are provided in [[!RTP-USAGE]] Section 4.9.
By default, ORTC implementations SHOULD set the CNAME to be the same within all RTCRtcpParameter
objects created within the
same Javascript sandbox. For backward compatibility with WebRTC 1.0, applications MAY set the CNAME; if unset, the CNAME is chosen by the browser.
Whether reduced size RTCP [[!RFC5506]] is configured (if true) or compound RTCP as specified in [[!RFC3550]] (if false).
The default is false
.
Whether RTP and RTCP are multiplexed, as specified in [[!RFC5761]].
The default is true
. If set to false
, the
RTCIceTransport
MUST have an associated RTCIceTransport
object with
a component of "RTCP",
in which case RTCP will be sent on the associated RTCIceTransport
.
RTCRtpCodecParameters provides information on codec settings.
The MIME media type. Valid types are listed in [[!IANA-RTP-2]]. The name MUST always be provided.
The value that goes in the RTP Payload Type Field [[!RFC3550]]. The payloadType MUST always be provided.
Codec clock rate expressed in Hertz, null if unset.
The number of channels supported (e.g. stereo). If unset for audio, use the codec default. For video, this can be left unset.
Transport layer and codec-specific feedback messages for this codec.
Codec-specific parameters available for signaling.
The capabilities for commonly implemented codecs described in Section 9.4.2, are also
used as codec parameters, with
RTCRtpCodecCapability.parameters
of the receiver used as
RTCRtpCodecParameters.parameters
of the sender, and
RTCRtpCodecCapability.parameters
of the sender used as
RTCRtpCodecParameters.parameters
of the receiver, with the Property Name
and Values unchanged.
The SSRC for this layering/encoding.
If ssrc is unset in a RTCRtpEncodingParameters
object passed to the RTCRtpReceiver.receive
method, the
next unhandled SSRC will match, and an RTCRtpUnhandledEvent
will not be fired.
If ssrc is unset in a RTCRtpEncodingParameters
object passed to the RTCRtpSender.send
method, the browser will choose, and the chosen value is not reflected in RTCRtpEncodingParameters.ssrc
.
If the browser chooses the
ssrc, it may change it due to a collision without firing an RTCSsrcConflictEvent
.
If ssrc is set in a RTCRtpEncodingParameters
object passed to the RTCRtpSender.send
method and an
SSRC conflict is detected, then an RTCSsrcConflictEvent
is fired (see Section 6.4).
For per-encoding codec specifications, give the codec Payload Type here. If unset, the browser will choose.
Specifies the FEC mechanism if set.
Specifies the RTX [[!RFC4588]] parameters if set.
The higher the value, the more the bits will be given to each as available bandwidth goes up. Default is 1.0.
For scalable video coding, this parameter is only relevant for the base layer.
This parameter is ignored in an RTCRtpReceiver
object.
If unset, the default is assumed.
Ramp up resolution/quality/framerate until this bitrate, if set. Summed when using dependent layers.
This parameter is ignored in scalable video coding, or in an RTCRtpReceiver
object.
If unset, there is no maximum bitrate.
Never send less than this quality. 1.0 = maximum attainable quality.
For scalable video coding, this parameter is only relevant for the base layer.
This parameter is ignored in an RTCRtpReceiver
object.
What to give more bits to, if available. 0.0 = strongly favor resolution or 1.0 = strongly favor
framerate. 0.5 = neither (default). For scalable video coding, this parameter is only relevant for the base layer.
This parameter is ignored in an RTCRtpReceiver
object.
If unset, the default is assumed.
Inverse of the input resolution fraction to be encoded, or die trying. Example: 1.0 = full resolution, 2.0 = one half of the full resolution. For scalable video coding, resolutionScale refers to the inverse aggregate fraction of the input resolution achieved by this layer when combined with all dependent layers.
Inverse of the input framerate fraction to be encoded. Example: 1.0 = full framerate, 2.0 = one half of the full framerate. For scalable video coding, framerateScale refers to the inverse of the aggregate fraction of input framerate achieved by this layer when combined with all dependent layers.
Whether the sender or receiver is active. If false, don't send any media right now. Disable is different than omitting the encoding; it can keep resources available to re-enable more quickly than re-adding. Plus, it still sends RTCP. If unset, the default is assumed.
An identifier for the encoding object. This identifier should be unique within the scope of the
localized sequence of RTCRtpEncodingParameters
for any given RTCRtpParameters
object.
The encodingId
s on which this layer depends.
Within this specification encodingId
s are permitted only within the same RTCRtpEncodingParameters
sequence.
In the future if MST were to be supported, then if searching within an
RTCRtpEncodingParameters
sequence did not produce a match, then a global search
would be carried out.
//Send a thumbnail along with regular size
var encodings = [{ ssrc: 1, priority: 1.0 }]
// Control the resolution and framerate with a different track and RtpSender.
var encodings = [{ ssrc: 2,
// Prioritize the thumbnail over the main video.
priority: 10.0 }];
// Sign Language (need high framerate, but don't get too bad quality)
var encodings = [{ minQuality: 0.2, framerateBias: 1.0 }];
// Screencast (High quality, framerate can be low)
var encodings = [{ framerateBias: 0.0 }];
//Remote Desktop (High framerate, must not downscale)
var encodings = [{ framerateBias: 1.0 }];
// Audio more important than video
var audioEncodings = [{ priority: 10.0 }];
var videoEncodings = [{ priority: 0.1 }];
//Video more important than audio
var audioEncodings = [{ priority: 0.1 }];
var videoEncodings = [{ priority: 10.0 }];
//Crank up the quality
var encodings = [{ maxBitrate: 10000000 }];
//Keep the bandwidth low
var encodings = [{ maxBitrate: 100000 }];
Example of 3-layer temporal scalability encoding
var encodings =[{
// Base framerate is one quarter of the input framerate
encodingId: "0",
framerateScale: 4.0
}, {
// Temporal enhancement (half the input framerate when combined with the base layer)
encodingId: "1",
dependencyEncodingIds: ["0"]
framerateScale: 2.0
}, {
// Another temporal enhancement layer (full input framerate when all layers combined)
encodingId: "2",
dependencyEncodingIds: ["0", "1"]
framerateScale: 1.0
}]
Example of 3-layer temporal scalability with all but the base layer disabled
var encodings =[{
encodingId: "0",
framerateScale: 4.0
}, {
encodingId: "1",
dependencyEncodingIds: ["0"],
framerateScale: 2.0,
active: false
}, {
encodingId: "2",
dependencyEncodingIds: ["0", "1"],
framerateScale: 1.0,
active: false
}];
Below is a representation of a 3-layer temporal scalability encoding. In the diagram, I0 is the base layer I-frame, and P0 represents base-layer P-frames. P1 represents the first temporal enhancement layer, and P2 represents the second temporal enhancement layer.
Example of 3-layer spatial simulcast
var encodings =[{
// Simulcast layer at one quarter scale
encodingId: "0",
resolutionScale: 4.0
}, {
// Simulcast layer at one half scale
encodingId: "1",
resolutionScale: 2.0
}, {
// Simulcast layer at full scale
encodingId: "2",
resolutionScale: 1.0
}]
Example of 3-layer spatial simulcast with all but the lowest resolution layer disabled
var encodings =[{
encodingId: "0",
resolutionScale: 4.0
}, {
encodingId: "1",
resolutionScale: 2.0,
active: false
}, {
encodingId: "2",
resolutionScale: 1.0,
active: false
}];
Example of 2-layer spatial simulcast combined with 2-layer temporal scalability
var encodings =[{
// Low resolution base layer (half the input framerate, half the input resolution)
encodingId: "0",
resolutionScale: 2.0,
framerateScale: 2.0
}, {
// Enhanced resolution Base layer (half the input framerate, full input resolution)
encodingId: "E0",
resolutionScale: 1.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the low resolution base layer (full input framerate, half resolution)
encodingId: "1",
dependencyEncodingIds: ["0"],
resolutionScale: 2.0,
framerateScale: 1.0
}, {
// Temporal enhancement to the enhanced resolution base layer (full input framerate and resolution)
encodingId: "E1",
dependencyEncodingIds: ["E0"],
resolutionScale: 1.0,
framerateScale: 1.0
}]
Below is a representation of 2-layer temporal scalability combined with 2-layer spatial simulcast. Solid arrows represent temporal prediction. In the diagram, I0 is the base-layer I-frame, and P0 represents base-layer P-frames. EI0 is an enhanced resolution base-layer I-frame, and EP0 represents P-frames within the enhanced resolution base layer. P1 represents the first temporal enhancement layer, and EP1 represents a temporal enhancement to the enhanced resolution simulcast base-layer.
Example of 3-layer spatial scalability encoding
var encodings =[{
// Base layer with one quarter input resolution
encodingId: "0",
resolutionScale: 4.0
}, {
// Spatial enhancement layer providing half input resolution when combined with the base layer
encodingId: "1",
dependencyEncodingIds: ["0"]
resolutionScale: 2.0
}, {
// Additional spatial enhancement layer providing full input resolution when combined with all layers
encodingId: "2",
dependencyEncodingIds: ["0", "1"]
resolutionScale: 1.0
}]
Example of 3-layer spatial scalability with all but the base layer disabled
var encodings =[{
encodingId: "0",
resolutionScale: 4.0
}, {
encodingId: "1",
dependencyEncodingIds: ["0"],
resolutionScale: 2.0,
active: false
}, {
encodingId: "2",
dependencyEncodingIds: ["0", "1"],
resolutionScale: 1.0,
active: false
}];
Example of 2-layer spatial scalability combined with 2-layer temporal scalability
var encodings =[{
// Base layer (half input framerate, half resolution)
encodingId: "0",
resolutionScale: 2.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the base layer (full input framerate, half resolution)
encodingId: "1",
dependencyEncodingIds: ["0"],
resolutionScale: 2.0,
framerateScale: 1.0
}, {
// Spatial enhancement to the base layer (half input framerate, full resolution)
encodingId: "E0",
dependencyEncodingIds: ["0"],
resolutionScale: 1.0,
framerateScale: 2.0
}, {
// Temporal enhancement to the spatial enhancement layer (full input framerate, full resolution)
encodingId: "E1",
dependencyEncodingIds: ["E0", "1"],
resolutionScale: 1.0,
framerateScale: 1.0
}]
Below is a representation of 2-layer temporal scalability combined with 2-layer spatial scalability. Solid arrows represent temporal prediction and dashed arrows represent inter-layer prediction. In the diagram, I0 is the base-layer I-frame, and EI0 is an intra spatial enhancement. P0 represents base-layer P-frames, and P1 represents the first temporal enhancement layer. EP0 represents a resolution enhancement to the base-layer P frames, and EP1 represents a resolution enhancement to the second temporal layer P-frames.
The SSRC to use for FEC.
If unset in an RTCRtpSender
object, the browser will choose.
The Forward Error Correction (FEC) mechanism to use.
The SSRC to use for RTX.
If unset in an RTCRtpSender
object, the browser will choose.
The media supported by the header extension: "audio" for an audio codec, "video" for a video codec, and "" for both.
The URI of the RTP header extension, as defined in [[!RFC5285]].
The preferred ID value that goes in the packet.
If true, it is preferred that the value in the header be encrypted as per [[!RFC6904]]. Default is to prefer unencrypted.
The URI of the RTP header extension, as defined in [[!RFC5285]].
The value that goes in the packet.
If true, the value in the header is encrypted as per [[!RFC6904]]. Default is unencrypted.
Registered RTP header extensions are listed in [[!IANA-RTP-10]]. Header extensions mentioned in [[!RTP-USAGE]] include:
Header Extension | Reference | Notes |
---|---|---|
Rapid Synchronization | [[RFC6051]] | This extension enables carriage of an NTP-format timestamp, as defined in [[!RFC6051]] Section 3.3. |
Client-to-Mixer Audio Level | [[!RFC6464]] | This extension indicates the audio level of the audio sample carried in an RTP packet. |
Mixer-to-Client Audio Level | [[RFC6465]] | This extension indicates the audio level of individual conference participants. |
MID | [[!BUNDLE]] | This extension defines a track identifier which can be used to identify the track corresponding to an RTP stream. |
An RTCDtmfSender instance allows sending DTMF tones to/from the remote peer, as per [[!RFC4733]].
An RTCDtmfSender object is constructed from an RTCRtpSender object.
Whether the RTCDtmfSender is capable of sending DTMF.
The RTCRtpSender instance
The ontonechange event handler uses the RTCDTMFToneChangeEvent interface to return the character for each tone as it is played out.
The toneBuffer attribute returns a list of the tones remaining to be played out.
The duration attribute returns the current tone duration
value in milliseconds. This value will be the value last set via the
insertDTMF()
method, or the default value of 70 ms if
insertDTMF()
was called without specifying the duration.
The interToneGap attribute returns the current value of
the between-tone gap. This value will be the value last set via the
insertDTMF()
method, or the default value of 70
ms if insertDTMF()
was called without specifying
the interToneGap.
The tonechange event uses the RTCDTMFToneChangeEvent interface.
Firing an tonechange event named e with a DOMString tone means that an event with the name e, which does not bubble (except where otherwise stated) and is not cancelable (except where otherwise stated), and which uses the RTCDTMFToneChangeEvent interface with the tone attribute set to tone, MUST be created and dispatched at the given target.
The tone
attribute contains the character for the tone that has just begun
playout (see insertDTMF()
). If the value is the
empty string, it indicates that the previous tone has completed
playback.
The tone parameter is treated as a series of characters. The characters 0 through 9, A through D, #, and * generate the associated DTMF tones. The characters a to d are equivalent to A to D. The character ',' indicates a delay of 2 seconds before processing the next character in the tones parameter. Unrecognized characters are ignored.
Examples assume that sendObject is an RTCRtpSender
object.
Sending the DTMF signal "1234" with 500 ms duration per tone:
var sender = new RTCDtmfSender(sendObject); if (sender.canInsertDTMF) { var duration = 500; sender.insertDTMF("1234", duration); } else log("DTMF function not available");
Send the DTMF signal "1234", and light up the active key using
lightKey(key)
while the tone is playing (assuming that
lightKey("")
will darken all the keys):
var sender = new RTCDtmfSender(sendObject); sender.ontonechange = function (e) { if (!e.tone) return; // light up the key when playout starts lightKey(e.tone); // turn off the light after tone duration setTimeout(lightKey, sender.duration, ""); }; sender.insertDTMF("1234");
Send a 1-second "1" tone followed by a 2-second "2" tone:
var sender = new RTCDtmfSender(sendObject); sender.ontonechange = function (e) { if (e.tone == "1") sender.insertDTMF("2", 2000); }; sender.insertDTMF("1", 1000);
It is always safe to append to the tone buffer. This example appends before any tone playout has started as well as during playout.
var sender = new RTCDtmfSender(sendObject); sender.insertDTMF("123"); // append more tones to the tone buffer before playout has begun sender.insertDTMF(sender.toneBuffer + "456"); sender.ontonechange = function (e) { if (e.tone == "1") // append more tones when playout has begun sender.insertDTMF(sender.toneBuffer + "789"); };
Send the DTMF signal "123" and abort after sending "2".
var sender = new RTCDtmfSender(sendObject); sender.ontonechange = function (e) { if (e.tone == "2") // empty the buffer to not play any tone after "2" sender.insertDTMF(""); }; sender.insertDTMF("123");
An RTCDataChannel class instance allows sending data messages to/from the remote peer.
An RTCDataChannel
object is constructed from an RTCDataTransport
object and
an RTCDataChannelParameters
object.
The RTCDataChannel
interface represents a bi-directional data channel between
two peers.
There are two ways to establish a connection with RTCDataChannel
.
The first way is to construct an RTCDataChannel
at one of the peers with the
RTCDataChannelParameters
.negotiated attribute unset or set to its default value false.
This will announce the new channel in-band and trigger an ondatachannel event with the
corresponding RTCDataChannel
object at the other peer.
The second way is to let the application negotiate the RTCDataChannel
.
To do this, create an RTCDataChannel
object with the RTCDataChannelParameters
.negotiated
dictionary member set to true, and signal out-of-band (e.g. via a web server) to the other
side that it should create a corresponding RTCDataChannel
with the
RTCDataChannelParameters
.negotiated dictionary member set to true and the same id.
This will connect the two separately created RTCDataChannel
objects.
The second way makes it possible to create channels with asymmetric properties and to
create channels in a declarative way by specifying matching ids.
Each RTCDataChannel
has an associated underlying data transport that is used
to transport actual data to the other peer.
The transport properties of the underlying data transport, such as in order delivery
settings and reliability mode, are configured by the peer as the channel is created.
The properties of a channel cannot change after the channel has been created.
The readonly attribute referring to the related transport object.
The parameters applying to this data channel.
The readyState
attribute represents the state of the RTCDataChannel object.
It MUST return the value to which the user agent last set it (as defined by the processing model algorithms).
The bufferedAmount
attribute
MUST return the number of bytes of application data
(UTF-8 text and binary data) that have been queued using send() but that, as of the last time
the event loop started executing a task, had not yet been transmitted to the network.
This includes any text sent during the execution of the current task, regardless of whether the
user agent is able to transmit text asynchronously with script execution.
This does not include framing overhead incurred by the protocol, or buffering done by the
operating system or network hardware.
If the channel is closed, this attribute's value will only increase with each call to the
send() method (the attribute does not reset to zero once the channel closes).
The binaryType
attribute
MUST, on getting, return the value to which it was last set.
On setting, the user agent MUST set the IDL attribute to the new value.
When an RTCDataChannel object is constructed, the binaryType
attribute
MUST be initialized to the string 'blob'.
This attribute controls how binary data is exposed to scripts.
See the [[WEBSOCKETS-API]] for more information.
Closes the RTCDataChannel.
It may be called regardless of whether the RTCDataChannel object was created by this peer or the remote peer.
When the close()
method is called, the user agent
MUST run the following steps:
1. Let channel be the RTCDataChannel object which is about to be closed.
2. If channel's readyState
is closing or closed, then abort these steps.
3. Set channel's readyState
attribute to closing.
4. If the closing procedure has not started yet, start it.
This event handler, of event handler type open
,
MUST be supported by all objects implementing the RTCDataChannel interface.
This event handler, of event handler type error
,
MUST be supported by all objects implementing the RTCDataChannel interface.
This event handler, of event handler type close
,
MUST be supported by all objects implementing the RTCDataChannel interface.
This event handler, of event handler event type message
,
MUST be fired to
allow a developer's JavaScript to receive data from a remote peer.
Event Argument | Description |
Object data | The received remote data. |
Run the steps described by the send()
algorithm with argument type string
object.
Run the steps described by the send()
algorithm with argument type Blob
object.
Run the steps described by the send()
algorithm with argument type ArrayBuffer
object.
Run the steps described by the send()
algorithm with argument type ArrayBufferView
object.
The user agent is attempting to establish the underlying data transport. This is the initial state of an RTCDataChannel object.
The underlying data transport is established and communication is possible. This is the initial state of an RTCDataChannel object dispatched as a part of an RTCDataChannelEvent.
The procedure to close down the underlying data transport has started.
The underlying data transport has been closed or could not be established.
An RTCDataChannel can be configured to operate in different reliability modes. A reliable channel ensures that the data is delivered at the other peer through retransmissions. An unreliable channel is configured to either limit the number of retransmissions (maxRetransmits ) or set a time during which transmissions (including retransmissions) are allowed (maxPacketLifeTime). These properties can not be used simultaneously and an attempt to do so will result in an error. Not setting any of these properties results in a reliable channel.
The label attribute represents a label that can be used to distinguish this RTCDataChannel object from other RTCDataChannel objects. The attribute MUST return the value to which it was set when the RTCDataChannel object was constructed. For an SCTP data channel, the label is carried in the DATA_CHANNEL_OPEN message defined in [[!DATA-PROT]] Section 5.1.
The ordered
attribute returns true if the RTCDataChannel is ordered, and
false if out of order delivery is allowed. Default is true.
The attribute MUST return the value to which it was set when the
RTCDataChannel was constructed.
The maxPacketLifetime
attribute represents the length of the time window (in milliseconds) during which
retransmissions may occur in unreliable mode, or null if unset.
The attribute MUST return the value to which it was set when the
RTCDataChannel was constructed.
The maxRetransmits
attribute returns the maximum number of
retransmissions that are attempted in unreliable mode, or null if unset.
The attribute MUST be initialized to null by default and
MUST return the
value to which it was set when the RTCDataChannel was constructed.
The name of the sub-protocol used with this RTCDataChannel if any, or the empty string otherwise (in which case the protocol is unspecified). The attribute MUST return the value to which it was set when the RTCDataChannel was constucted. Sub-protocols are registered in the 'Websocket Subprotocol Name Registry' created in [[RFC6455]] Section 11.5.
The negotiated
attribute returns true if this RTCDataChannel
was negotiated by the application, or false otherwise. The attribute MUST
be initialized to false
by default and MUST return the value to which it
was set when the RTCDataChannel was constructed.
If set to true, the application developer MUST signal to the remote peer to
construct an RTCDataChannel object with the same id for the data channel
to be open.
If set to false, the remote party will receive an ondatachannel event with
a system constructed RTCDataChannel object.
The id attribute returns the id for this RTCDataChannel, or null if unset. The id was either assigned by the user agent at channel creation time or was selected by the script. For SCTP, the id represents a stream identifier, as discussed in [[!DATA]] Section 6.5. The attribute MUST return the value to which it was set when the RTCDataChannel was constructed.
The RTCSctpTransport includes information relating to Stream Control Transmission Protocol (SCTP) transport.
An RTCSctpTransport inherits from an RTCDataTransport object, which is associated to an RTCDataChannel object.
An RTCSctpTransport is constructed from an RTCDtlsTransport object.
The RTCDtlsTransport instance the RTCSctpTransport object is sending over.
Retrieves the RTCSctpCapabilities of the RTCSctpTransport instance.
Stops the RTCSctpTransport instance.
The ondatachannel event handler, of type datachannel
, MUST be
supported by all objects implementing the
RTCSctpTransport
interface.
If the remote peers sets RTCDataChannelParameters
.negotiated
to false,
then the event will fire indicating a new RTCDataChannel
object has been
constructed to connect with the RTCDataChannel
constructed by the remote peer.
Maximum message size.
The datachannel
event
uses the RTCDataChannelEvent
interface.
Firing a datachannel event named
e with a RTCDataChannel
channel means that an event with the name e, which
does not bubble (except where otherwise stated) and is not cancelable
(except where otherwise stated), and which uses the
RTCDataChannelEvent
interface with the channel
attribute set to
channel, MUST be created and dispatched at the given
target.
The channel
attribute
represents the RTCDataChannel
object associated
with the event.
TODO
function initiate(signaller) {
var dtls = ...; // See ICE/DTLS example.
var sctp = new RTCSctpTransport(dtls);
var parameters = ...; // Construct RTCDataChannelParameters object
signaller.sendInitiate({
// ... include ICE/DTLS info from other example.
"sctpCapabilities": RTCSctpTransport.getCapabilities()
}, function(remote) {
sctp.start(remote.sctpCapabilities);
});
var channel = new RTCDataChannel (sctp, parameters);
channel.send("foo");
}
function accept(signaller, remote) {
var dtls = ...; // See ICE/DTLS example.
signaller.sendAccept({
// ... include ICE/DTLS info from other example.
"sctpCapabilities": RTCSctpTransport.getCapabilities()
});
var sctp = new RTCSctpTransport(dtls);
sctp.start(remote.sctpCapabilties);
// Assume in-band signalling. We could also easily add
// RTCDataChannelParameters into the out-of-band signalling
// And construct the data channel with with negotiated: true.
sctp.ondatachannel = function(channel) {
channel.onmessage = function(message) {
if (message == "foo") {
channel.send("bar");
}
}
}
The Statistics API enables retrieval of statistics relating to RTCRtpSender
,
RTCRtpReceiver
,
RTCDtlsTransport
, RTCIceTransport
and
RTCSctpTransport
objects.
Gathers stats for the given object and reports the result asynchronously.
When the getStats() method is invoked, the user agent MUST queue a task to run the following steps:
If the object's
RTCRtpParameters.RTCRtpEncodingParameters.active
state is false
, throw an
InvalidStateError
exception.
Return, but continue the following steps in the background.
Start gathering the stats.
When the relevant stats have been gathered, return a
new RTCStatsReport
object, representing the
gathered stats.
The getStats()
method delivers a successful result in the form of a
RTCStatsReport
object. A
RTCStatsReport
object represents a map between
strings, identifying the inspected objects (RTCStats.id), and their corresponding
RTCStats
objects.
An RTCStatsReport
may be composed of several
RTCStats
objects, each reporting stats for one
underlying object.
One achieves the total for the object by summing over all stats of a
certain type; for instance, if an RTCRtpSender object is sending
RTP streams involving multiple SSRCs over the network, the
RTCStatsReport
may contain one RTCStats
object per SSRC (which can be distinguished by the value of the "ssrc"
stats attribute).
Getter to retrieve the RTCStats
objects that
this stats report is composed of.
The set of supported property names [[!WEBIDL]] is defined as the
ids of all the RTCStats
objects that has been
generated for this stats report. The order of the property names is
left to the user agent.
An RTCStats
dictionary represents the stats
gathered by inspecting a specific object.
The RTCStats
dictionary is a base type that specifies as set of default attributes,
such as timestamp and type. Specific stats are added by extending the
RTCStats
dictionary.
Note that while stats names are standardized, any given implementation may be using experimental values or values not yet known to the Web application. Thus, applications MUST be prepared to deal with unknown stats.
Statistics need to be synchronized with each other in order to yield
reasonable values in computation; for instance, if "bytesSent" and
"packetsSent" are both reported, they both need to be reported over the
same interval, so that "average packet size" can be computed as "bytes /
packets" - if the intervals are different, this will yield errors. Thus
implementations MUST return synchronized values for all stats in a
RTCStats
object.
The timestamp
,
of type DOMHiResTimeStamp
[[!HIGHRES-TIME]], associated
with this object. The time is relative to the UNIX epoch (Jan 1,
1970, UTC).
The type of this object.
The type
attribute
MUST be initialized to the name of the most specific type this
RTCStats
dictionary represents.
A unique id
that is
associated with the object that was inspected to produce this
RTCStats
object. Two RTCStats
objects, extracted from two different
RTCStatsReport
objects, MUST have the same id if
they were produced by inspecting the same underlying object. User
agents are free to pick any format for the id as long as it meets the
requirements above.
Inbound RTP. Relevant to RTCRtpReceiver
objects.
Outbound RTP. Relevant to RTCRtpSender
objects.
Transport statistics. Relevant to RTCDtlsTransport
objects.
ICE candidate pair statistics. Relevant to RTCIceTransport
objects.
ICE local candidate statistics. Relevant to RTCIceTransport
objects.
ICE remote candidate statistics. Relevant to RTCIceTransport
objects.
...
The remoteId
can be used to look up the corresponding
RTCStats
object that represents stats reported by
the other peer.
Count of FIR packets, defined in [[!RFC5104]] Section 4.3.1. We are not counting the FIR defined in RFC 2032 Section 5.2.1, which was deprecated in [[RFC4587]].
Count of PLI packets, defined in [[!RFC4585]] Section 6.3.1.
Count of NACK packets, defined in [[!RFC4585]] Section 6.2.1.
Count of SLI packets, defined in [[!RFC4585]] Section 6.3.2.
RTCInboundRTPStreamStats are relevant to RTCRtpReceiver
objects.
Packets received.
Bytes received.
Packets lost.
Jitter, as calculated in [[!RFC3550]] Section 6.4.1, but given in seconds.
RTCOutboundRTPStreamStats are relevant to RTCRtpSender
objects.
Packets sent.
Bytes sent.
Presently configured bitrate target of this SSRC, in bits per second. Typically this is a configuration parameter of the codec's encoder.
Estimated round trip time (seconds) based on the RTCP timestamp, as described in [[!RFC3550]] Section 6.4.1.
RTCMediaStreamTrackStats are relevant to MediaStreamTrack
s.
track.id property
SSRCs.
The nominal FPS value.
Only makes sense for remoteSource=true.
Same definition as droppedVideoFrames in media-source VideoPlaybackQuality
Values 0..1, linear, with 1.0 = 0dBov as defined in [[!RFC6464]].
As defined in G.168 (2012) Section 3.14, in decibels.
As above, Section 3.15.
RTCMediaStreamStats are relevant to MediaStream
s.
stream.id property
This is the id of the stats object, not the track.id
RTCDataChannelStats are relevant to RTCDataChannel
s.
The 'id' attribute of the RTCDataChannel
object.
The state of the RTCDataChannel
object.
Number of API 'message' events.
RTCTransportStats are relevant to RTCDtlsTransport
objects.
If RTP and RTCP are not multiplexed, this is the ID of the transport that gives stats for the RTCP component, and this record has only the RTP component stats.
RTCStatsIceCandidatePairState provides the state of an ICE candidate pair.
The candidate pair is frozen.
The candidate pair is waiting for a connectivity check to go out.
A connectivity check is in progress.
The connectivity check has failed.
The connectivity check has succeeded.
The connectivity check has been cancelled.
RTCStatsIceCandidatePairStats provides statistics on ICE candidate pairs.
Bits per second, implementation dependent computation.
Bits per second, implementation dependent computation.
RTCCertificateStats provides information on certificates.
As defined in RFC 4572 section 5 - the fingerprint value only
For instance 'sha-256'.
DER-encoded, base-64 representation of the certificate.
Consider the case where the user is experiencing bad sound and the application wants to determine if the cause of it is packet loss. The following example code might be used:
var mySender = new RTCRtpSender(myTrack); var myPreviousReport = null; // ... wait a bit setTimeout(function () { mySender.getStats().then(function (report) { processStats(report); myPreviousReport = report; }); }, aBit); function processStats(currentReport) { if (myPreviousReport === null) return; // currentReport + myPreviousReport are an RTCStatsReport interface // compare the elements from the current report with the baseline for (var now in currentReport) { if (now.type != "outbound-rtp") continue; // get the corresponding stats from the previous report base = myPreviousReport[now.id]; // base + now will be of RTCRtpStreamStats dictionary type if (base) { remoteNow = currentReport[now.remoteId]; remoteBase = myPreviousReport[base.remoteId]; var packetsSent = now.packetsSent - base.packetsSent; var packetsReceived = remoteNow.packetsReceived - remoteBase.packetsReceived; // if fractionLost is > 0.3, we have probably found the culprit var fractionLost = (packetsSent - packetsReceived) / packetsSent; } } }
An RTCIdentity
instance enables authentication of a DTLS transport using a
web-based identity provider (IdP).
The idea is that the initiator acts as the Authenticating Party
(AP) and obtains an identity assertion from the IdP which is then conveyed in signaling.
The responder acts as the Relying Party (RP) and verifies the assertion.
The interaction with the IdP is designed to decouple the browser from any particular identity provider, so that the browser need only know how to load the IdP's Javascript (which is deterministic from the IdP's identity), and the generic protocol for requesting and verifying assertions. The IdP provides whatever logic is necessary to bridge the generic protocol to the IdP's specific requirements. Thus, a single browser can support any number of identity protocols, including being forward compatible with IdPs which did not exist at the time the Identity Provider API was implemented. The generic protocol details are described in [[!RTCWEB-SECURITY-ARCH]]. This section specifices the procedures required to instantiate the IdP proxy, request identity assertions, and consume the results.
A RTCIdentity instance is constructed from an RTCDtlsTransport object.
In order to communicate with the IdP, the browser instantiates an isolated interpreted context, effectively an invisible IFRAME. The initial contents of the context are loaded from a URI derived from the IdP's domain name, as described in [[!RTCWEB-SECURITY-ARCH]].
For purposes of generating assertions, the IdP shall be chosen as follows:
getIdentityAssertion()
method has been called,
the IdP provided shall be used.getIdentityAssertion()
method has not been
called, then the browser can use an IdP configured into the
browser.In order to verify assertions, the IdP domain name and protocol are
taken from the domain
and protocol
fields of
the identity assertion.
The browser creates an IdP proxy by loading an isolated, invisible
IFRAME with HTML content from the IdP URI. The URI for the IdP is a
well-known URI formed from the domain
and protocol
fields, as specified in [[!RTCWEB-SECURITY-ARCH]].
When an IdP proxy is required, the browser performs the following steps:
sandbox
attribute is set to
"allow-forms allow-scripts allow-same-origin" to limit the
capabilities available to the IdP. The browser MUST prevent the IdP
proxy from navigating the browsing context to a different location.
The browser MUST prevent the IdP proxy from interacting with the user
(this includes, in particular, popup windows and user dialogs).MessageChannel
[[!webmessaging]] within the context of
the IdP proxy and assigns one port from the channel to a variable
named rtcwebIdentityPort on the window. This
message channel forms the basis of communication between the browser
and the IdP proxy. Since it is an essential security property of the
web sandbox that a page is unable to insert objects into content from
another origin, this ensures that the IdP proxy can trust that
messages originating from window.rtcwebIdentityPort are
from RTCIdentity
and not some other page. This
protection ensures that pages from other origins are unable to
instantiate IdP proxies and obtain identity assertions.RTCIdentity
object that it is ready by sending a "READY"
message to the message channel port [[!RTCWEB-SECURITY-ARCH]]. Once
this message is received by the RTCIdentity
object, the
IdP is considered ready to receive requests to generate or verify
identity assertions.[TODO: This is not sufficient unless we expect the IdP to protect this information. Otherwise, the identity information can be copied from a session with "good" properties to any other session with the same fingerprint information. Since we want to reuse credentials, that would be bad.] The identity mechanism MUST provide an indication to the remote side of whether it requires the stream contents to be protected. Implementations MUST have an user interface that indicates the different cases and identity for these.
The identity assertion request process involves the following steps:
RTCIdentity
instantiates an IdP proxy as
described in Identity
Provider Selection section and waits
for the IdP to signal that it is ready.RTCIdentity
object desires to be bound to the user's
identity.RTCIdentity
object over the message channel.RTCIdentity
object MAY store the identity assertion.
The format and contents of the messages that are exchanged are described in detail in [[!RTCWEB-SECURITY-ARCH]].
The IdP proxy can return an "ERROR" response. If an error is
encountered, the getIdentityAssertion
Promise MUST
be rejected.
The browser SHOULD limit the time that it will allow for this process. This includes both the loading of the IdP proxy and the identity assertion generation. Failure to do so potentially causes the corresponding operation to take an indefinite amount of time. This timer can be cancelled when the IdP produces a response. The timer running to completion can be treated as equivalent to an error from the IdP.
NOTE: Where RTP and RTCP are not multiplexed, distinct RTCRtpIceTransport
, RTCRtpDtlsTransport
and RTCIdentity
objects can be constructed for RTP and RTCP.
However, while it is possible for getIdentityAssertion()
to be called with different values of provider, protocol and username for the RTP and RTCP
RTCIdentity
objects, application developers desiring backward compatibility with WebRTC 1.0 are strongly
discouraged from doing so, since this is likely to result in an error.
An IdP could respond to a request to generate an identity assertion with a "LOGINNEEDED" error. This indicates that the site does not have the necessary information available to it (such as cookies) to authorize the creation of an identity assertion.
The "LOGINNEEDED" response includes a URL for a page where the
authorization process can be completed. This URL is exposed to the
application through the loginUrl
attribute
of the RTCIdentityError
object.
This URL might be to a page where a user is able to enter their (IdP)
username and password, or otherwise provide any information the IdP
needs to authorize a assertion request.
An application can load the login URL in an IFRAME or popup; the resulting page then provides the user with an opportunity to provide information necessary to complete the authorization process.
Once the authorization process is complete, the page loaded in the IFRAME or popup sends a message using postMessage [[!webmessaging]] to the page that loaded it (through the window.opener attribute for popups, or through window.parent for pages loaded in an IFRAME). The message MUST be the DOMString "LOGINDONE". This message informs the application that another attempt at generating an identity assertion is likely to be successful.
Identity assertion validation happens
when setIdentityAssertion()
is invoked. The process runs
asynchronously.
The identity assertion validation process involves the following steps:
RTCIdentity
instantiates an IdP proxy as
described in Identity
Provider Selection section and waits
for the IdP to signal that it is ready.RTCIdentity
object over the message channel.RTCIdentity
object validates that the fingerprint
provided by the IdP in the validation response matches the certificate
fingerprint that is, or will be, used for communications. This is done by
waiting for the DTLS connection to be established and checking
that the certificate fingerprint on the connection matches the one
provided by the IdP.RTCIdentity
validates that the domain portion
of the identity matches the domain of the IdP as described in [[!RTCWEB-SECURITY-ARCH]].RTCIdentity
stores the assertion in the
peerIdentity
, and returns an RTCIdentityAssertion
object
when the Promise from setIdentityAssertion()
is fulfilled.
The assertion
information to be displayed MUST contain the domain name of the IdP as
provided in the assertion.The IdP might fail to validate the identity assertion by providing an "ERROR" response to the validation request. Validation can also fail due to the additional checks performed by the browser. In both cases, the process terminates and no identity information is exposed to the application or the user.
The browser MUST cause the Promise of setIdentityAssertion()
to be rejected if
validation of an identity assertion fails for any reason.
The browser SHOULD limit the time that it will allow for this process. This includes both the loading of the IdP proxy and the identity assertion validation. Failure to do so potentially causes the corresponding operation to take an indefinite amount of time. This timer can be cancelled when the IdP produces a response. The timer running to completion can be treated as equivalent to an error from the IdP.
The format and contents of the messages that are exchanged are described in detail in [[!RTCWEB-SECURITY-ARCH]].
NOTE: Where RTP and RTCP are not multiplexed, it is possible that the assertions for both the RTP and RTCP will be validated, but that the identities will not be equivalent. For applications requiring backward compatibility with WebRTC 1.0, this MUST be considered an error. However, if backward compatibility with WebRTC 1.0 is not required the application MAY consider an alternative, such as ignoring the RTCP identity assertion.
The Identity API is described below.
peerIdentity contains the peer identity assertion information if an identity assertion was provided and verified. Once this value is set to a non-null value, it cannot change.
The RTCDtlsTransport
to be authenticated.
Sets the identity provider to be used for a given
RTCIdentity
object, and initiates the process of obtaining an identity assertion.
When getIdentityAssertion() is invoked, the user agent MUST run the following steps:
If transport.state
is closed
, throw an
InvalidStateError
exception and abort these
steps.
Set the current identity provider values to the triplet
(provider
, protocol
,
username
).
If any identity provider value has changed, discard any stored identity assertion.
Request an identity assertion from the IdP.
If the IdP proxy provides an assertion over the message channel,
the Promise is fulfilled, and the assertion is returned (equivalent to onidentityresult
in the
WebRTC 1.0 API). If the IdP proxy returns an "ERROR" response, the Promise is rejected, and an RTCIdentityError
object is returned,
(equivalent to onidpassertionerror
in the WebRTC 1.0 API).
Validates the identity assertion. If the Promise is fulfilled,
an RTCIdentityAssertion
is returned.
If the Promise is rejected, an RTCIdentityError
object is returned, (equivalent to
onidpvalidationerror
in the WebRTC 1.0 API).
The domain name of the identity provider that is providing the error response.
The IdP protocol that is in use.
An IdP that is unable to generate an identity assertion due to a lack of sufficient user authentication information can provide a URL to a page where the user can complete authentication. If the IdP provides this URL, this attribute includes the value provided by the IdP.
A domain name representing the identity provider.
A representation of the verified peer identity conforming to [[RFC5322]]. This identity will have been verified via the procedures described in [[!RTCWEB-SECURITY-ARCH]].
The identity system is designed so that applications need not take any special action in order for users to generate and verify identity assertions; if a user has configured an IdP into their browser, then the browser will automatically request/generate assertions and the other side will automatically verify them and display the results. However, applications may wish to exercise tighter control over the identity system as shown by the following examples.
This example shows how to configure the identity provider and protocol, and consume identity assertions.
var iceOptions = ...; var ice = new RTCIceTransport(iceOptions); var dtls = new RTCDtlsTransport(ice); var identity = new RTCIdentity(dtls); identity.getIdentityAssertion("example.com", "default", "alice@example.com").then(signalAssertion(assertion) ,function (e) { console.log("Could not obtain an Identity Assertion. idp: ",e.idp,"Protocol: ",e.protocol,"loginUrl: ",e.loginUrl); }); function signalAssertion(assertion){ mySignalInitiate( { "myAssertion": assertion }, function (response) { identity.setIdentityAssertion(response.myAssertion).then(function (peerAssertion) { console.log("Peer identity assertion validated. idp: ",peerAssertion.idp, "name: ", peerAssertion.name); }, function (e) { console.log("Could not validate peer assertion. idp: ", e.idp, "Protocol: ",e.protocol); }); }); }
The following events fire on RTCDtlsTransport
objects:
Event name | Interface | Fired when... |
---|---|---|
error |
Event
|
The RTCDtlsTransport object has
received a DTLS Alert. |
dtlsstatechange |
RTCDtlsTransportStateChangedEvent
|
The RTCDtlsTransportState changed. |
The following events fire on RTCIceTransport
objects:
Event name | Interface | Fired when... |
---|---|---|
icestatechange |
RTCIceTransportStateChangedEvent
|
The RTCIceTransportState changed. |
icecandidatepairchange |
RTCIceCandidatePairChangedEvent
|
The nominated RTCIceCandidatePair changed. |
The following events fire on RTCIceListener
objects:
Event name | Interface | Fired when... |
---|---|---|
error |
Event
|
The RTCIceListener object has
experienced an ICE gathering failure (such as an authentication failure with TURN credentials). |
icecandidate |
RTCIceListener
|
A new RTCIceGatherCandidate is made available to the script. |
The following events fire on RTCRtpSender
objects:
Event name | Interface | Fired when... |
---|---|---|
error |
Event
|
An error has been detected within the RTCRtpSender object.
This is not used for programmatic exceptions. |
ssrcconflict |
RTCSsrcConflictEvent
|
An SSRC conflict has been detected. |
The following event fires on RTCRtpReceiver
objects:
Event name | Interface | Fired when... |
---|---|---|
error |
Event
|
An error has been detected within the RTCRtpReceiver object,
such as an issue with RTCRtpParameters that could not be detected
until media arrival.
This is not used for programmatic exceptions. |
The following events fire on RTCRtpListener
objects:
Event name | Interface | Fired when... |
---|---|---|
unhandledrtp
|
RTCRtpUnhandledEvent
|
The RTCRtpListener object has received an
RTP packet that it cannot deliver to an RTCRtpReceiver object.
|
The following events fire on RTCDTMFSender
objects:
Event name | Interface | Fired when... |
---|---|---|
tonechange
|
Event
|
The RTCDTMFSender object has either just
begun playout of a tone (returned as the tone
attribute) or just ended playout of a tone (returned as an empty
value in the tone attribute). |
The following events fire on RTCDataChannel
objects:
Event name | Interface | Fired when... |
---|---|---|
open
|
Event
|
The RTCDataChannel object's underlying data transport
has been established (or re-established).
|
MessageEvent
|
Event
|
A message was successfully received. TODO: Ref where MessageEvent is defined? |
error
|
Event
|
TODO. |
close
|
Event
|
The RTCDataChannel object's underlying data transport
has been closed.
|
The following events fire on RTCSctpTransport
objects:
Event name | Interface | Fired when... |
---|---|---|
datachannel
|
RTCDataChannelEvent
|
A new RTCDataChannel
is dispatched to the script in response to the
other peer creating a channel.
|
It is a goal of the ORTC API to provide the functionality of the WebRTC 1.0 API [[WEBRTC10]], as well as to enable the WebRTC 1.0 API to be implemented on top of the ORTC API, utilizing a Javascript "shim" library. This section discusses WebRTC 1.0 compatibility issues that have been encountered by ORTC API implementers.
[[WEBRTC10]] Section 4.2.4 defines the RTCOfferOptions
dictionary, which includes the voiceActivityDetection attribute,
which determines whether Voice Activity Detection (VAD) is enabled within the Offer produced by createOffer()
.
The effect of setting voiceActivityDetection to true is to include the Comfort Noice (CN) codec defined in
[[RFC3389]] within the Offer.
Within ORTC API, equivalent behavior can be obtained by configuring the Comfort Noise codec for use within the RTCRtpParameters
object,
or configuring a codec with built-in support for Comfort Noise (such as Opus) to enable comfort noise.
This example code provides a basic audio and video session between two browsers.
RTCRtpParameters function myCapsToSendParams (RTCRtpCapabilities sendCaps, RTCRtpCapabilities remoteRecvCaps) { // Function returning the sender RTCRtpParameters, based on the local sender and remote receiver capabilities. // The goal is to enable a single stream audio and video call with minimum fuss. // // Steps to be followed: // 1. Determine the RTP features that the receiver and sender have in common. // 2. Determine the codecs that the sender and receiver have in common. // 3. Within each common codec, determine the common formats, header extensions and rtcpFeedback mechanisms. // 4. Determine the payloadType to be used, based on the receiver preferredPayloadType. // 5. Set RTCRtcpParameters such as mux to their default values. // 6. Return RTCRtpParameters enablig the jointly supported features and codecs. } RTCRtpParameters function myCapsToRecvParams (RTCRtpCapabilities recvCaps, RTCRtpCapabilities remoteSendCaps) { // Function returning the receiver RTCRtpParameters, based on the local receiver and remote sender capabilities. return myCapsToSendParams(remoteSendCaps, recvCaps); }
The editor wishes to thank Erik Lagerway for his support. Substantial text in this specification was provided by many people including Peter Thatcher, Martin Thomson, IƱaki Baz Castillo, Jose Luis Millan, Christoph Dorn, Roman Shpount, Emil Ivov, Shijun Sun and Jason Ausborn.
This section will be removed before publication.
send()
and receive()
usage as noted in:
Issue 119RTCRtpSender
, as described in:
Issue 143RTCRtpCodecParameters
as described in
Issue 113RTCRtpSender
and RTCRtpReceiver
objects, as described in
Issue 116onerror
from the RTCIceTransport
object to the RTCIceListener
object as described in
Issue 121