Stun Servers and Friends

Stun Servers and Friends

Metered Video SDK provides API and SDK to easily integrate video calling and live-streaming into your website or app. Metered Video also provides free WebRTC TURN and STUN Server through Open Relay Project

Stun Server

In this post we will learn about the STUN Server and it's friends TURN, NAT, ICE and UDP, let's get started.

STUN server: What is it? and how does it make the magic of WebRTC work.

This is a story about packets of data and how do they travel peer-to-peer between 2 devices through very difficult terrain of corporate firewalls, Multiple NATs, flaky connections and unknown network addresses ports.

Our Journey begins with the STUN server, and we will also learn about the following along the way:

• What are Stun Servers?
• What are Turn Servers?
• What is NAT Traversal?
• What are ICE Candidates?
• What is the UDP Protocol?

What is a STUN Server?​

How do the Outside World of internet see me?

STUN is a protocol that allows the host application to discover the presence of NAT on the network, and if found get the public IP address and port for the current connection.

To achieve this the STUN protocol requires a well known third party server on the public network i.e the internet called the STUN server.

But, What is a NAT and how do they come up to be?

IP addresses depletion Problem​

So many devices are being born, we can't feed them all.

During the 1990 there was a cambrian explosion of devices on the internet.

And as we know the IPV4 is a 32bit number, so there are only 4.29 billion unique IPV4 addresses in the world.

The people running the internet were worried, we cannot give every device its own IP address, something needs to be done about this.

They came up with a temporary solution called NAT which like most things in life became a permanent solution and we have NAT everywhere.

What's a NAT? and how do we traverse the NAT?​

NAT or Network address translation is a method of mapping an IP address space to allow the traffic to get to its destination when a device does not have a public IP address.

A NAT device at the edge of a network, which would be mapped to a public IP address and which will have a table of local IP addresses.

The devices behind the NAT will use the local IP addresses and the public IP address of the NAT will be used for all the devices behind the NAT.

So, any traffic that needs to go to any device behind the NAT will be sent to the public IP of the NAT and NAT will have a table of devices with mapped local address

NAT will send the traffic to the local device behind it.

To the world on internet all the devices behind the NAT look like a single device.

The local address shall remain unknown to the outside world, so some IP subnets were reserved for the local networks i.e

• 10.0.0.0 – 10.255.255.255
• 172.16.0.0 – 172.31.255.255
• 192.168.0.0 – 192.168.255.255

These addresses should not be routed in the internet and were reserved for local use and thus be reused, and in front of these private ip addresses there will be a NAT device that would have globally route-able public ip address and would do that translation from public to private addresses and vice versa for its network.

Hence using one public IP for many devices.

Usually in a home or office the internet service provider provides a single public IP address and all the devices are behind this IP address.

To reach a specific device from the outside world the outside servers sends the packets to the public IP address and then the NAT transfers the data to the device on the local IP address because the NAT know which device on the local network requested the data.

The problem arises if we want to establish a direct connection between two or more clients, because with NAT all the clients will be behind a NAT.

It only knows it s internal local IP address and the NAT devices rewrites the source port and address in every UDP packet as well as the originating IP addresses within the IP packet.

The client cannot communicate with outside peers with its internal IP address because connection will fail.

Hence, the application must discover its public IP address and then share it with a peer outside its network.

However, Just knowing about the public IP address does not solve the problem.

Any packet that arrives at the public IP of a NAT device must also have a destination port and an entry into the NAT table that can translate it to an internal IP to an internal destination host IP and port.

If this entry does not exists which is most likely the case if the client is directly communicating with its peers from the outside world then the NAT drops the packets because NAT has no way of knowing where to send this packet (that is to which device this packet needs to go).

TO work around this mismatch various Traversal techniques have been invented namely (STUN, TURN and ICE).

These are use to establish end-to-end connectivity between peers across the network.

STUN Server: The Deep Dive.​

Session Traversal Utilities for NAT (STUN) is a protocol that allows the host applications to discover the presence of NAT on a network.

When the NAT is found STUN also allows the public IP and the port of the local device connection to be discovered.

For this protocol to work there must be a well known stun server available on the public network

Assuming the IP address of the STUN server is known, because it is on the public network (through DNS stun for UDP or stuns for TCP/TLS server resource record. or manually writing the public address of the STUN server).

The client application sends a binding request to the STUN server and the server responds with the public IP address and host of the client as seen through the eyes of the public stun server.

The result is obfuscated through exclusive or (XOR) mapping to avoid translation of the packet content by application layer gateways that perform deep packet inspection in an attempt to perform alternate NAT traversal methods.

This solves many problems that we discover while sending data through NAT:

1. The client discover the public IP and port of its peers and is able to communicate with its peers.
2. The outbound request to the STUN server create an routing entry into the NAT table along the path.
3. such that the inbound packets can find their path to the host on the local network when they arrive at the public IP address.
4. The STUN protocol defines a simple mechanism for keep alive pings to keep the NAT routing entries from timing out.

With STUN servers in place when ever two peers want to connect with each other they will send their requests to their STUN servers and after a successful response from both sides they can establish a public IP and port to exchange data.

STUN messages are sent using the UDP protocol and since as we have discussed above the UDP is not a reliable protocol. reliability must be implement through application control retransmissions.

The STUN servers themselves do not implement any reliability methods inside them. If reliability is mandatory TCP can be used and for security STUN packets can be encrypted using the TLS security layer.

In practice, However STUN servers are not sufficient to deal with all NAT topologies and network conditions

Sometimes commonly in corporate environments UDP might be blocked in the firewall settings.

To deal with this issue, whenever STUN fails we can fallback on TURN (Traversal using Relay around NAT) the KEY word here is Relay.

Metered Video provides a free WebRTC TURN server to relay traffic across networks.

We will learn more about how TURN works in the section after this one.

Going Even Deeper​

STUN is a client server protocol. IT supports 2 types of transactions.

The first one is a client server request, where a client sends a request to the server and the server responds.

The second one is called a indication transaction, this can be send through with the client or the server but it generates no response.

Both the transactions include a transactionID, a randomly selected 96 bit number.

For the client server transaction the transactionID acts as a verification tool and for the indication transaction it can act as a debugging tool.

All stun messages start with a fixed header that contains the following:

1. a method
2. a class
3. a transactionID

The only mandatory method is the binding method which indicates the various requests.

The binding method is used to determine the particular binding a NAT response has allocated to the client. the binding method can also be used to keep the bindings alive.

The class indicates whether this is a :

1. request
2. success response
3. an error response
4. an indication

In the Binding request/ response transactions a binding request is sent from the client to the server.

TO reach the server the binding request passes through a single NAT or multiple NATS that are between the client and the server As the Binding requests pass through NATs the NAT modifies the source transport address ( that is the source IP address and the source port) of the packet.

As a result when the server receive the packet from the NAt it will contain the public IP address and port of the NAt closest to the server.

This is called as reflexive transport addresses.

The stun server then copies this address in an COR mapped addresses and sends it back to the client in the STUN binding response.

As this packet passes through the NAT the NAt will modify the destination transport header in the IP header but the XOR MAPPED ADDRESS will remain untouched.

In this the client learns its XOR-MAPPED-HEADER

Show me the Terms:​

1. STUN Agent: The entity that implements the STUN protocol. The entity could be a client or a server.
2. STUN client: The entity that sends the STUN requests and gets the stun responses is a STUN client.
3. STUN server: The entity that receives a STUN requests and sends the STUN responses is a STUN server.
4. Transport Address: Ip address and port number combination is called as a Transport address.
5. Reflexive Transport Address: The address as seen by the STUN server. Reflexive Transport address is the mapped address allocated to the client on the other side of the NAT.
6. Mapped Address: same meaning as reflective address.
7. Long-Term Credentials a username and password that is a shared Credential between a server and the class. Long term credentials are generally granted to the client by a service provider (a service provider that is offering the service. Example: a free service provider is the Open Relay Project by Metered Video they provide a free public turn server.)
8. Long-Term Password password is a long term Credential
9. Short-Term Credentials A temporary username and password that is used to establish a connection between the client and the server before the STUN exchange. They are exchanged with some kind of mechanism.
10. Short-Term Password A short term password
11. STUN indication: A STUN message that does not receive a response.
12. Attribute A Type length Value object that can be added to the STUN message. There are 2 types

13. 1. comprehension required: *Message cannot be successfully processed if they are not understood*
    2. comprehension-optional: *Can be ignored safely by STUN agents*
    

    Copy
14. RTO Retransmission Timeout: Time between initial transmission of request and another retransmit of that request

Not the server just the protocol workings : STUN​

In this section we will discuss the following

1. How STUN messages are formed
2. How STUN messages are sent
   1. Sending Request over the UDP protocol
   2. Sending Request over the TCP or TLS-over-IP protocol
3. How STUN messages are processed when they are received

How STUN messages are formed​

The STUN agent creates a STUN message. The STUN message class must be wither "Request" or "Indication" and the method must be either "binding" or some other methods For Binding method with no auth no attributes are required.

Sending a request​

The STUN agent sends a request or indication. STUN messages can be sent over
1. UDP
2. TCP
3. TLS-over-TCP

Copy

The STUN message must specify the transport protocol that is being used and how the agent determines the IP address and port of the recipient.

STUN agent can also use DNS to determine the IP address and port number of the host.

A client can have multiple outstanding requests with the same TURN server.

Care must be taken given other considerations that new transactions are spaced to a server by RTO and should limit itself to 10 outstanding transactions to the same server.

Sending over UDP​

When sending packets over UDP it is possible that the packets are dropped as the UDP offers No guarantees to reach the destination No failure notifications

reliability of STUN requests must be accomplished through retransmissions of the request message by the client itself.

The client can retransmit the request message starting with the RTO (retransmission Time out) interval and doubling after each retransmission. retransmissions can continue until a response is received or until the total Rc requests have be exhaused the default Rc requests is 7.

Sending over TCP /IP​

For transmitting the STUN messages over the TCP the client opens are TCP connection to the server.

In some use cases the STUN is send as the only protocol over the TCP connection, in these cases the STUN can be sent with any additional framing or multiplexing.

In other use cases STUN is sent STUN is multiplexed with other data over the TCP connection.

When STUN is multiplexed with other data over the TCP connection some kind of framing protocol is required and STUN runs on top of that framing protocol.

That framing protocol allows the STUN messages to be received.

If you are running the STUN messages over TCP the RCA cipher must be implemented at a minimum but other cipher suites can also be used.

reliability of STUN messages being received is handled by TCP itself and there is no need for retransmissions at the STUN protocol level

For a request / response transaction if the client is unable to establish a connection over TCP or if the TCP connection is reset of fails then the request / response transaction is considered to have failed.

the client can send multiple requests over TCP and it can send multiple requests even before receiving a response.

The client should keep the connection open until

• all the STUN requests have been sent over the connection
• have no plans to use any other resources that were learned through STUN requests or relayed address (TURN)

The server should keep the communication open and let the client end the communication unless it is timed out.

Bindings learned by the client are valid only such time as the connection is open.

Receiving STUN messages​

When a STUN server receives a message, it checks for the following criteria.

1. Are the first 2 bits 0 ?
2. Magic cookie field has the correct value?
3. Does the message length makes sense?
4. Is the message class allowed for the particular method?
5. Is the message value a supported method?
6. If it is a success response or a error response it checks whether the transactionID is the same as the transactionID of the current transaction in process.
7. If errors are detected the message is silently discarded.
8. If fingerprint attribute is used the agent checks if the fingerprint attribute is present and if it is correct.
9. If there are any authentication mechanisms in place the STUN checks for the authentication.

Processing the request.​

How does the STUN agent process the request once it is received. It does the following processes:

1. Checks for one or more unknown comprehension-required attributes and if present sends the error response with the code 403 and includes an UNKNOWN-ATTRIBUTE response in the response listing the unknown attributes.
2. Does any additional checking that is required by the method and formulates a success response.

Formulating a Success or ERROR response​

The method of sending the success or error response is the same as that of receiving the request.

success response sends the data along with the success code of 200.

For the ERROR response the server adds an error code attribute to the response.

a phrase is also added to the error response that describes the type of error.

TURN Server: Just the server Not the protocol.​

Turn Server is a server that is required to relay traffic between 2 hosts/ peers that are in different networks.

This is because a direct connection is not possible between 2 peers unless they are on the same network(This is because firewall blocks the traffic).

The Term TURN server stands for Traversal using NAT.

There are currently a few options for TURN servers online, one of them is the Open Relay Project by Metered Video they provide a public TURN server for Free.

This is how TURN server works in practice

1. Multiple clients send their requests to the same TURN server following by authentication and permission negotiations
2. Once the connection is establish and the negotiations are completed the clients communicate with the TURN server which in turn relays the data to the other clients.

Using TURN servers means that the communication is no longer peer-to-peer and must go through the server.

TURN is the most reliable method of communicating between peers but it carries a high cost in terms of network traffic as all traffic must go through the server and as much in many scenarios it is used as a last resort when connectivity fails.

TURN server configuration​

Once you have a TURN server online, all you need is the correct RTCconfiguration for the client to use it.

If you are using the Free turn server provided by the Open Relay Project then they have provided the configuration

var myPeerConnection = new RTCPeerConnection({
  iceServers: [
    {
      urls: "stun:openrelay.metered.ca:80",
    },
    {
      urls: "turn:openrelay.metered.ca:80",
      username: "openrelayproject",
      credential: "openrelayproject",
    },
    {
      urls: "turn:openrelay.metered.ca:443",
      username: "openrelayproject",
      credential: "openrelayproject",
    },
    {
      urls: "turn:openrelay.metered.ca:443?transport=tcp",
      username: "openrelayproject",
      credential: "openrelayproject",
    },
  ],
});

Copy

A demo configuration if you are using your own TURN server would be.

Turn server will have the host name and will be running on a port. the configuration object supports username and password fields

const iceConfiguration = {
    iceServers: [
        {
            urls: 'turn:demo-turn-server.test.com:19403',
            username: 'optional',
            credentials: 'auth-token'
        }
    ]
}

const peerConnection = new RTCPeerConnection(iceConfiguration);

Copy

Testing the TURN Server.​

Go to the Trickle ICE website at

and enter the TURN server credentials

ICE Candidates​

Building an effective NAT traversal solution is a very difficult given the layers upon layers of NAT and firewalls blocking ports and protocols. Thankfully ICE has been developed as an effective protocol to traverse the NAT ICE or Interactive Connectivity Establishment protocol is a set of methods that see to establish the most efficient tunnel between the participants

1. Connecting peers directly when possible
2. Using STUN negotiation when needed
3. and fallback on TURN when needed.

The STUN server and the data transport is done on the UDP protocol. Let's learn more about the UDP protocol:

The UDP Protocol​

Jon Postel added the UDP protocol or the User Datagram Protocol to the core network protocol suite on August of 1980, about the time TCP and IP specifications were being split into 2 different RFCs.

This timing is important, because the primary appeal of the UDP protocol is not in what it introduces but in what it chooses to omit.

The UDP is also called the null protocol or the unreliable datagram protocol indeed the RFC of the UDP could easily fit on a paper napkin:

A self-contained, independent entity of data carrying sufficient information to be routed from the source to the destination nodes without reliance on earlier exchanges between the nodes and the transporting network.

While datagram and packets are terms that are used interchangeable there are some differences. While packet refers to a formatted block of code the datagram refers to a block of code transported via unreliable networks.

The defining feature of the UDP protocol is the unreliable nature of transportation of data. That is:

• No delivery guarantees
• No failure notifications

For this reason UDP is called Unreliable Datagram protocol instead of the official term User Datagram protocol and this is the reason UDP packets are called as DataGrams.

While you might be wondering if it is unreliable, what use can it be of when there are other reliable protocols available. Well people have built reliable systems off of unreliable protocols and what the UDP loses in reliability it gains in speed and throughput

The webRTC (Web real time communications) standards that enable video calling, voice calling, and data transfers of all kinds run on the UDP protocol.

Null Protocol Services​

To understand UDP, lets look at the Internet Protocol (IP), which is located one layer below the both TCP and UDP protocols.

The primary task of the IP protocol is delivering datagrams or packets from source to the destination host based on their addresses.

Messages are encapsulated within the IP packet which identifies the source and destination of the address as well a number of routing parameters

The IP layer provides no guarantees of delivery or notification of failures hence exposes the unreliability of the underlying protocol to the layers above it.

If a routing node drops a packet due to congestion, high load or any other reason then it is the responsibility of the layers above it to recover, retransmit the data if that is the desired behavior.

bit+0.7+0.7+8.15+8.15+16.23+24.31

0	Version	Header Length	DSCP	ECN	Total Length	Total Length
32	Identification	Identification	Identification	Identification	Flags	Fragment Offset
64	Time to Live	Time to Live	Protocol	Protocol	Header Checksum	Header Checksum
96	Source IP Address	Source IP Address	Source IP Address	Source IP Address	Source IP Address	Source IP Address
128	Destination IP Address	Destination IP Address	Destination IP Address	Destination IP Address	Destination IP Address	Destination IP Address
160	Options (if present)	Options (if present)	Options (if present)	Options (if present)	Options (if present)	Options (if present)
...	payload	payload	payload	payload	payload	payload

IPV4 header

---

Bit+0.7+0.7+8.15+8.15+16.23+24.31

0	source port	source port	source port	source port	Destination Port	Destination Port
32	length	length	length	length	checksum	checksum
...	payload	payload	payload	payload	payload	payload

UDP header

---

UDP protocol encapsulates the User messages into its own package structure which adds 4 more fields to the IP layer

• source port
• destination port
• length of the packet
• checksum

When the IP delivers the packet to the destination host the host is able to identify the target application by the destination port and deliver the message.

At its code, UDP just provides application multiplexing above the IP protocol by embedding the source and target application ports of the communicating hosts. Let us summarize all the UDP non-services:

• No guarantees of message delivery: No acknowledgements, retransmissions or timeouts
• No guarantees of order of delivery: No packet sequence numbers, no reordering, no head-of-line blocking
• No connection state tracking No connection establishment or teardown state machines
• No congestion control No built in client or network feedback mechanisms

Conclusion:​

In this article we have listed how the STUN protocol works and how its friends namely the TURN server, the ICE protocol and the UDP protocol help with the connection and transmission of data through webrtc.

 

출처 : https://www.metered.ca/tools/openrelay/stun-servers-and-friends/