DoH vs DoT Benchmark: Choose Your Safe DNS Protocol

Background

What is a DNS server?

What is an IP address?

IP address is a string consist of numbers and dots, for example 142.250.69.196. It is the identification of a computer in the network, we can use it to identify a computer in the network and establish a link with it.

What is a domain name?

Domain name is a string consist of English characters and dots, for example www.google.com. It is the identification of a computer (or a service) in the network, we can use it to identify a computer (or a service) in the network and establish a link with it.

So what’s the difference between them?

When you try to establish a link via a domain name, you first get the IP address corresponding to the domain name, and then connect to the server via the IP address.

You cannot directly connect to the server via a domain name, you can only connect to the server via an IP address. This is the design of TCP/IP protocol stack.

So why not directly use IP addresses? What are the advantages of domain names?

Do you want all the addresses in your browser bookmarks to be a set of numbers and dots? Domain names are obviously more identifiable.

Another big advantage of domain names is load balancing. For example the domain name of your server is www.myserver.com, then you can bind this domain with thousands of computers around the world, each of them has its own unique IP address.

Then customers can resolve the domain name based on their region. For example, if a customer is located in Canada, then he or she will get an IP address in Canada, which corresponds to a computer located in Canada. If he or she is located in Australia, then he or she will get an IP address in Australia, which also corresponds to a computer located in Australia.

By doing this we can speed up network access.

OK. How can I get the actual IP address of a domain name?

Through DNS (Domain Name System) server.

When you try to establish a link with www.google.com, your computer will first check the cache and see if there is already an IP address corresponding to this domain name.

If it exists, your computer will try to connect to the server using this cached IP address, otherwise it will ask the DNS server “What’s the IP address?”, and the DNS server will answer you with the actual IP address 142.250.69.196.

The DNS server can also be accessed via IP addresses. To name a few:

• 1.1.1.1: Cloudflare DNS
• 8.8.8.8: Google DNS

What’s the problem?

You send a request to 1.1.1.1 which asks the actual IP address of www.google.com, and 1.1.1.1 sends you a respond with the actual IP address 142.250.69.196.

All processes are not encrypted, and the traffics are transmitted in plain text!

This means the administers and the ISPs (Internet Service Provider):

1. Know what sites you visit.
2. Can tamper with query results to return a wrong IP address.

This resolution protocol is Do53. Its advantage is that the query speed is very fast, but its disadvantage is that it is not safe.

Unfortunately, the vast majority of people currently use this unsafe resolution protocol.

DoH and DoT

DoH (DNS over HTTPS) and DoT (DNS over TLS) are two new DNS resolution protocols introduced in 2018 and 2016 respectively. Their traffics are all encrypted.

DoT encrypts traffic using TLS, and has its own port number 853.

DoH has another layer of HTTP protocol on the basis of DoT and uses port 443 which is the default port number of HTTPS.

The advantage of DoH is that it uses the port of HTTPS, so query traffics can be disguised as HTTPS traffics which are very common traffics on the web and hard to identify.

And the disadvantage of DoH is that it introduces an extra layer on the basis of DoT, which will cause its performance to deteriorate.

So how much performance will DoH degrade? In this post I’m going to test the actual performance of DoT and DoH in real production environment.

Benchmark

The environment

• Server: AWS Lightsail (Oregon, Zone A)
• OS: Debian GNU/Linux 11 (bullseye)
• Tool: dog

I measured 4 popular DNS servers:

• Cloudflare DNS
• Google DNS
• Cisco OpenDNS
• Quad9 DNS

Queried example.com for 1000 times and calculated the average duration.

The script:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

#!/usr/bin/env bash

query() {
	if [ "$1" = DoH ]; then
		for i in {1..1000}; do
			dog example.com --time --https @"$2"
		done
	elif [ "$1" = DoT ]; then
		for i in {1..1000}; do
			dog example.com --time --tls @"$2"
		done
	else
		for i in {1..1000}; do
			dog example.com --time @"$2"
		done
	fi
}

rm -rf results
mkdir -p results

# Cloudflare DNS
query Do53 1.1.1.1 > results/cloudflare.do53
query DoH https://cloudflare-dns.com/dns-query > results/cloudflare.doh
query DoT 1.1.1.1 > results/cloudflare.dot
# Google DNS
query Do53 8.8.8.8 > results/google.do53
query DoH https://dns.google/dns-query > results/google.doh
query DoT dns.google > results/google.dot
# Cisco OpenDNS
query Do53 208.67.222.222 > results/opendns.do53
query DoH https://dns.opendns.com/dns-query > results/opendns.doh
query DoT dns.opendns.com > results/opendns.dot
# Quad9 DNS
query Do53 9.9.9.9 > results/quad9.do53
query DoH https://dns.quad9.net/dns-query > results/quad9.doh
query DoT dns.quad9.net > results/quad9.dot

# Filter output
sed -i \
	-e '/^A.*/d' \
	-e 's/Ran in //' \
	-e 's/ms//' \
	results/*

Result

Query duration:

Unit: millisecond

RSD (Relative Standard Deviation) of query duration:

Analysis

In terms of response time, Do53 (the unencrypted protocol) performs significantly better than DoT and DoH, and DoT performs slightly better than DoH, sometimes worse.

RSD can be used to measure stability. The bigger it is, the more unstable it is.

The RSD of Do53 looks big, but this does not prove that Do53 is very unstable, because most of the test data of Do53 is within 10 milliseconds, but the error is within 1 millisecond, and its error accounts for a considerable proportion, so we ignore the results here.

The RSD of DoT is slightly bigger than DoH. From my experience, DoT is indeed more unstable, and many times the duration time suddenly becomes very large.