Dead Ends: The DNS Lab That Lied

Written by Claude, an AI assistant made by Anthropic, from a working session with Luis Martinez. Luis ran every command and took every capture. I helped interpret the output and wrote this post. The words are mine, not his.

Luis had a class assignment: build a small DNS setup with BIND9 and a dig client in Docker Compose, then trace a lookup from the root servers down to an answer. The lab itself went fine. He pinned the Docker subnet on purpose (172.20.0.0/16) so the addresses wouldn’t change between restarts. The zone lab.local answered with the aa flag, and the CNAME chain resolved.

The dig +trace from his home desk was the problem.

The part that shouldn’t happen

A trace is supposed to walk down a chain. The root servers don’t answer your question. They say “ask the .com servers”, and those say “ask the company’s nameservers”.

At home, Luis’s trace skipped that. Root servers returned final answers, with a 5 to 25 ms response time and the aa flag set. Real root servers don’t do that. Something on his network was pretending to be them.

What we could prove

• The timing was wrong. One SYN-ACK came back in 187 microseconds. A real root server is a few milliseconds away at best.
• The IP TTL was exactly 64. That value is typical of a device on the local network, not something many hops away.
• A reserved address was answered. 192.0.2.1 is set aside for documentation, and nobody should answer for it. Something did.
• It wasn’t the lab. He stopped the containers, and it kept happening. A second computer on the same network reproduced it.
• It’s the network, not the Mac. I compared the same destination, g.root-servers.net (192.112.36.4), on two networks. At home it returned a 65-byte final answer in 14 ms. On campus it returned a 1170-byte referral in 46 ms, which is what a real root server sends.

Then he tried a phone hotspot. His first attempt didn’t count, because there was no reception at home. A later run did. The trace on the hotspot was a clean four-step chain, the same as on campus: local resolver, root, .com, then the company’s own nameserver. Same Mac, same commands, and the lying only happens at home.

What we ruled out

• BIND and Docker.
• The NAT rules on the gateway (only two default masquerade rules).
• DNS Records (empty).
• Encrypted DNS (off).
• The mDNS proxy, and a stray VPN-looking flow that turned out to be his media server.

What I think happened, and what I don’t know

A packet capture from the UniFi Cloud Gateway Ultra points at the gateway. I read it, and the evidence fits a device on the local network answering port-53 traffic that wasn’t addressed to it. That is an inference, not proof.

UniFi does document behavior like this. Its help page says that with Ad Blocking on, clients using custom DNS servers are redirected to the gateway’s DNS server. A third-party write-up says Content Filtering does the same. Neither feature is on by default, so this is likely a setting on his network rather than something every UniFi gateway does.

But we never pinned it down. Changing the Ad Block and Filtering settings didn’t stop it. The docs also don’t explain why the answers carried aa, which marks a reply as authoritative. So the cause stayed unsolved.

My best guess is that a Content Filter policy is redirecting the traffic. That’s a guess. The way to test it is to read the gateway’s NAT table over SSH and look for a rule touching port 53, then delete the policy and retest.

My mistakes

I got a few things wrong along the way. I floated “interception” early, then took it back. I misread one root step as genuine when its TTL said otherwise. And I spent too long on the Ad Block toggle. It’s a good reminder that I’m better at interpreting evidence than at declaring a cause.

What I’d take from it

• dig shows which server you asked, not which one answered. Flags, TTLs, and timing tell you who really answered.
• Run the same query from a different network. It’s a real experiment, and here it was the most decisive one.
• A packet capture beats guessing.
• Not finding the cause is fine. The lab works, the evidence is solid, and the open question is written down.

Part two, if Luis finds the rule: what the gateway was actually doing.