Don’t stop using IPsec just yet

Today, Laura Poitras and Jake Appelbaum spoke at the 31C3 conference and in collaboration with Der Spiegel published an interesting article on VPN exploitation by the NSA.

The “TL;DR” summary of what follows below is: If you configure your IPsec based VPN properly, you are not affected. Always use Perfect Forward Secrecy (“pfs=yes” wich is the default in libreswan IPsec) and avoid PreSharedKeys (authby=secret which is not the default in libreswan IPsec). If you really need to use PSK, use a strong shared secret that cannot be brute forced. The NSA has their own version of IKEcrack running on millions of dollars worth of CPU’s. Also, the NSA sneaks into your router to steal your PSK’s so they can decrypt all your traffic.

Update 1: from media-35513.pdf (“TURMOIL/APEX/APEX High Level Description Document”):

CES generally requires the packets from both sides of an IKE exchange and knowledge of the associated pre-shared key (PSK) in order to have a chance of recovering a key for the corresponding cipher (ESP). A major goal of APEX is to access two sides of key exchanges of traffic of interest”

Update 2: To stop the NSA HappyDance, I just commited code to libreswan IPsec to warn about weak PSKs with a message notifying the sysadmin that as of July 1st, 2015, libreswan will refuse those weak PSKs. To determine the strength of the PSK, I used the Shannon Entropy value. If the PSK has a Shannon Entropy of less than 3.5, a warning will sound and in 6 months it will refuse to use that PSK.

Update 3: Just to make it clear – To break IKE PSKs, you first need to break the initial DiffieHellman exchange, which is usually MODP1024 or MODP1536 in the bad cases (and MODP2048+ in the good cases). The exception to this is IKEv1 Aggressive Mode, where the MAC computed with the PSK as secret key is sent
in the clear. As a result one can mount an offline dictionary attack.

IPsec is not IKE

First, let’s clarify things a little bit. Although everyone is talking about breaking IPsec, what they really mean is breaking the Internet Key Exchange (IKE) that is used to negotiate and create symmetric keys that are used with IPsec for encryption and decryption. IPsec itself can use various ciphers and algorithms. It can use a separate encryption (AES, CAMELLIA, SERPENT, TWOFISH, 3DES, etc) + integrity (SHA2, SHA1, AES_XCBC, MD5, etc) mode or it can use an AEAD cipher that combines these two into one (AES GCM, AES CCM, CAMELLIA GCM, etc)

But Jake said to stop using IPsec

None of what was said by Jake or published by Der Spiegel shows any attack on these ciphers or algorithms. Even the MD5 use within IPsec (HMAC-MD5) is not as weak as people usually believe it to be by confusing it with non-hmac use of MD5. The comments Jake made and the material in the published slides show attacks against weak IKE configurations, router compromises, and possibly against some IPsec hardware accelerators. Below are my comments on those slides that are interesting in the context of IKE and IPsec.

slide7

The focus of the VPN decryption team lies with IPsec and SSH. I think that fits the deployment. These two protocols are the most widely used encryption protocols to transfer bulk data. While the hacker community might focus on OpenVPN, that’s not where the real deployment is. The mention of PPTP is curious. From what I see, PPTP is basically dead as a VPN technology, although it is still widely used in Russia as an encapsulation technique for ISPs to connect endusers. It’s probably just listed there for intercept because it is so trivial to do, you might as well gather it.

page15

This slide is harder to make sense of. I think that here the NSA is talking about IKE and IPsec combined, although they call it IPsec. Perhaps once they decrypt traffic, it goes into XKEYSCORE? They don’t seem to use XKEYSCORE as an input to crack the IKE PSKs? I’m amused by the name VULCAN-DEATHGRIP which might indicate that the HiFN (now Exar) Vulcan IPsec accelerator card might have a flaw in it that allows them to decrypt those IPsec VPNs. Also, I hope they properly mask their perl scripts for malicious decrypted traffic :P

page16

This is probably the most interesting slide. It basically states that IPsec VPNs are compromised by router compromises that steal the IKE PSKs. This might be a good time to upgrade your router firmware to the latest version, and change to new strong PSKs (or switch it over to RSA 2048+ instead)
The AppID refers to the XKEYSCORE signature name. I’m glad to see that IKEv2 is finally catching on, even within the NSA :)

page26

This slide is mentioning IPsec scripts, which might refer back to the previously mentioned genericIPSec_wrapper.pl script. This clearly indicates an IKE PSK brute force decrypting engine. If you are using weak PSKs, the NSA is decrypting your IKE traffic which will give them the keys to decrypt your IPsec traffic. Enabling PFS will help you a bit, but ultimtely you should really just switch to using RSA (or ECP) authentication and skip PSK’s alltogether. Note that the FIPS standard already disallows using PSK’s for IKE/IPsec. The NSA knows it is just too dangerous and weak.

Note to the NSA, please see RFC 4301 Section 1.1 which states:

The spelling “IPsec” is preferred and used throughout this and all related IPsec standards. All other capitalizations of IPsec (e.g., IPSEC, IPSec, ipsec) are deprecated.

On slide 31 it states: IPSec: 6640 (protocols) and 6648 (ports)

I don’t know what that refers to. Possible backhaul transport ports or part of an API port ?

On slide 33 it states: Need all the pieces (IKE and ESP for IPSec)

This really shows that the NSA is breaking IPsec by breaking IKE, which mostly seems to come down to weak PSKs and no PFS. And again, in case you still did not get it on slide 39:

page39

“TAO got the configuration files which provided us the PSKs to enable passive exploitation”

So your IPsec VPN is getting owned because your VPN gateway got owned! Run your VPN on a dedicated machine, preferably opensource, and lock remote access via ssh down using strong keys and IP filtering.

And the last interesting nugget is on page 40:

page40

“NSP had an implant which allows passive exploitation with just ESP”.

I read this to mean that the hardware or software of the system running IPsec was compromised, causing it to send valid protocol ESP packets, but creating those in such a way that these could be decrypted without knowing the ESP session keys (from IKE). Possibly by subverting the hardware number generator, or functions related to IV / ICV’s / nonces that would appear to be random but were not. Or less likely if the hardware/software was using padding bytes to reveal decryption keys.

Conclusion

If anything, I would say that IPsec and IKE have withstood the attacks of the NSA. The fact that the NSA needs implants and needs to use brutce force attacks against IKE PSKs shows that despite its kitchen sink nature, IKE and IPsec are doing quite well if you configure it properly and run it on a secure host. But misconfiguration of IKE is something we did a poor job of preventing. We can and should have more automatic IKE/IPsec deployed using strong default configurations.

And you should hear back from The Libreswan Team in the next two months regarding just that. Opportunistic Encryption: Round Two

deFUDing “DNSSEC has failed”

It’s easy to forward or like an article of which the title says something you agree with, without actually reading the content. It seems “DNSSEC has failed” is one of those that showed up in my twitter feed. It’s so defunct of facts, it’s pretty clear most people have just read the title and retweeted it. Hopefully, they will at least read this post before not retweeting it.

It starts comparing the DNSSEC protocol with the SSH protocol. It claims ssh just works. It fails to take into consideration that the two are completely different trust models. With ssh, you tend to own both end points. So there is no issue of PKI or key distribution or trust. And even so, I am willing to bet the author never actually checks his SSH fingerprints when connecting to new servers, and just depends on the ssh leap of faith. Well, security is easy when you can assume you are not under attack. Most ironically, the easiest and only cross-trust-domain SSH public key distribution system out there is the DNSSEC protected SSHFP record. The only other one I know of is GSSAPI and other hooks into centrally-managed systems – eg all systems under one administrative control. Trust is pretty easy when you centrally distribute trust anchors.
Let’s also not talk about the debian ssh disaster, which was a pretty spectacular failure compromising thousands of ssh servers. Let me conclude the ssh comparison with inviting the author to login to bofh.nohats.ca using ssh in a fully trusted matter. He will end up 1) calling me to verify the ssh host key, 2) trusting the SSHFP key in DNSSEC or 3) leap-of-faithing it and hoping he was not under attack.

Next, the author brings up the tremendously easy to use and success of TLS. Never mind that we still need to run SSL 2.x for WinXP compatibility, most servers run TLS 1.0 which is severely broken, we just had the TLS 1.2 resumption/renegotiation disaster, about 4 CA compromises in 2013, and browsers running to patchwork solutions like hardcoded certificate pinning and Certificate Transparency registries that uses “n of m” style “it is probably the right certificate” solutions. There are about 600 (sub)CA’s that can sign any TLS certificate for any domain on the planet, ranging from Russia, to Iran to the U.S – all parties that are known to coerce parties and with a heavy industrial espionage component.

If TLS was successful, port 80 would be dead. Port 80 is the proof that the TLS trust model has failed. It involves money and so these certifications are only available to corporations. While DV certificates are now basically free, they are also considered worthless (protection against passive attack only) EV certificates are now dropping in price – which will make them finally available for more people but unfortunately because of that also lose their value. And all of this security hinges on the (mostly hidden) CABforum and browser vendor decisions on which CAs to include, and which CAs not to include.

While on most proprietary OSes, these preloaded CAs are at least managed by the vendor, the situation in Linux is a disaster. OpenSSL doesn’t support blacklists, NSS does. GnuTLS I don’t even know. Where are these CAs and blacklists? Do you know applications using openssl for their TLS are using the same CA bundle as applications using NSS or GnuTLS? Only very recently has Fedora and RHEL moved to a system where this is guaranteed. Even worse, most python wrappers using TLS don’t even check anything of your TLS connection. Presenting the wrong certificate will just cause that software to continue. And then look at the latest Apple TLS bug where anyone could bypass all TLS checks completely. How the author can even point to TLS as a security success is a complete mystery to me. It’s the worst computer security disaster we ever created! Look at the amount of complete failure TLS deployments at SSL Pulse – 30% allows weak ciphers, 6% has a broken certificate chain, less then 30% supports TLS 1.2, 6% vulnerable to the renegotiation/resumption attack, 70% vulnerable to the BEAST attack, 13% vulnerable to the compression attack, 56% allow RC4 and 50% does not support PFS. And that according to the author is the “success” that DNSSEC should take a lesson from?

Having listed the “success” stories of ssh and tls, he than looks at DNSSEC. His first factual statement regarding DNSSEC is “Most resolving software supports DNSSEC, but none have enabled it by default“. He is wrong. I have been the package (co)maintainer of bind, unbound and nsd for Fedora and EPEL for many years. On Fedora and EPEL/RHEL the default install enables DNSSEC for all three, and has done so for many years – even before the root was signed with DNSSEC!

Next he claims “To enable DNSSEC we have a ‘tutorial’ by Olaf Kolkman which spans a whopping sixty-nine pages“. As I said, the author is wrong. Simply “yum install bind nsd unbound” and the software comes with dnssec enabled, using the root key as trust anchor.

He then points to a DNSSEC training that takes “multiple days” and claiming that shows the complexity. The same vendor lists a two day DHCP course, a two and a five day “plain DNS & bind” course, and a two and five day IPv6 course. Does that mean those technologies are all failures too because it takes so many days to teach a course about it? According to the author, DHCP would be a failure and no one can deploy it because it “requires” a two day training? It’s just simplistic cherry-picked FUD.

His next argument is “Furthermore, there are almost no simple instructions for enabling DNSSEC, especially none that use KSKs and ZSKs, and include instructions on how to do key rollovers.“.

First, let me ask you how you do an ssh host key rollover? How do you make that seamless so it won’t trigger anyone who ever connected before from seeing a changed key? Where do they go to confirm the changed key is installed by the administrator and not the result of a hacked server or connection? Does this mean ssh has failed too? Again, amusingly enough the only way to rollover ssh keys is piggybacking on SSHFP in DNSSEC or GSSAPI trust models. But I don’t see the author claiming ssh has failed because it cannot do rollovers on its own. More cherry-picking.

The reality is, ssh and DNSSEC can do fine without rollovers. Only TLS requires rollovers because the trust model depends on the fact that you regularly pay the trustees managing that whole disaster. But if you do want or need to do a rollover, with ssh you’re fucked, with TLS you have to pay and with DNSSEC you have a seamless transition method. Let me repeat that – only DNSSEC actually implements a free and seamless key rollover mechanism! So of course, the author just states it is too hard to use. But again does not compare it against TLS or ssh at all. Cherry-picking all the way down. And by the way, there are also good reasons why you do not need to worry about DNSSEC rollovers at all.

As I said before, DNSSEC is already “enabled”, so let’s interpret the author’s quest for “simple instructions to enable DNSSEC” to mean “how do I sign my zone?”. Let us assume he is running bind right now with his zone in /var/named/1sand0s.nl. The author now has to issue the following complicated sequence of commands:


$ dnssec-keygen -f KSK -a RSASHA256 -b 2048 -n ZONE 1sand0s.nl
Generating key pair........+++ ..+++
K1sand0s.nl.+008+01699
$ dnssec-keygen -a RSASHA256 -b 1024 -n ZONE 1sand0s.nl
Generating key pair........................................++++++ ..................++++++
K1sand0s.nl.+008+24363
$ cat K1sand0s.nl.*.key >> 1sand0s.nl
$ dnssec-signzone -o 1sand0s.nl -k K1sand0s.nl.+008+01699 1sand0s.nl K1sand0s.nl.+008+24363
Verifying the zone using the following algorithms: RSASHA256.
Zone fully signed:
Algorithm: RSASHA256: KSKs: 1 active, 0 stand-by, 0 revoked
ZSKs: 1 active, 0 stand-by, 0 revoked
1sand0s.nl.signed

Edit named.conf and load the file 1sand0s.nl.signed instead of 1sand0s.nl and you’re done. And whenever you change your zone, re-run that dnssec-signer command.
(NOTE: and as pointed out to me, if you don’t have zone edits every week, do run that signer command in a cron job weekly to refresh the DNSSEC signature records)

You can put this into a git hook and maintain the whole thing without ever realising there is a signing step in between. Hardly rocket science. I think we can argue whether or not this is harder or easier than creating a CSR for Apache, get it signed, and than installed (and too many people will forget to actually get the intermediate CAs installed in apache, and their deployment is actually even broken!)

The author’s issue with opendnssec is that it is “yet another daemon” using a “database backend”. I guess he is also not running postfix because that thing has more daemons than I can remember the names of. Opendnssec’s default “database backend” is sqlite. If you hate those, there is a lot of software you’ll going to have to purge from your servers. Sqlite is a perfect backend for small databases that are just too complex for simple text files. For those of us who don’t mind two extra daemons and some sqlite files, to sign your domain from scratch with opendnssec:

# yum install opendnssec
# systemctl enable opendnssec.service
# systemctl start opendnssec.service
# ods-ksmutil zone add –zone 1sand0s.nl –input /var/named/1sand0s.nl –output /var/named/1sand0s.nl.signed
# ods-signer sign 1sand0s.nl

Change named to load the .signed zone instead of the unsigned zone and done. Shockingly complicated! Unsuitable for “common engineers” according to the author.

To make these instructions complete, the last step is to send your DS record to your Registrar which involves using their custom webgui to send them the following data:

yum install ldns
dig dnskey 1sand0s.nl > 1sand0s.nl.dnskey
ldns-key2ds -n 1sand0s.nl.dnskey

And if you want to do all of this within the same bind daemon and without sqlite, you can use bind’s builtin inline-signing management features instead.

Conclusion

Saying the tools are there for ssh (which doesn’t have ANY rollover support) and tls (which over half the world has misconfigured regardless of rollovers) but not for dnssec, is really short-sighted and wrong. What the author has done is taken his prejudice, and written some cherry-picked text justifying it. The only credit I can give him, is that he did not have to rely on the FUD statements by djb.

Development of libreswan vs openswan

animated

UPDATE: Statistics updated to include 2017

Yesterday, openswan released version 2.6.40 to address CVE-2013-6466. You might be confused by its changelog (not the non-updated CHANGES) crediting me for the vast majority of code changes. . Basically all changes are pulled from the libreswan repository and are backports to openswan. The exception is their version of a patch for CVE-2013-6466. Libreswan’s fix is not a band-aid but an updated state machine. The backported libreswan fix is what is going into the updated openswan packages for RHEL5 and RHEL6 and are available at libreswan.org/security/openswan/CVE-2013-6466. RHEL7 will contain libreswan.

Basically, yesterday’s openswan 2.6.40 release brings it up to the initial libreswan-3.0 release of two years ago, plus the two CVE issues. Except that it crashes KLIPS and backported a libreswan commit that broke all non-XAUTH IPsec connections.

History and implementation status of Opportunistic Encryption for IPsec

(as sent to the cryptography mailing list)

FreeS/WAN

In light of the NSA achievements, a few people asked about the FreeS/WAN IPsec OE efforts and whatever happened to it.

The short answer is, we failed and got distracted. The long answer follows below. At the end I will talk about the current plans that have lingered in the last two years to revive this initiative. Below I will use the word “we” a lot. Its meaning changes based on the context as various communities touched, merged, intersected and drifted apart.

NOTE: On September 28, 2013 there is be a memorial service in Ann Arbour for Hugh Daniel, manager of the old IPsec FreeS/WAN Project. Various crypto people will attend, including a bunch of us from freeswan. Hugh would have loved nothing better than his memorial service being used as a focal point to talk about “new OE”, so that’s what we will do on Saturday and Sunday. If you are interested in attending, feel free to contact me.

OE in a nutshell

For those not familiar with IPsec OE as per FreeS/WAN implementation. When activated, a host would install a blocking policy for 0.0.0.0/0. Every packet to an IP address would trigger the kernel to hold the packet and signal the IKE daemon to go find an IPsec policy for that destination. If found, the tunnel would be build, and an IPsec tunnel to the remote IP would be established, and packets would flow. If no policy was found, a “pass” hole was poked so packets would go out unencrypted. Public keys for IP addresses were looked up in the reverse DNS by the IKE daemon based on the destination address. To help with roaming clients (roadwarriors), initiators could store their public key in their FQDN, and convey their FQDN as ID when performing IKE so the remote peer could look up their public key in the forward DNS. This came at the price of two dynamic clients not being able to do OE to each other. (turns out they couldn’t anyway, because of NAT)

What were the reasons for failing to encrypt the internet with OE IPsec (in no particular order):

1) Fragmentation of IPsec kernel stacks

In part due to the early history of FreeS/WAN combined with the export restrictions at the time. Instead of spending more time on IKE and key management for large scale enduser IPsec, we ended up wasting a lot of time fixing the FreeS/WAN KLIPS IPsec stack module for each Linux release. Another IPsec stack, which we dubbed XFRM/NETKEY appeared around 2.6.9 and was backported to 2.4.x. It was terribly incomplete and severely broken. With KLIPS not being within the kernel tree, it was never taken into account. XFRM/NETKEY remained totally unsuitable for OE for a decade. XFRM/NETKEY now has almost all functionality needed – I found out today it shoudl finally have first+last packet caching for dynamic tunnels, which are essential for OE. Since the application’s first packet triggered the IKE mechanism, the application would start retransmitting before IKE was completed. Even when the tunnel finally came up, the application was usually still waiting on that TCP retransmit. David McCullough and I still spend a lot of time fixing up KLIPS to work with the current Linux kernel. Look at ipsec_kversion.h just to see what a nightmare it has been to support Linux 2.0 to 2.6 (libreswan removed support for anything lower then recent 2.4.x kernels)

Linux IPsec Crypto hardware acceleration in practise is only possible with KLIPS + OCF, as the mainstraim async crypto is lacking in hardware driver support. If you want to build OE into people’s router/modem/setup box, this is important, though admittingly less so as time has moved on and even embedded hardware and phones are multicore or have special crypto CPU instructions.

An effort to make the kernel the sole provider of crypto algorithms that everyone could use also failed, and the idea was abandoned when CPU crypto instructions appeared directly accessable from userland.

2) US citizens could not contribute code or patches to FreeS/WAN

This was John Gilmore’s policy to ensure the software remained free for US citizens. If no US citizen touched the code, it would be immune to any presidential National Security Letter. I believe this was actually the main reason for KLIPS not going in mainstream kernel, although personal egos of kernel people seemed to have played a role here as well. Freeswan people really tried had in 2000/2001 to hook KLIPS into the kernel just the way the kernel people wanted. (Ironically, the XFRM/NETKEY hook so bad, it even confuses tcpdump and with it every sysadmin trying to see whether or not their traffic is encrypted) I still don’t fully understand why it was never merged, as the code was GPL, and it should have just been merged in, even against John’s wishes. Someone would have stepped in as maintainer – after all the initial brunt of the work had been done and we had a functional IPsec stack.

In the summer of 2003, I talked to John and together we agreed it was time to fork. Openswan was born to clearly indicate US coders could contribute. However, at that point the (then crappy) FRM/NETKEY IPsec stack was there to prevent OE from working due to the missing first+last packet caching. The FreeS/WAN Project ended and Openswan continued. At first in good pace, but that later slowed down and OE was no longer its focal point. (Due to legal reasons, I cannot go into details regarding the openswan history)

3) Not using DNS without DNSSEC

There were various issues that caused DNSSEC to get massively delayed. We needed DNSSEC to secure our DNS based distributed public key platform. Although it would have worked fine to use DNS against passive attackers (NSA trawling), we believed it was principly wrong to trust cryptographic material that was untrusted and vulnerable against active attacks. So while the developers encouraged people to put keys in DNS even without security, no one else picked it up. It sucks to need to say ‘we told you so’. But we should have really not waited on DNSSEC.

4) Dealing with the DNS working groups at IETF

The DNS community is one of the most pedantic group of people I know. They are very smart, often right, and had been known to be extremely defense of their DNS turf. (Note that things have improved considerably and if you think this is still an issue, I’m happy to try and help)

IETF was divided about the convergence of the “security of the DNS” and the “DNS as PKI” despite that this had always been a goal of DNSSEC for a large group of people within the IETF. The FreeS/WAN people were driving DNSSEC not so much for DNS as for the key distribution. After all, you can detect DNS forging if you know your public keys.

When we had the KEY/SIG records ready to go, it was decreed that it could only be used for the DNS itself. Applications could not use this KEY record. To make that distinction more clear, on the next change in the draft protocol, KEY was obsoleted and DNSKEY introduced. So IPsec keys were relegated back to TXT, since at the time we had no Generic Record format (RFC 3597) support, so waiting for any new RRtype to get any deployment to become usuable would take years. Almost everyone was on bind4 and never upgraded left us with no other choice but the TXT. Even though we wrote the OE and IPSECKEY RFCs, OE’s only deployments were done using TXT records.

5) DNSSEC was delayed by a decade

DNSSEC deployment was slowly gaining traction, but I think we really needed the Kaminsky bug to get that extra push for DNSSEC outside the geeks of the IETF. The US government mandate for DNSSEC in .GOV helped as well. But by this time, OE was mostly forgotten.

djb repeatedly tried to peddle his own warez. While not at all realistic, it always gained a lot of hype and media attention and probably did cause delays of DNSSEC deployment.

Kaminsky himself was shooting down DNSSEC too. I personally heckled him at various Black Hat’s and ICANN conferences until we finally sat down for a couple of hours to talk about DNSSEC’s history and design goals. I’ll claim my 15 minutes of fame for having converted him. It helped having Kaminsky say that although he didn’t like the complexity, he couldn’t see anything better. DNSSEC was needed for everyone.

DNSSEC was gaining traction. Then we ran into a bunch of DNSSEC deployment issues. We had the delays due to NSEC vs NSEC3 with OPTIN, and then on top of that in 2008 when the first big ISP in Sweden turned on DNSSEC in their resolvers all that traction was blown away.

Most consumer routers ran DNS proxies that implemented DNS as “known bitstreams” instead of implemeting the actual DNS protocol. The DNSSEC OK bit caused thousands of routers to drop DNSSEC packets as “invalid DNS”. The only realistic solution: Turn it off and wait two years for those routers to get obsoleted by faster wifi standards and talk to those vendors so they would not repeat their mistake with their next generation of routers.

We now have the IPSECKEY record format (though RFC 4025 is not useful, see below) and RFC 3597 for the generic DNS record deployed on all DNS servers. And we’re on our way to have DNSSEC on every end node (see also draft-wouters-edns-tcp-chain-query-00 I just submitted to the IETF)

We have a mostly clean working UDP/TCP port 53 transport for DNSSEC on most networks (in part thanks to Google DNS). Although our hotspot handling is still a little rough, with dnssec-trigger the only tool to hack configurable DNSSEC support into the OS for our coffee shop visits when we need to rely on forged DNS.

6) When you’re NAT on the net, you’re NOT on the net.

Opportunistic Encryption relied on a clear peer to peer connection. But we managed to degrade the internet into servers and clients. NAT was the biggest problem, and with CGN around the corner, it’s not something that is going away despite IPv6 offering enough IPs for everyone. In fact, for our “new OE”, this is the biggest hurdle to overcome. When Alice cannot talk to Bob because she cannot reach him due to a (carrier grade) NAT, we are stuck wildly poking holes and hoping packets flow.

7) The reverse DNS tree is dead Jim

OE depended on the reverse tree as a security mechanism that someone who was claiming a public key for a specific IP range was actually the legitimate owner of that IP space. It was the security method for RFC-4025.

But unless you are running in a datacenter, you do not have access to the reverse DNS. It is useless as key distribtion method. On top of that, large IPv6 deployments don’t even care any more to run any authoritative DNS for their reverse.

8) BTNS

The IETF tried to revive this OE with the Better Then Nothing Security (“BTNS”) working group. Contrary to the name, they also fell into the “perfect is the enemy of good” trap and most discussion seemed to go into “channel binding” to upgrade anonymous IPsec to some kind of authenticated IPsec – at least by the time I became aware of them. In other words, the most important problem of key distribution was left outside the scope and no one actually seemed to have implemented anything. Though I have to admit, I’m behind on reading the VPN auto-discovery drafts. It is just
very discouraging to still be reading problem statement drafts. More over, I don’t think we should setup IPsec tunnels based on packets hitting the kernel. We have better ways now that we can leverage DNSSEC.

9) We were all complacent

The only interest for IPsec was for corporate VPNs. During the above listed problem periods, OE people gave up. Some walked away from IETF. While everyone gained an always-on portable IP device,
their crypto capabilities were practically non-existent. My current iphone 5 can connect to a corporate VPN, but trying to make it _just_ send out encrypted packets is impossible. Some trickery can be used to cause almost any packet to setup the VPN, but while that’s going on it is still leaking like a sieve. VPN is seen by phone vendors as a method to gain some enterprise users, not as the tool to protect the consumer. The Apple VPN client is a 10+ year old patched version of racoon. The only vendor that took VPNs seriously was RIM and we punished them by not buying their products, because we had other priorities like FourSquare, Facebook and Twitter.

We can only hope that those PRISM players are now put under economic pressure by frightened consumers to fix this. But as long as VPNs and DNSSEC is slow and error-prone, it is better for them not to go there.

The New Opportunistic Encryption

I’ve been brainstorming with various people on how to put IPsec OE back on the table. I’ve discussed this with a bunch of people around me, including the late Hugh Daniel, John Gilmore and Hugh Redelmeier of freeswan.

The packet capturing 0.0.0.0/0 policy is not a good method because we cannot make any decision on where to find a public key for an IP address. The reverse is unusable, and IP addresses change often. We used it because we had nothing better. But now we do. Since every (secure) platform now runs DNSSEC on the end node, we can use this as our decision making point. Imagine my phone running a DNSSEC resolver (say unbound) and an IKE daemon (say libreswan). The DNS server has access to the set of DNS name and matching IP address. It can lookup the key in the forward DNS zone, and hand over the public key, dns name and IP address to the IKE daemon!

1) User tells browser to go to www.cypherpunks.ca

2) browser does a lookup for the A/AAAA record of www.cypherpunks.ca

3) DNSSEC resolver performs the lookup/validation for the A/AAAA record of www.cypherpunks.ca and additionally looks up the IPSECKEY record of www.cypherpunks.ca.

4a) The resolver will wait with returning the A/AAAA record to the browser until it knows if the IPSECKEY record exists or not. If not, it releases the A/AAA answer to the application. Packets flow in the clear.

4b) The resolver finds an IPSECKEY record. It sends the pubic key, the FQDN and the IP address(es) to the IKE daemon and waits for a response. Meanwhile it does _not_ release the A/AAAA record to the application.

5) The IKE daemon sets up the IPsec tunnel. We haven’t reached agreement yet over how this should be done. There are two choices:

a) The client uses an “@anonymous” ID for itself along with sending its public key inline with IKE. The client is responsible for ensuring there is no MITM attack, as it knows the server’s public key (from DNSSEC). The responding server will just use any key it received inline if it was received for the “@anonymous” ID.

b) The initiator (aka client) uses its own FQDN-based ID. It has preconfigured its DNS so that an IPSECKEY record exists for its FQDN (protected by DNSSEC). The key is not send inline with IKE. Instead, when the responder (aka server) sees the non-anonymous ID, it will perform a DNSSEC secured lookup to obtain the IPSECKEY out of band. Both parties confirm there is no MITM.

The advantage of a) is that it leaks less user information and makes tracking users harder. The client can regularly generate another anonymous keypair. The disadvantage of a) is that it turns peers into clients and servers. And two clients cannot initiate OE to each other.

6) The tunnel is established and the IKE daemon notifies the local DNSSEC server that had instructed it to setup the IPsec tunnel.

7) The resolver releases the IP address to the application.

8) The applications starts sending packets and the IPsec policy encrypts them al.

I’m personally in favour of the @anonymous solution. But there is no reason why support for both could not be implemented.

What are some of the obstacles and work to do:

1) writing the unbound plugin

2) writing the support for @anonymous for the server-side. This includes raw keys for IKEv2 (draft-ietf-ipsecme-oob-pubkey)

3) With NAT, the client suggests an inner-IP. This could be abused or clash, We need to ‘contain’ each connection, possibly using generated ipv6 addresses 4) We cannot use the “gateway” field of RFC-4025, or people could trick a server into giving a client all communication to a certain IP address that does not belong to them

5) anonymous connections should generate throw-away keys to remain anonymous

6) implement draft-wouters-edns-tcp-chain or else latency/RTTs will prevent real-life deployment of DNSSEC validated IPSECKEYs on mobile devices.

7) This allows no upgrading from anonymous to mutually authenticated, but IKE policies can be added to the server/client that would match on different IDs (eg X.509) that work independantly of OE without introducing complicated channel binding promotion code. Other IKEv2 extensions could possible be applied to facilitate promotions.

I’m sure more implementation issues will show up once we get this going, but there are no real fundamental issues why we cannot deploy this in a couple of months of time. My plan is to get libreswan to support this version of OE. Additionally, once we use draft-wouters-edns-tcp-chain, it becomes cheap to do these lookups through the tor network. If the tor exit nodes then also feed each other with DNSSEC cache material, it should make tracing individual clients even harder.

(anyone willing to assist, especially with coding, do contact me)

Can the NSEC3-OPTOUT record cover no RRTYPE’s?

At first glance, the answer seems obvious: No. The point of NSEC3 is to proof the (non)existence of data. With the OPT-OUT flag set, you skip all the non-secure data. So you would expect to have some data to cover for such an NSEC3 record. If you read RFC 5155 Section 3.2.1 it is not entirely clear, though I’m tempted to think these should not happen in the wild. Except we just found one, generated by bind’s dnssec-signzone:

aaa.ottawa.on.ca.  IN NS foo.
problementry.ottawa.on.ca.  IN NS foo.
problementry.ottawa.on.ca.  IN DS blob.
zzz.ottawa.on.ca.  IN NS foo.

The change was that “problementry.ottawa.on.ca” submitted a DS record which was added to the zone. Note also that  neither “on.ca.” or “ottawa.on.ca.” are zone cuts, although there are many delegations underneath XXXX.ottawa.on.ca. (but none with a DS record or orphan glue)

This caused bind to add two NSEC3 entries as part of the NSEC3 chain, an entry for HASH(on.ca) and an entry for HASH(ottawa.on.ca) (forgive the wordpress wrapping of lines)

+qf6k201pudkgikusoidrujdfikql15qc.ca.   3600 IN      NSEC3   1 1 5 – qnkecttopnji0h479fhpjmv18gsl1sdk ;{ flags: optout}

+qnkecttopnji0h479fhpjmv18gsl1sdk.ca.   3600    IN      NSEC3   1 1 5 – r85l4g712aibs65e47aj79e7odi202h9 ;{ flags: optout}

Note how these NSEC3 entries do not cover a single RRTYPE, because there is no zonecut and there are no RRTYPE’s for these entries.

When signing this zone with opendnssec, instead of bind, the entries are not added. Who is right? Before using my brains, I decided to use other people’s brains.

named-checkzone loaded both the bind and ods version with “OK”. So no help there. It’s a bug because it should declare at least one of the copies as containing errors in the NSEC3 hash chain

ldns-verify complained about ALL the non-zone-cuts in the zones but otherwise showed no difference. So that’s also a failure, on top of the failure of assuming dots are zone cuts.

validns found the bind zone OK, but complained about the ods zone with:

ca.signed.ods-signer-01.ldns:2898958: no corresponding NSEC3 found for on.ca.
ca.signed.ods-signer-01.ldns:2907968: no corresponding NSEC3 found for ottawa.on.ca.

So validns seems to agree with bind. Which software is right? The only thing I”m sure of now, is that I need more coffee and time to re-read RFC 5155, and specifically look at how things are supposed to work in this case.

I’m continuously surprised at how many TLD’s have rolled out DNSSEC, yet we’re still seeing differences between signer engines and validators all the time……

 

DNSSEC software bug causing nohats.ca down time – possible catch22

I sign my own domain nohats.ca using OpenDNSSEC. Since .ca is not yet signed, I added my key to the ISC DLV Registry. It is enabled by default by Fedora and RHEL if you install the bind/unbound name server for resolving.

Today, I removed and added a few zones to my opendnssec (an alpha version, 1.4.0a) based signer. These domains were unrelated to nohats.ca. But somehow I ended up with a “signed” nohats.ca zone that contained NSEC3 records but no RRSIG records, and one other domain that only contained 1 RRSIG record. As a result, anyone who was using DNSSEC could no longer resolve my domain nohats.ca. That included me. The nohats.ca domain is still in this weird state inside opendnssec, so I decided to remove the DLV record for that domain. Turns out I forgot the password to my login at dlv.isc.org. With my mouse hovering over the “request password reset” option, I realised that my MX records point to “nohats.ca”. If the dlv.isc.org site uses DNSSEC, they will fail to resolve the MX record to send me my password reset information to fix my DNSSEC setting. Catch 22!

Whether by design or by sheer luck, this did not seem to be the case and I received my password reset, and have removed the DLV record for nohats.ca.

I upgraded opendnssec to 1.4.0a2, but this has not resolved the issue of ods-signerd giving me a zone without RRSIGs. I’ll have to investigate this more tomorrow. Our tools really still have lots of room for improvement.

But it does bring up a delicate point. Registrars should ensure that there is a way you can remove a DS record using an account that somehow can do a password reset even if your only email goes into a failing DNSSEC domain. Perhaps a two factor using a text message to a phone? Or perhaps allow more then one email address for a reset, so that people could include use two different email addresses in two different domains. Though one should not point password resets to domains that are not secured by DNSSEC, because these are precisely the kind of messages people could abuse to hack access to your domain account. Another security catch 22.

So if you are a registrar, please think about this issue. Sooner or later one of your customers will be the position I found myself in…..

You can’t P2P the DNS and have it too

As DNSSEC evangelist, I am often told that DNSSEC cannot solve the fundamental problem of censorship and domain take down. And I agree. However, these people then follow up with mumblings about “P2P DNS” and how a decentralized DNS would solve this censorship and central control problem.

Unfortunately, per definition, there cannot be a decentralized name lookup service. Let’s take a step back and think again what the DNS is.

An address is only an address if we all agree

I live in Toronto. You might know where that city is located. If not, you can ask someone else you trust. Or you can ask a trusted company, like Google maps, about this “Toronto” thing, and it will tell you where this city is. Or you open 3 different atlas books by different vendors and you look up “Toronto” and compare the results.

The underlying assumption here is that there is only one LOCATION that is known by the NAME Toronto. We all know where “Toronto, Canada” is because we all agreed to name this particular location on the Earth with that globally unique identifier (mnemonic) that is easier to remember then “Let’s have coffee at Latitude 43.6444208 and Longitude -79.4025781”

And we see where this system can break down. Depending on who you ask, “China” refers to something slightly different. According to some, “Taiwan” does not exist. Do you remember a few years back when you suddenly realized that the city of Mumbai was actually what you knew until then as the city of Bombay? In a relatively brief period (a few years) everyone around you started using Mumbai instead of Bombay. This name change came in through some centralized authority and spread down the hierarchy all over the world. The name change within Mumbai undoubtedly went much faster then the update of the school atlas in Wiggins, Mississippi, USA. And just like DNS, this update of a city took some time, because people “knew”  (cached) that it was called Bombay without looking at the new atlas version. Every piece of information update has a “time to live” where you just cannot assume people are aware of the new name.

People have to agree to what an address location is or it won’t work. If I decide
to rename “100 Queen Street West, Toronto” into “Fnord Hall” and no one else knows
about the name change, we won’t be able to meet up for coffee at Fnord Hall because
you have no idea where it is or even how to look up what addressing scheme I used.
Similarly, if the people of Queen Street replace all the signs and decide to call
it King Street, there will be some confusion as to which of the two King Street’s
you might be referring to.

In other words, address agreements are not “peer to peer”. There is a centralized, or rather a hierarchical method for assigning, reassigning and distributing location name updates. And the only reason it works, is because there is censorship and central control.

The DNS is an addressing system. Any P2P DNS alternative still needs to have this property of everyone agreeing to give a certain name to a certain location. And with that comes authority to make changes and disagreements about certain changes (censorship)

The Name mapping problem

The Domain Name System is a lookup mechanism similar to the Geo Location Name System (AKA “atlas”) The DNS serves to match an easy to remember mnemonic (“nohats.ca”) to an impossible to remember IPV4 or IPv6 address. It ensures that no two entities can claim the same mnemonic. And it is also really really big. If we look just at the COM, NET and ORG we’re talking in the order of magnitude of 100 million entries. If you do not live in Canada, your atlas likely does not contain the exact location of 100 Queen Street West, Toronto, Canada. If it did, your house would be filled with just your atlas.

The DNS is hierarchical,and its trust model is hierarchical, assuming you deploy it with DNSSEC. It has censorship and central control. And that’s why it works.

So let’s say you have the perfect decentralized P2P DNS table. It can uniquely represent every “domain name”, using some bitcoin like system of random numbers and transaction logs copied all over the world in shorter times then current DNS TTL levels, supporting over 100M ever changing entries. This in itself would be an incredible, though unlikely, engineering achievement, but let’s assume you people are super smart, outpacing the collective 25 years of IETF DNS experiences, and you managed to built, distribute and continuously update a secure P2P DNS table.

So let’s say my web server is at 438946936129363483649513524349734902630640.namecoin.

Well that’s great. This is about as useful as saying that my web server is at 2001:888:2003:1004:c2ff:eeff:fe00:102. The only difference is that the first would be (assumed) cryptographically secure somehow. But no one is going to advertise their site as 438946936129363483649513524349734902630640.namecoin.

So now you have to design some mapping system, where this name maps to something easy like “nohats.namecoin”. Great, so now who runs this mapping service? Who is preventing someone else from taking” nohats.namecoin” and pointing it to their server? Who is going to release this name in 100 years when I’m long gone? When you start answering these questions, you quickly develop something of a Registry-Registrar-Registrant system, and you just re-implemented DNS except in a new TLD called “.namecoin” (or .onion, or whatever your name scheme will be)

To use this mapping system, you either need to have clients to do the lookups all by themselves (sort of like running a DNS resolver on your laptop) or you are going to chain proxies (sort of like recursive nameservers of your ISP). And guess what, governments will filter it forcing you to use their proxies, which will conveniently filter out some entries. At most, you can reach the level of DNSSEC where this tampering can be detected. But so far, you haven’t actually improved on the current DNS/DNSSEC system.

How can we ensure trust in the mapping service?How can you detect tampering? In the “trust no one” model, this needs to become an “n out of m” trust system. Not even Chuck Norris or Moxie Marlinspike  can vouch for 100M name mappings. Can you trust 100 Ambassadors? Or 90 out of 100? And what do you do when you DO find a lack of trust level in a name mapping? Is your browser just going to fallback to DNS or prompt you with “Add security exception”?

Systems like this have been devised for SSL certificates. CertPatrol, EFF’s trusted certificates, and Moxie’s Convergence all have this problem, and so will any P2P DNS approach. These systems fail because they don’t scale. If an update happens in these systems, these “trusted ambassadors” have to play catch up with reality. So every time your P2P DNS would change its mapping, it will take some time and false positives before things work again. What is really needed is that these updates are authenticated by the publisher, who is the only entity that really knows what is real and what is forged. And this is exactly what DNSSEC provides.

To summarize this part, any cryptographically secure unique identifier that is a replacement for DNS will introduce the exact same problem of needing a mnemonic mapping service that such a new solution is trying to obsolete.

Domain confiscations
Another motivation for P2P DNS is that no central authority can take a name identity away from the “real” owner. ICANN or Verisign can take away any com/net/org domain, and we have seen this happen. The solution is not to replace DNS with some decentralized system, because again that would be moving the problem around.

Who runs the new mapping service? Under what legal system will these mapping servers be running? Or alternatively, which guns are going to enforce that these mapping servers will be hosting, or not hosting, certain content? Or which banks will block payments to keep this alternative name mapping system up and running? Look at how torrenting distributed the crime of downloading movies illegally away from centralized websites to individual participants. People are still sued under the legal system of the physical location they reside in.

Unfortunately, all these new TLDs by ICANN will be under the same controls as com/net/org, so don’t get your hopes up there. The only known solution to defend against this, is to use the TLDs of various different countries at the same time. Remember when wikileaks.org was down and you could still access them via wikileaks.ch, wikileaks.de and wikileaks.is. That’s the only way I know of ensuring your “name” will be up in one of the various legal systems in the world.
Your chances of name survival increases if your do not use US based Registrars and DNS Operators for your non-US based domain registrations, or else you are still putting all your eggs in one legal system.

DNSSEC cannot be trusted because the USG holds the root key.

Finally, people tend to claim that the root key is Sauron’s ring, and whoever has it will get tainted by political or military requests to forge or remove data the DNS hierarchy. And they have the cryptographic key to do so because everyone is forced to trust this key as entry point into the DNS.

First of all, there is an Enigma Problem with that. As soon as someone posts a forged DNS record signed by the root key that is obviously not legitimate, people will quickly move away from such a root key. Very likely, the ccTLDs would get together and create an alternative root zone, split off their hundreds of root nameservers from the Verisign a.root-servers.net server, and then the US would never have this power again in the future. Such an event will happen exactly once.  Whoever would be the target of such forging would have to be a very valuable target indeed. If that is you, you have better things to do then reading this posting.

Second, DNSSEC did not change any of this. The legal system in place already allows the US government to take a domain away. And all the individual countries have similar provisions, or can enact such laws at any time. It is the legal system that will force the hand of the TLD operator. This situation is not different with or without DNSSEC. The
only difference is that with DNSSEC, there is a trail of cryptographic proof, so outsiders
actually can tell much better what happened and who was responsible for it.

More importantly, if you really don’t want to trust Verisign, or PIR, or IANA, you can sit
down with the people you DO trust and start one of these ambassador/convergence schemes for DNSSEC keys just like these systems do for SSL certificates. Collect and distribute all the ccTLD DNSSEC keys and publish it somewhere with appropriate cryptographic security and authenticity. Anyone can then regularly grab that information and prime their DNS servers with it, and all your ccTLDs will be safe from tampering by the US Government, ICANN, Versign, IANA or the root zone operators.

Back to reality: Registrar compromise does affect DNSSEC

A much more likely scenario is that of a compromised registrar, where an attacker gains access to your domain information including NS servers and DNSSEC public key material (DS records) and is able to change the entire domain to point to their own servers, update your DNSSEC trust anchor for theirs, and so validate their hostile take-over as being the cryptographically proven site you are running.

For your own domains, you really want your resolver to hard code that key, overriding any parental DNSSEC information. A registrar compromise will at least not compromise you further as you use passwords or other credentials against what you (cryptographically) think are your own proven servers, but which are really malicious. It might not help the world at large, but if you are guaranteed to find out, at least you can start the repair process. And you already have some kind of monitoring for this in place called “ssh known hosts”.

Image one day you’re having coffee at your local Star Bucks, and you cnnect to your
server and see this:

[paul@thinkpad ~]$ ssh nohats.ca
[paul@thinkpad ~]$ ssh root@nohats.ca
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
69:e5:1e:eb:2b:7c:71:59:ca:6f:17:3a:ce:6c:80:b6.
Please contact your system administrator.
Update the SSHFP RR in DNS with the new host key to get rid of this message.
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the RSA key sent by the remote host is
69:e5:1e:eb:2b:7c:71:59:ca:6f:17:3a:ce:6c:80:b6.
Please contact your system administrator.
Add correct host key in /home/paul/.ssh/known_hosts to get rid of this message.
Offending RSA key in /home/paul/.ssh/known_hosts:560
RSA host key for nohats.ca has changed and you have requested strict checking.
Host key verification failed.

This particular errors shows up when your server was compromised or your routing was changed, while your DNSSEC was intact. If they had also changed your DNSSEC, you could not see the message about the SSHFP record being incorrect, but you would still see the RSA changed key warning, pointing out a strange new discrepancy between ssh reality and DNSSEC reality

You can decide to monitor a few domains you do not control but that you find important, like twitter or Facebook or Gmail. But again, this runs into the problem described above where you will suffer false positives when they legitimately change something.

DNSSEC actually helps you

With more and more devices running DNSSEC locally, it is actually becoming much harder to spoof or forge DNS. When governments dictate that ISPs need to filter or rewrite DNS requests, DNSSEC will detect this as malicious. This is why SOPA really failed. Not because politicians changed their mind about the legality or morality of hostile domain take over. SOPA failed because the DNS experts convinced the politicians it was totally useless because of DNSSEC.

DNSSEC even helps you against the NSA

Let’s assume Verisign had to give the NSA copies of the private key of the root zone, and the NSA can sign anything from the root down. They don’t have a copy of the .ca key, but since they have the root key, they can replace the public key for .ca (DS record) in the root zone with a new RSA key they just generated for this purpose. And then they repeat this process for “nohats.ca”.

To mislead my readers who think they are going to “nohats.ca”, the NSA intercepts your laptop’s DNS, and uses DNSSEC signed data to convince you my server is in Virginia. Doesn’t this mean DNSSEC would be useless?

Not really. Because this intercepting of your DNS is actually not an easy problem. First of all, they need to hide their tracks. If they change my domains in the “global DNS view”, then this is bound to be detected by me. So they have to do this in a very targeted matter that would only affect you, but not the masses. But with DNSSEC, you can ask any server in the world via any VPN tunnel or TLS connection for DNS data. In fact, currently my laptop will attempt to talk DNS over TLS to Fedora Project DNS servers when it detects that regular port 53 DNS is getting mangled by a hotspot. Or I could be using TCP connections via the TOR network to get my DNS lookups from a server in The Netherlands. Or I would be connected to the Red Hat VPN, and my DNS lookups originate from Rayleigh, North Carolina. So while my laptop is at a certain Star Bucks wifi location in Toronto, the NSA cannot just forge that local hotspot DNS. They will basically have to contaminate the global DNS view to ensure I am being properly misled. So anyone who runs a simple monitor script to check their DNS contents would immediately know their domain’s DNSSEC key was compromised. Or again, you could run some ambassador or n out of m system for this monitoring to detect such DNSSEC key compromise, though this system would also have to deal with false positives when real DNS updates happen.

Note that if the NSA would be that interested in your laptop traffic, they would be better of just redirecting your IP stream and leave the DNS alone. Though that method will finally come to an end soon with the new TLSA DNS record type, which I will explain in another post.

Conclusion

DNSSEC provides valuable additional security to the Domain Name System, even if you believe the root components such as Verisign cannot be trusted. DNSSEC does not protect you against the laws of nation states you disagree with, but then again, no technology does that with perhaps the exception of the cloaking device.

Any P2P DNS system that claims to better then DNSSEC, has not understood the actual problems. You just need to find and point out where in their system they are outsourcing their trust. Just like the DNS system.