Posts /

Memory Security in Go

Twitter Facebook Google+ Subscribe
03 Aug 2017

This post is intended as a starting point for anyone needing to manage sensitive information in Go, and as far as I can tell, this is the only post of its kind.

A few months ago, while working on dissident, I started looking around for guidance on how I should manage encryption keys. I found a few references here and there, but the best method I could patch together, with the limited information I had, was something like:

  1. Call mlock(2) (or VirtualLock) on any sensitive resources.
  2. Overwrite the resources when finished with them.
  3. Allow them to run out of scope.
  4. Pray for the garbage-collector to clean them up, or force it with runtime.GC().

So, I patched together a little library that handled some of it for me, chucked it on HN, and moved on.

A short time later, I realised that beef was kicking off. As it turned out, the aforementioned approach was fundamentally flawed as one thing hadn’t been accounted for: the garbage-collector. It goes around doing whatever it feels like doing; making a copy here; moving something around there; it’s a real pain from a security standpoint.

I really didn’t have a choice at that point: I had to dedicate all the time I could spare to the project, for the ten days it took to develop and release the fix.

A few people had mentioned wrapping libsodium, but I wanted a pure-go solution, so that wasn’t ideal. Instead, myself and one of my closest friends, @dotcppfile, began analysing how libsodium actually worked. Well, I say “we”; more like he pretty much audited the entire library while I checked the documentation and implemented the relevant system calls.

Within a few days, we had a pretty solid understanding of libsodium and we were ready with a new and improved plan. I think the best way to explain it is to introduce you to the end product: memguard.

Alright, now say you need to generate an encryption key and store it securely. You would probably write something like:

On line 16, a new read-only LockedBuffer is created, filled with cryptographically-secure random bytes. When creating it, the first thing we do is calculate the number of pages that we have to allocate. In this case, the length of the buffer is 32 bytes and we can assume the system page-size to be 4096 bytes. The data is stored between two guard pages and is prepended with a random canary of length 32 bytes (more on these later). So, since the data and the canary together will comfortably fit into a single page, we need to allocate just three pages.

But we can’t ask the Go runtime for the memory—since then it is free to mess around with it—so how do we do it? Well, there are a few ways to accomplish this, but we decided to go with Joseph Richey’s suggestion of using mmap(2) (or VirtualAlloc on Windows), since the system-call is natively implemented, and that allowed us to avoid a dirty cgo solution.

Well, the Unix system-calls were natively-implemented at least, the Windows ones were not. Luckily, there was this library by Alex Brainman that we could vendor instead, and it proved invaluable. (I did later add the missing system-calls to the standard library to remove the dependency.)

So, now that the pages are allocated, what next? Well, we should probably create our guard pages. We tell the kernel to disallow all reads and writes to the first and last pages, so if anything does try to do so, a SIGSEGV access violation is thrown and the process panics. This way, buffer overflows can be detected immediately, and it becomes almost impossible for other processes to locate and access the data.

The remaining page, the one sandwiched between the guard pages, needs to be protected too. You see, as system memory runs out, the kernel copies over the memory of inactive processes to the disk, and that is something we would like to avoid. So, we tell the kernel to leave this middle page alone.

The last thing is the canary: a random value placed just before the data. If it ever changes, we know that something went wrong—probably a buffer underflow. When the program first ran, we generated a global value for the canary, so we just set the canary bytes to that, and the container is pretty much ready for use.

A diagram showing how our three pages look now.

All that is left to do now is handle the data itself. In our case the function NewImmutableRandom() was called, which fills the created buffer with cryptographically-secure random bytes after it is created. A read-only status was also requested, so after the buffer is filled, we tell the kernel to only allow reads from the middle page. As before, any attempts to write to the buffer will trigger a SIGSEGV access violation and the process will panic.

And that’s pretty much it. You can view the official project page here, and full documentation can be found here.

Now, memguard is not meant to absolutely guarantee the security of the data, but it is the best that you can reasonably hope to achieve. If you have a suggestion for improvement, feel free to open a pull-request: contributions are welcome.

Like what you’ve read or have something to say? Contact me on Twitter!

Twitter Facebook Google+ Subscribe