Ubuntu You’re Not Ready For Production Yet

Posted: January 15, 2012 in Guides
Tags: , , , , ,

This past week I had a very stimulating discussion with a very talented co-worker of mine. This discussion was about Exec-Shield (DEP for all my Windows fans) and ASLR under RedHat. Through the course of this discussion, we discovered a very interesting script that helped us detect which of our running processes were protected by exec-shield as well as some nifty kernel tunables for exec-shield, and ASLR. Incidentally, he wrote a very informative article about it over at prefetch.net, you should check it out. Also he has a lot of other great nix sysadmin stuff there that is definitely worth your time.

Now you’re probably thinking. Hey DT what’s this got to do with Ubuntu, you’re talking about RedHat, what’s with the sensational title?

To that, I say. “Dear reader, you catch on fast. It has absolutely NOTHING to do with Ubuntu.” Therein lies our problem, for a server OS that Canonical touts as production ready with nifty slogans like “We are cloud.” I was somewhat astonished to find that exec-shield support in Ubuntu is rather limited (read nonexistant).

What is this exec-shield and why do I want it?

Exec-Shield also referred to as NX bit and DEP is a system designed for 64 Bit PAE (Physical Address Extension) systems to divide executable code into two sections, a .data section and the actual executable instructions. Basically this helps you mitigate stack, heap and int based overflows by mitigating the executability (is that a word?) of injected shell code. The overflow will still happen, however no payload execution happens. This is obviously a nice thing to have happening.

How do I know it’s enabled?

Well quite simply on a RHEL/CentOS/FC box if you do the following

sudo sysctl kernel.exec-shield

You will get some output that signifies what setting you have for exec-shield. Default is 1

kernel.exec-shield = 1

It should look like that. Really quick, a rundown on the possible values.

  • 0 – Disabled, regardless of whether the application is compiled to support it.
  • 1 – Enabled if the application is compiled to support it.
  • 2 – Enabled for all applications unless they are compiled without it.
  • 3 – Enabled regardless of what the application is compiled to support.

But what about Ubuntu? If you were following along with Ubuntu you likely got an error when you typed that command. Why is that you ask? Quite simply, your kernel does not support Exec-Shield.

Time to switch to RHEL?

Ideally, for production systems yes, for many reasons not just this, but it’s expensive (CentOS isn’t ;-)). That being said you know I love Ubuntu. So, I have an alternative solution for you.

Emulating Exec-Shield on Ubuntu

There are a couple of things to understand here before we start. Debian upstream and thus Ubuntu removed the exec-shield patch from the distribution a few years ago, claiming it was replaced by gcc’s compile time optimizations. There is a major flaw in this logic, these enforcements are made at the application level, which is a terrible place to have them made. The kernel level is much more suitable for this type of control. Incidentally, the two work wonderfully together.

We do not have access to exec-shield, however we can emulate it through grsec, which is exactly what we are going to be doing here.

Note : this involves recompiling your kernel, so breakage could ensue.

Note+ : You will also need build essential for this

sudo apt-get install build essential

Note++ : For make menuconfig we’ll also need ncurses-dev.

sudo apt-get install libncurses5-dev

Step 1 : Downloading grsec

The first step is to obtain the latest stable release of grsec.

wget http://grsecurity.net/stable/grsecurity-2.2.2-2.6.32.53-201201101724.patch

Step 2 : Downloading Kernel Source

Next we need the kernel source, this has to be done with an unpatched kernel so we can not use our existing kernel source.

wget http://www.kernel.org/pub/linux/kernel/v2.6/longterm/v2.6.32/linux-2.6.32.54.tar.gz

Once it’s downloaded extract your kernel source

tar xzvf linux-2.6.32.54.tar.gz

Step 3 : Patching the kernel

Now we’ll patch our kernel source.

cd linux-2.6.32.54
sudo su -
patch -p1 < ../grsecurity-2.2.2-2.6.32.53-201201101724.patch

Step 4 : Configuring our patched Kernel

Now we run

make menuconfig

Most of the defaults are acceptable however we need to look at our “Security” Options

 

 

 

 

 

 

 

The first thing we’ll look at is GRSecurity. There are a variety of options in here. You will need to explore the ones that fit your needs best. However I recommend enabling the following

  • Security Level High
  • Hide /proc from users
  • Sysctl Support On
  • Enable Features by default On

There are many more tunables that you can fiddle with here.

There are also quite a few tunables under PaX that can be adjusted here however, the defaults are quite acceptable.

 

 

 

 

 

 

 

Once you’re done making changes we’re ready to compile the kernel.

Step 5 : Compiling our Grsecurity Kernel

For non-Ubuntu distros (since this has been pretty generic up until this point) we can simply

make

Alternatively for Ubuntu users we can build our kernel in the Debian method by doing the following.

make-kpkg clean
sed -rie 's/echo "\+"/#echo "\+"/' scripts/setlocalversion
rm localversion-grsec
fakeroot make-kpkg --initrd kernel_image kernel_headers

Note : If you follow this step you may simply use dpkg -i to install your new kernel. Users following the other method should update their grub config and install the kernel in the normal method.

After Kernel Installation

Once we have our GRSecurity enabled kernel installed we can use the gradmn tool to put GRSecurity in learning mode and create a policy which will be enforced.
You can learn more about doing that here.
Once you have done this your Ubuntu server will be much more ready for production usage. Though…Installing CentOS takes far less time 😉

Advertisements
Comments
  1. Dan Dieterle says:

    Hey bro! Long time no hear. Great post, I just have a quick question. It’s kinda off the topic, but I have friends that support a large Ubuntu desktop install base and they are furious with Ubuntu 11.10.

    They hate the Unity only be default install and they have had a lot of upgrade issues when upgrading to 11.10.

    They (and I) don’t want to switch to another Linux distro, just curious if you have heard anything or if the next version would be any better?

    Thanks Adam,

    Dan

    • dangertux says:

      Hey Dan! Great to hear from you again. I’ve been a little sketchy with blogging lately 😦

      However, to answer your question the new version of Ubuntu (12.04 LTS) is currently in alpha and is available here : http://cdimage.ubuntu.com/daily-live/20120114/

      More directly, Unity is here to stay. (Whether we like it or not). I’m not personally a fan however, we still have options. While Unity is Ubuntu’s flagship desktop environment we have the choices of KUbuntu, LUbuntu and XUbuntu which use KDE, LXDE and XFCE respectively as their desktop environments.

      I am somewhat partial to KDE and use it regularly in Fedora, BT and Ubuntu over Gnome. You can get Kubuntu here : http://www.kubuntu.org/getkubuntu

      These options keep Ubuntu, the underlying OS the same however switch out the desktop environment for something a little more traditional. Also you can install Gnome Shell, which is Gnome 3’s default environment, though it is more similar to Unity so might not go over well either.

      Additionally if you want a Ubuntu-like experience while still maintaining that gnome2 feel you can check out Linux Mint that uses MSGE extentions for Gnome 3 to deliver the classic “Gnome 2” look. Available here : http://www.linuxmint.com/

      At its base Mint is either Ubuntu or Debian, so not much different from an admins point of view.

      I believe you can also install the MSGE extensions directly into Ubuntu and keep the rest 100% Ubuntu. That is explained here : http://askubuntu.com/questions/79701/linux-mint-12s-mgse-interface-in-ubuntu

      I hope all is well with you and yours, and I hope this has been helpful!

      Adam

  2. cypherpunks says:

    Quick question: How does ExecShield differ from hardware-supported NX? The Ubuntu security Features page says that NX hardware support prevents non-executable memory from running code. This sounds very similar to what ExecShield provides in software. See https://wiki.ubuntu.com/Security/Features#nx

    The main thing that interests me is that I’ve noticed that on RHEL/Fedora, ExecShield is often disabled on the programs that need it most for desktop users: Firefox, Firefox plugins, and java-based apps. Basically anything that needs a JIT to rewrite code.. I am wondering how NX support handles this case, if at all. There seems to be no way to tell.

    Can you please enlighten me?

    • dangertux says:

      ExecShield compliments hardware NX in systems that allow it (64 bit physical address extension systems). It compliments it by giving the option to enforce it over pid’s that would not normally be protected due to the fact they were not compiled to support NX. This of course can and likely will cause breakage.

      In legacy systems it can bolster a system that does not innately support NX via the use of a PAE kernel. In this use case it is emulated in the kernel layer as opposed to being hardware enforced.

      In Just in Time compiler based applications like you mentioned NX is unfortunately not really an option and the best remaining option here would be mandatory access controls.

      Hope this helps

      • cypherpunks says:

        Do you know if there is a way to tell which apps were compiled to support NX in the absence of ExecShield on Ubuntu, and which were not? I do have this kernel message in dmesg:
        [ 0.000000] NX (Execute Disable) protection: active

        Inspecting /proc/pid/maps seems to show that there are many mappings without the execute bit for apps like Firefox.. Does this mean I have some protection there?

  3. dangertux says:

    If there is no mapping for the execute bit, then it would have to be created at run time via mmap you should be able to strace whatever pid you’re trying to figure this out for and determine if this is happening.

    Also if it IS happening via mmap/mprotect then you could use something like SELinux to control that since there is no discretionary control over these.

    I hope that this didn’t get too convoluted. Also there is a script linked in the article over on prefetch.net that works under RHEL, you might be able to hack something up to work with Ubuntu out of that.

    Hope this helps.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s