LUKS without LVM (for Dropbox)

Since Dropbox fucked us all over by dropping support for filesystems that aren’t ext4, all the cool things you used to be able to do like encrypting your home folder dead easily with ecryptfs, snapshots with btrfs, LVM etc. are all stuffed. Now the OS must only see a pure ext4 file system, otherwise the Dropbox client will detect it and refuse to synchronise.

There is a solution using loopback mounting to create an ext4 specific to Dropbox; that’s fine but it means you are setting aside a certain amount of space to dedicate to Dropbox, and if your Dropbox gets bigger than that you’re going to have a pain resizing it. Instead I wanted to encrypt the root partition as well, and I want the simplicity and flexibility of one home directory not carved up into special partitions for Dropbox. So this is how I used LUKS to encrypt all my partitions, but without putting LVM in the middle.

This was tested with Mint 19. Should work on Ubuntu and probably Debian.

Intended audience

You:

  • Have some experience with Linux so don’t need total hand holding
  • Don’t want to run a script to configure all this as you want to see/learn what’s going on
  • Aren’t in super-paranoid security mode
  • Want to dual boot
  • Not bothered about hibernation
  • Happy with a swap file instead of a swap partition (default in recent ubuntus)

Windows preparation

 

  1. Boot new laptop into Windows
  2. Perform all Windows updates
  3. Perform all manufacturer driver updates, especially BIOS
  4. Create a recovery drive (allows us to delete the recovery partition and get that space back)
  5. Run Manage BitLocker and switch off BitLocker (needed so that the Linux installer can safely resize it).
  6. Run Disk management and make sure you understand what partitions exist and what they are all for.
  7. Use the BIOS to disable Secure Boot. You can switch it back on later and it’s just going to cause you a problem in the short term.

Partition preparation and OS installation

  1. Boot from Linux live USB (in my case Mint 19)
  2. Confirm that you have booted in UEFI mode (and not legacy mode). Open a terminal and run:
    efibootmgr

    It should return a list of bootable devices and the current boot order. If it returns an error about EFI variables not being supported, then you are booted in legacy mode and need to fix that first.

  3. Run gparted (you can’t do this using the partitioning tool built in to the installer, because the moment it has finished creating the partitions it will then try to start installing to them. You need to insert the encryption set up in between these steps, so we’ll do it before even running the installer).
  4. Delete recovery partitions you have decided you don’t want; you won’t need these if you made the recovery drive above. The partition that was called WINRETOOLS in Windows (about 1Gb) can go.
  5. Resize the main Windows partition as small as you dare to without making it unusable. These days I’d recommend at least 90Gb if it’s a secondary OS. Mine’s 55Gb after updates are all applied and before I’ve installed any apps or put any data on it. Yikes.
  6. In the free space you now have, create your desired Linux partitions as ext4. Make sure you set labels and partitions names. You WILL need a separate /boot partition, and GRUB has to be able to find the kernel image in an unencrypted form in order to then access then other encrypted partitions.Here’s my choice for a 512Gb drive:
    New size (Mb) Mount point File system type File system label
    800 /boot ext4 BOOT
    50000 / ext4 ROOT
    Rest /home ext4 HOME

    I separate root and home so that I can reduce the reserved space on /home to 0 whilst maintaining it at 5% on /; I can fill my home drive to the tippy-top without destabilising the system.

    50Gb might seem a lot for the root partition but Linux distributions are getting as big as Windows. I’ve been running a 25Gb root partition for the last few years and I’m getting bored with having to keep tidying it up so that I’ve got room to download updates.

  7. Open a terminal, and enter
    sudo bash

    to switch to a root prompt.

  8. Next enter
    fdisk -l

    list all your new partitions, and note the Device names for them.

  9. Now we’ll encrypt the non-boot partitions. You’ll need to invent some secure passphrases for them. For the root partition you’ll be entering this every time you turn the computer on, so make this secure but typeable. For the others they’ll just be emergency backups so you can make them a bit weirder if you like. To encrypt:
    cryptsetup luksFormat 

    once for each partition that is NOT /boot. So for me:

    cryptsetup luksFormat /dev/sda5
    cryptsetup luksFormat /dev/sda6

    You’ll get prompted for your chosen passphrase for each. Beware keyboard layouts; the live USB may have chosen a US keyboard layout, so if your chosen passphrase contains any characters that would be different on a US keyboard then you will not be setting the password to what you think you are. Remember that in future if something has gone wrong during boot and you are dropped to a root prompt or even being prompted by the kernel for the passphrase for the root paritition, you may not have the luxury of selecting they keyboard layout. So probably choose a passphrase with just letters and numbers. Or even just letters: making it longer is much better than using a bigger character set.

  10. Now we’ll unlock them so that the installer can see them and use them as partitions. To unlock:
    cryptsetup open  

    once for each partition that is NOT /boot. So for me:

    cryptsetup open /dev/sda5 cryptroot
    cryptsetup open /dev/sda6 crypthome

    This creates the two pseudo-partitions /dev/mapper/cryptroot and /dev/mapper/crypthome which the installer can treat as unencrypted partitions and write to normally.

  11. Now we’ll format them:
    mkfs.ext4 -L  /dev/mapper/

    once for each partition that is NOT /boot. So for me:

    mkfs.ext4 -L ROOT /dev/mapper/cryptroot
    mkfs.ext4 -L HOME /dev/mapper/crypthome
  12. All is now ready for OS installation. You’ll need at some point to override the installer’s partitioning scheme, choosing Custom or Something else or whatever option allows you to specify what partitions are used and where. Then set the unencrypted boot partition to be mounted at /boot, and the new /dev/mapper/cryptroot and other partitions to have their appropriate mount points. You don’t need to format them as they are new. Make sure the boot loader is configured to be installed on /dev/sda. Don’t select any options for encryption in the installer – you’ve already done it. Once installation is complete, don’t reboot. There’s more to do yet.

Post installation fiddling

From this point onwards I’m going to assume that you are capable of substituting your device names, map names, labels etc. if you have used different values to me, and I’ll just show you what I used.

  1. Now we’ll mount the newly-populated partitions to do some fiddling. Run
    swapoff -a
    umount /dev/mapper/cryptroot
    umount /dev/mapper/crypthome
    mkdir -p /mnt/root/boot/efi
    mount /dev/mapper/cryptroot /mnt/root
    mount /dev/mapper/crypthome /mnt/root/home
    mount /dev/sda4 /mnt/root/boot
    mount /dev/sda1 /mnt/root/boot/efi
    mount --bind /dev /mnt/root/dev
    mount --bind /proc /mnt/root/proc
    mount --bind /sys /mnt/root/sys
    

    /dev/sda4 is my boot partition, and /dev/sda1 is my EFI partition (created by Windows). Consult your fdisk -l output from earlier to see which ones yours are.

  2. Now we’ll grab the UUIDs of the encrypted partitions we made:
    lsblk --paths --output=NAME,UUID | grep -v mapper
    

    You’ll need the UUIDs of your root and home and any other partitions.

  3. Now we’ll tell the OS to unlock our encrypted drives while it is booting. Add lines like this to the new file /mnt/root/etc/crypttab:
    cryptroot UUID=756125d1-7845-ab56-cd56-780a-4a45cb56d4cb none luks,discard,noearly
    crypthome UUID=aa54bc47-2356-554d-d5c6-41ac-4a45cb5456a8 none luks,discard,noearly

    Note that these are the UUIDs of /dev/sda5 etc., not the UUIDs of /dev/mapper/cryptroot etc

  4. Improve the security on this file for when we later embed passphrases into it:
    chmod -rw /mnt/root/etc/crypttab
  5. Set these values in /mnt/root/etc/default/grub:
    GRUB_LINUX_CMDLINE="cryptdevice=UUID=756125d1-7845-ab56-cd56-780a-4a45cb56d4cb:cryptroot"
    

    Again this is the UUID of the underlying root device such as /dev/sda5.

  6. Now re-configure the Grub menu entries:
    update-grub
    
  7. That’s it. You can reboot. Your OS should now boot. You’ll get prompted twice to enter the passwords for your encrypted partition, once for your root partition and once for your home (and more for any others you made). In the next section we’ll eliminate some of those. For now, give yourself a pat on the back for getting this far.

Fewer passphrases

All well and good so far, but having to unlock each partition separately is a bit of a pain in the bum. Time to rectify that. Each partition can have up to eight passphrases associated with it, and those passphrases can be files as well as typed-in passphrases. So next we’ll let the root partition auto-mount the /home partition without us having to type in the passphrase.

  1. Open a terminal on your new installation. Type:
    sudo bash
    dd if=/dev/urandom of=/etc/crypt.home count=1 bs=512
    cryptsetup luksAddKey /dev/sda6 /etc/crypt.home
    

    This puts 512 bytes of randomness into the file /etc/crypt.home, and stores it as a second password for the /home partition.

  2. Next we open the file /etc/crypttab. Against the home partition where we had none (which causes it to prompt you for a passphrase), we will now specify a keyfile instead:
     
    crypthome UUID=aa54bc47-2356-554d-d5c6-41ac-4a45cb5456a8 /etc/crypt.home luks,discard,noearly
    
  3. Reboot. Now you should only be prompted for the password for your root partition, and home should auto-mount.

Should something ever go wrong, the password you originally set for the home partition is still valid, so you can use that as a recovery password for getting in to it. Store it safely.

No passphrases

By now you’re thinking “Why don’t I do this for the root partition too?”. You’d have to store it somewhere else that wasn’t the (not-yet-unlocked) root partition. That couldn’t be the /boot partition, as that isn’t encrypted (the way we’ve done it), so you’d be storing the keys to the kingdom in the clear for your laptop thief to recover. Instead, here’s a nicer way – store it on a USB drive. If the drive is plugged in, use it to open the root partition automatically. If it isn’t prompt for a password. If you leave this drive plugged in to a USB hub at home, then whenever you are at home it will be there and unlock automatically, but if you take the laptop elsewhere the drive won’t be attached and the passphrase will be requested. Automatic security the moment you leave the house!

  1. Get a sacrificable USB drive. You will erase all its content, and probably never use it for anything else again. I would recommend one with a clearly visible light, and without any fancy retraction mechanisms. You don’t want there to be iffiness about whether it’s plugged in or not. Smaller is better, it really only needs to be a few Mb in size.
  2. plug in the drive and open a terminal again:
    sudo bash
    dmesg
    

    Look at the end of the output from dmesg to see what device name your drive has been given. If you only have one internal drive and this is the only stick plugged in it will probably be sdb, but it’s best to check.

  3. Destroy the contents of the stick by filling it with random data:
    dd if=/dev/urandom of=/dev/sdb bs=1
    

    The bigger the stick the longer this will take.

  4. Extract a key from the randomness written to the drive and add it as a passphrase for the root partition:
    dd if=/dev/sdb of=/etc/crypt.root bs=1 count=512
    cryptsetup luksAddKey /dev/sda5 /etc/crypt.root
    
  5. Note down the path to the USB drive:
    ls /dev/disk/by-id
    

    If it’s not obvious which one it is, you can pull the stick out and ls again to see which entry has disappeared. Copy this into your clipboard.

  6. Create this file as _/usr/local/sbin/unlockusbkey.sh_:
    !/bin/sh
    USBKEY=/dev/disk/by-id/usb-USB_FLASH_DISK-0\:0
    
    # flag tracking key-file availability
    OPENED=1
    
    # give the system time to settle and open the USB device
    sleep 2
    
    # check for the specific USB key
    if [ -b $USBKEY ]; then
    # if device exists then output the keyfile from the usb key
    dd if=$USBKEY bs=1 count=512 | cat
    OPENED=0
    fi
    
    if [ $OPENED -ne 0 ]; then
    echo "FAILED to get USB key file ..." >&2
    if [ -x /bin/plymouth ] && plymouth --ping; then
    plymouth ask-for-password --prompt "Enter passphrase"
    else
    /lib/cryptsetup/askpass "Enter passphrase"
    fi
    else
    echo "Success loading key file. Moving on." >&2
    fi
    
    sleep 1
    exit 0
    
  7. Modify the root partition entry of _/etc/crypttab_ to add the keyscript option on to the end:
    cryptroot UUID=756125d1-7845-ab56-cd56-780a-4a45cb56d4cb none luks,discard,noearly,keyscript=/usr/local/sbin/unlockusbkey.sh_
    
  8. Finally, rebuild initramfs:
    update-initramfs -u
    

All done. Now if you boot with the stick in the root partition should be unlocked automatically, and if you boot without you’ll be promptd for its password. Enjoy!

Advertisements

Puppet module test debug output

This one took hours to find out.

If you are using pdk to run your unit tests, all console output from your module gets squashed. Which is a bit rude, frankly. I could not find anyway of making it print stuff to the console so that I had half a hope of debugging my failing test.

Tha answer in the end was not to use

pdk test unit

But to run the tests with rspec directly. To do this (on Mint):


sudo apt-get install ruby-bundler
sudo bundle install

Then you can


bundle exec rake spec

from the directory which contains your spec folder. You will get different (more) output, and if you use Ruby’s puts you will be able to print to the console.

To keep track of when this gets fixed so that output is retained, follow this issue.

Land Rover Discovery 3 key case replacement

Bought a cheapie key case from eBay. Here’s how I replaced it.

  • Buy case and get the key part cut at a locksmith. You don’t need a specialist car locksmith.
    The new case with key part cut

    The new case with key part cut

  • Assembled and tested but the slightest touch of the key was activating the buttons, so the key was unlocking and locking like mad. Eventually I discovered the rubber studs in the key were activating the circuit board buttons. Glued a piece of card in to insulate it.
    Glued the button studs (super glue)

    Glued the button studs (super glue)

    Weighed down the cardboard with a penny

    Weighted down the cardboard with a penny

  • Still wasn’t working. Provided switch film didn’t have sufficient height to be work properly, so I used the old one.
    Original on right, useless replacement  on left

    Original on right, useless replacement on left

  • Re-assembled the correct way up 2015-05-29-13-55-00
  • Insert the transponder RFID blob in the corner 2015-05-29-13-55-24
  • Fit circuit board and battery2015-05-29-13-55-35

How to repair a brake pad wear sensor

I get pretty annoyed by repairable parts being binned. Brake pad wear sensors often get replaced when they could still work perfectly well, and at up to 50% of the price of the brake pads they’re really expensive to start with. So remove it and look at it, and work out how it works. In the case of this one off a Range Rover, for example:

A worn brake pad wear sensor with the wear link broken

A worn brake pad wear sensor with the wear link broken

It’s easy to see that one of the two wires coming into this sensor are really one loop, exposed at the “sensor” end. When this side of the sensor gets close enough to the brake disc, the metal is worn away, breaking the loop and preventing current from flowing. This is what the ECU senses as it will show a brake warning symbol on the dash. Clean it up and solder it together again:

The same wear sensor with a blob of solder reconnecting the two remaining stubs of metal.

The same wear sensor with a blob of solder reconnecting the two remaining stubs of metal.

Solder is very soft and will wear just as nicely as the metal that was there before, so the “sensor” will still work. Just refit.

Multiple dropboxes stops working; only one at a time

There are lots of posts out there about how to use multiple (fake) home directories to force Dropbox to run multiple times on Linux. This approach is great, but it breaks when Dropbox upgrades. If you have had this approach working previously and suddenly it stops working, with only one Dropbox running at a time, it’s likely because the Dropbox client has updated itself in the background. The dropbox client has two components, the one you installed, and then the brains that it downloaded during installation. When an update becomes available, the brains get downloaded again. That’s fine if you only have one Dropbox, but when you have two it makes a mess, because you cannot have two different versions of Dropbox running at the same time.

The good news is that you can fix this relatively easily, without re-downloading your files and without re-building your index.

The background

I had multiple dropboxes working fine for months on two different computers. One morning I switched them on and only had one Dropbox, and I knew I hadn’t applied any OS updates, so I suspected a Dropbox update. The brains that self-updates is contained in a folder called .dropbox-dist. You should just have one instance of this folder. To see if all is well, try this:

sudo updatedb && locate -r .dropbox-dist$

If you don’t have updatedb, you need the package mlocate installed. Hopefully the result of this will be one directory, /var/lib/dropbox/.dropbox-dist. On my machine there were two:

/var/lib/dropbox/.dropbox-dist
/home/user/.dropbox_work/.dropbox-dist

Digging in further, I had:

/var/lib/dropbox/.dropbox-dist/dropbox-lnx.x86_64-3.12.6
/home/user/.dropbox_work/.dropbox-dist/dropbox-lnx.x86_64-3.14.5

Uh-oh. Two different versions. To see what was happening, I manually ran the two dropbox instances:

HOME=$HOME/.dropbox_personal; dropbox start
HOME=$HOME/.dropbox_work; dropbox start

Whichever I ran second killed the first. I also ran ps -ef while each was running, and here’s what I got:

dropbox_personal: /var/lib/dropbox/.dropbox-dist/dropbox-lnx.x86_64-3.12.6/dropbox
dropbox_work: /home/user/.dropbox_work/.dropbox-dist/dropbox-lnx.x86_64-3.14.5/dropbox /newerversion

So not, in fact, two instances of the same program running in different home directories, but two different programs. Also playing a part here is how you installed dropbox; if you installed in Ubuntu or any of its derivatives using the package manager (the package you have installed is nautilus-dropbox) then you will find .dropbox-dist in /var/lib/dropbox. If you installed it by downloading the deb ifrom the Dropbox website (the package you have installed is dropbox) then it will be under a home directory.

How to fix it

Debian and its derivatives

If you have Debian’s nautilus-dropbox installed, run these one at a time:

killall dropbox
sudo rm -rf /var/lib/dropbox/.dropbox-dist
rm -rf /home/user/.dropbox_one/.dropbox-dist
rm -rf /home/user/.dropbox_two/.dropbox-dist
HOME=/home/user/.dropbox_one; dropbox start -i
HOME=/home/user/.dropbox_two; dropbox start
ps -ef | grep dropbox

Others

If you have dropbox installed, run these one at a time:

killall dropbox
rm -rf /home/user/.dropbox_one/.dropbox-dist
rm -rf /home/user/.dropbox_two/.dropbox-dist
HOME=/home/user/.dropbox_one; dropbox start -i
HOME=/home/user/.dropbox_two; dropbox start -i
ps -ef | grep dropbox

Check it

Now you should have two instances of the same program, with the later version:

user 8139 1 2 11:42 ? 00:00:38 /var/lib/dropbox/.dropbox-dist/dropbox-lnx.x86_64-3.14.5/dropbox
user 8140 1 1 11:42 ? 00:00:28 /var/lib/dropbox/.dropbox-dist/dropbox-lnx.x86_64-3.14.5/dropbox

Huzzah! All is well, and you can revert to launching your Dropboxes however you were before.

librealflash OpenSCAD library

Just in case someone who has downloaded one of my OpenSCAD designs gets confused as to why OpenSCAD is complaining about a missing file librealflashX.scad, it’s because I have abstracted a bunch of modules I found useful into this file, so most of my designs depend upon a certain version of it.

The correct version should be uploaded with the thing, but in case I haven’t done that, you can find all the versions in this GitHub repository.