Making ssh-add -d work

Periodically, I end up with my SSH agent tracking an absurd number of keys, many of them no longer relevant, and wish to remove them. In my experience, ssh-add -d /path/to/keyfile rarely works. Today, with a bit of help from Erik, I figured out why I get annoying stuff like this:

$ ssh-add -d /Users/matthew.wagner/.ssh/cpanel-east.pem
Bad key file /Users/matthew.wagner/.ssh/cpanel-east.pem: No such file or directory

That key file exists, is loaded in ssh-agent, and has the correct permissions. The issue is that -d expects a public key, even though ssh-add -l doesn’t list those.

If you have a .pub version of the key, ssh-add -d will automatically find it and remove it. In some cases, I don’t have the public key, such as keys downloaded from AWS. The good news is, it’s easy to generate them:

$ ssh-keygen -y -f ~/.ssh/cpanel-east.pem > ~/.ssh/

Removal is then a breeze: $ ssh-add -d /Users/matthew.wagner/.ssh/cpanel-east.pem
Identity removed: /Users/matthew.wagner/.ssh/cpanel-east.pem (/Users/matthew.wagner/.ssh/

tl;dr – If ssh-add -d says it can’t remove a key that plainly exists, you need to generate a public key for it.

macOS / OS X disk compression

While reading about AppNap and the fun tricks Apple employs to improve battery life, I happened across something I never knew: the HFS+ filesystem supports transparent compression. It seems like Apple’s intention was for this to be used for shipping system files, and not applied to user files, as evidenced by the fact that it’s virtually impossible to even figure out if it’s enabled, much less a checkbox to simply turn it on for the filesystem.

But, enter afsctool. (Also available through Homebrew.) Wanting to try it out, I for some reason decided to try to compress the 1.5GB git repo I spend most of my time in at work. Let’s look at compression stats (-v, indirectly) for PHP files (-t php) and then the whole directory:

$ afsctool -vt php .

File content type: public.php-script
File extension(s): php
Number of HFS+ compressed files: 4895
Total number of files: 4904
File(s) size (uncompressed; reported size by Mac OS 10.6+ Finder): 22705078 bytes / 34.8 MB (megabytes) / 33.2 MiB (mebibytes)
File(s) size (compressed - decmpfs xattr; reported size by Mac OS 10.0-10.5 Finder): 1616066 bytes / 2.1 MB (megabytes) / 2 MiB (mebibytes)
File(s) size (compressed): 5948389 bytes / 6.4 MB (megabytes) / 6.1 MiB (mebibytes)
Compression savings: 73.8%
Approximate total file(s) size (files + file overhead): 9050906 bytes / 9.1 MB (megabytes) / 8.6 MiB (mebibytes)

Number of HFS+ compressed files: 17951
Total number of files: 21797
Total number of folders: 3706
Total number of items (number of files + number of folders): 25503
Folder size (uncompressed; reported size by Mac OS 10.6+ Finder): 1697530893 bytes / 1.75 GB (gigabytes) / 1.63 GiB (gibibytes)
Folder size (compressed - decmpfs xattr; reported size by Mac OS 10.0-10.5 Finder): 1615203435 bytes / 1.62 GB (gigabytes) / 1.51 GiB (gibibytes)
Folder size (compressed): 1628829083 bytes / 1.64 GB (gigabytes) / 1.52 GiB (gibibytes)
Compression savings: 4.0%
Approximate total folder size (files + file overhead + folder overhead): 1648262029 bytes / 1.65 GB (gigabytes) / 1.54 GiB (gibibytes)

The results really shouldn’t be surprising. The .git/ files are already stored compressed, so there’s not much to be gained there, hence an overall savings of only 4% in the repo. PHP files averaged a 73.8% reduction in size thanks to compression… Saving me approximately 30MB on a 512GB disk. Hardly worthwhile, and I have to imagine this is going to come bite me down the road. (Why would I even think that compressing stuff I use hundreds of times a day was a good idea?!) afsctool -d will de-compress a directory, though, so, assuming you don’t corrupt anything, it’s easy enough to roll back things that you compress for no good reason. (You could also use something like ascftool -c -s10 dirname/ to skip files unless they could be compressed more than 10%, to avoid doing the nonsense I did and “compressing” already-compressed files.)

As I referenced in the beginning, I see no way to enable this for newly created files. You can compress the contents of an existing directory, but new stuff written there won’t benefit. It’s possible I haven’t found it yet, but it really feels like compression was intended for Apple to use at install-time.

With that in mind, here was a perhaps-more-reasonable thing to do:

# afsctool -s 10 -vc /Applications/Adobe\ Photoshop\ CC\ 2015
/Applications/Adobe Photoshop CC 2015:
Number of HFS+ compressed files: 7286
Total number of files: 8218
Total number of folders: 2499
Total number of items (number of files + number of folders): 10717
Folder size (uncompressed; reported size by Mac OS 10.6+ Finder): 1887395578 bytes / 1.91 GB (gigabytes) / 1.78 GiB (gibibytes)
Folder size (compressed - decmpfs xattr; reported size by Mac OS 10.0-10.5 Finder): 1030642518 bytes / 1.04 GB (gigabytes) / 991.6 MiB (mebibytes)
Folder size (compressed): 1035466738 bytes / 1.04 GB (gigabytes) / 996.2 MiB (mebibytes)
Compression savings: 45.1%
Approximate total folder size (files + file overhead + folder overhead): 1049476316 bytes / 1.05 GB (gigabytes) / 1000.9 MiB (mebibytes)

I shrunk Photoshop by 45%. Note that some things, like Apple’s iMovie, are already compressed. (Also: why is iMovie on my work laptop?!) It seems like most non-Apple applications are uncompressed, and on average drop somewhere close to 50% in size.

Of course, it’s very clear that, armed with this new hammer, every directory is a nail. Compressing rarely-used applications like Photoshop is probably reasonable; attempting to compress my working git repo was clearly not. Still, a neat tool for your toolbox.

5 indispensable bash tricks

Don’t mind the lame Buzzfeed title… Here are a few handy bash tricks and tips that people either use every day or never knew existed. Hopefully I can help move some of you into the first camp!

Introductory notes

A few of these commands involve working with bash history. On the advice of a coworker, I dropped this in my .bashrc to keep tons of history:

HISTSIZE=100000 # keep 100k commands in a session history (memory)
HISTFILESIZE=200000 # store 200k commands in my history file (on disk)

Disk space is cheap, as is memory. The number of times (prior to this change) that I wanted a command that had aged out of my bash history is much greater than the number of times I’ve found bash cumbersome because my history file is almost 1MB in size (when I have a 500GB SSD and 16GB RAM in my 2-year-old laptop).

Meta key

A number of bash commands reference a Meta key. In general, on a Mac, the Escape key fills that roll. On Linux, it’s generally the Alt key. You can change that, but if you’ve done so, you don’t need me to tell you about it. My examples will use Esc for these commands, but if you’re on a Linux box, you will likely want to substitute Alt for it.

Esc-. | Insert last argument

Described in the docs as insert-last-argument (M-., M-_), this keyboard shortcut will spit out the last argument to the previous command. On a Mac, the Meta key is Escape; on Linux, it’s often Alt.

Example usage:

$ mkdir -p long/directory/name/that/would_suck_to/type
$ cd Esc .

The Esc + . will be expanded into long/directory/name/that/would_suck_to/type.

Note that Esc + _ is bound to the same function, but is a bit tougher to type.

Ctrl+R | Reverse history search

This one is tough to explain, but magical. Have you ever hit the up arrow a bunch of times to scroll through history, trying to find something you ran recently? Ctrl + r will open up an interactive search, or reverse-i-search in bash parlance.

Recently used vim on a file with a long filename? Press Ctrl + r and start typing vim. The most recent command matching vim will be showed. Keep typing to make your search more specific, or press Ctrl + r again to scroll to the next-newest one. When you find what you want, press Enter to run it, or the right arrow to start moving the cursor through the command. (Or something like Ctrl+E to jump to the end of the line.)

If you want to be really nutty, you can start commenting your commands at the end. vim /etc/X11/xorg.conf # fix video settings will allow Ctrl + r + video to match a search, for example. I’ve been known to throw in random keywords I think I might try looking for later on.

cd – | Return to previous directory

pushd and popd are awesome and you should use them. But sometimes you forget. bash has got your back. cd - will return you to the previous directory you were in. (This is stored in the OLDPWD environment variable.)

git checkout – | Switch back to the previous branch

If you use git, you’ll be delighted to know that it does something similar. git checkout - will check out the previous branch you were on. I’m often bad at cleaning up topic branches, and will git checkout master to do some catching up, and then realize I don’t remember what my topic branch was called. Sure, it would probably take me all of 30 seconds to figure it out, but checking out - is so much easier.

!! | Re-run the previous command

!! will re-run the command you just ran. Why not just hit the up arrow? Because !! can be combined. The most command usage:

$ cat /root/whatever
Permission denied
$ sudo !!
sudo cat /root/whatever

Hope you learned something useful! What other neat tricks should I know about?

Using msmtp with Rackspace Email

When using mutt on my Mac, I use msmtp to continue the alliteration as an SMTP agent, to send mail through an actual authenticated SMTP server versus trying to connect from my laptop, which not many mailservers will accept.

I’m either missing something, or it’s a real pain with keys when using TLS, especially on the Mac, where the CA certs aren’t present except in the Keychain. I found some guides to getting this working with Gmail, but not Rackspace’s email service.

This is the .msmtprc file I ended up using:

account default
port 587
tls on
tls_starttls on
tls_fingerprint CD:E1:CD:60:FC:8C:8F:3B:6F:17:62:70:61:51:75:3D
auth on
user ""
password "maybe you do not want it here"

Don’t trust me on the tls_fingerprint line. (I’m not up to anything, but you don’t know that.)

This page documents their SMTP settings, including the hostname. It doesn’t give you TLS fingerprints or a CA cert file, because no one on the Internet does that.

Following this advice concerning Gmail, I adapted it to find the fingerprint for Rackspace:

echo -n | openssl s_client -connect \
 -starttls smtp -showcerts > x.tmp

That will save the exchange, which includes the key. You could probably extract it from there, but it was easier for me to go on and just get the fingerprint:

openssl x509 -noout -fingerprint -md5 -in x.tmp

Take the bit after MD5 Fingerprint= and drop that into .msmtprc on the tls_fingerprint line.

There’s got to be an easier way…