S/MIME gotchas

S/MIME is an attractive option for implementing secure messages, since it works OOTB in more MUAs than OpenPGP. However, for an 11-year-old standard, there are a lot of limitations on the lowest-common-denominator of what works.

A good (and fairly ubiquitous) implementation seemed to be the OpenSSL smime(1) command. This has a few quirks:

  • It doesn’t use CRLF line endings in the outer message as required by RFC 2822. In newer versions there is an undocumented -crlfeol option, but even this doesn’t work properly. I’ve submitted a patch on their tracker.
  • You probably want to use the -text option and not the -binary option when encrypting to fix the CR/LF issues for a text/plain payload.
  • To generate certificates non-interactively, you are probably going to need an OpenSSL config file to go along with the req -batch option. Something like:
    [ req ]
    prompt = no
    default_bits = 2048
    extendedKeyUsage=emailProtection
    subjectAltName=email:foo@example.com
    distinguished_name = dn
    req_extensions = req_extensions
    x509_extensions = req_extensions
    
    [ req_extensions ]
    extendedKeyUsage=emailProtection
    basicConstraints=CA:FALSE
    
    [ dn ]
    O=Example Corp
    C=AU
    CN=Example Corp Secured Email <foo@example.com>
    emailAddress=foo@example.com
    

Below is the micro-HOWTO for non-interactively doing stuff (you can use pipes instead of the files in all these instances when driving OpenSSL via IPC). Note that the certificates are read before the message payload, and you want to get the I/O order right to avoid a deadlock.

openssl req -x509 -nodes -days 14 -newkey rsa:1024 \
-keyout private.pem -out public.pem
openssl pkcs12 -export -passout fd:0 -inkey private.pem -in public.pem \
-out private.pfx
openssl smime -encrypt -des3 -text -in payload.txt \
-to 'foo@example.com' -from 'Me <bar@example.net>' \
-subject "An encrypted email" \
-out message.eml public.pem # or pipe stdout to sendmail
openssl smime -decrypt -in message.eml -out payload2.txt \
-inkey private.pem public.pem
# for low-level debugging:
openssl smime -pk7out -in message.eml | openssl asn1parse

On to the MUAs. The certificate import for Apple Mail is pretty painless with the PKCS#12 key being imported by Keychain Access. Later (it didn’t work immediately for me) extra buttons for signing and encrypting automagically appear in the message editing window on the From line, but they are disabled unless your From address has a certificate. Hence you need a certificate yourself, even if you’re not signing, in order to send an encrypted email to a friend.

Apple Mail gets very confused if the CR/LF stuff isn’t correct: it will display the same message as blank, just the base64-encoded S/MIME part, and the correct decoded text on different occasions.

Thunderbird isn’t very friendly toward self-signed certificates. Choosing the .pfx from Preferences > Advanced > View Certificates > Import will get the key to show up, but when you try to encrypt an email, you get an error message like “the application failed to find an encryption certificate for <foo@example.com>”. However, Thunderbird will use the key for decrypting.

Microsoft Outlook and Outlook Express will use the Windows certificate store, which imports a .pfx with a wizard by default when you open one. You can import it from a shell (or more likely, from wine) with rundll32.exe cryptext.dll,CryptExtAddPFX private.pfx, or use the UI from Internet Explorer (Tools > Internet Options > Contents > Certificates).

Apple Mail and Thunderbird both handle multipart/mixed with text/plain and smime-envelope parts correctly (by concatenating the unencrypted and envelope contents in the message window), although they will report the message as unencrypted (which is true I guess, because only one part of the message is). However, Microsoft Outlook won’t display anything for the secure part (not even an error).

Decent terminal font

I haven’t been able to find a good replacement for Monaco for use as a terminal font on OS X.

try figuring this one out with bleary eyes
try figuring this one out with bleary eyes

My personal favourite terminal font, neep alt isn’t readily useable from OS X.

None of the Proggy fonts are as clean as Monaco, although Proggy Tiny 11 comes close.

  • lower-case “a” should be double-storey, so it doesn’t look anything like lower-case “o”
  • zero “0” should be slashed
  • lower-case “l” should have a serif on the bottom like lower-case “t”, and not look like numeral “1” or upper-case “I”.
  • upper-case “U” and “V” differ by at least 7 pixels
  • gentle curves rather than blocky rectangles, please

It’s disappointing that Monaco can get so much right and get lower-case “a” wrong. Still, if anyone wants to port neep alt to a format OS X and iTerm can handle (even if it’s only at 13pt semicondensed)… I tried once with fontforge but it was pretty broken.

Twitter not sending 304s

The twitter JSON API that I’ve been using for my status widget has a caching problem, which has caused it to be broken in Opera for a while now. Opera is quite aggressive in re-using its cache (which IMHO is a good thing). However, bad things happen when webservices deviate from the HTTP cache validation model. Twitter is recognising that the browser should be hitting its cache, but its response is broken.

Here’s how it goes:

  1. Load up the JSON document for the first time with an empty cache.
  2. Twitter sends a Last-Modified header and the expected JSON document.
  3. Refresh the document (in Opera, hit enter in the address bar as opposed to clicking the Reload button, since the latter forces a cache refresh). Opera sends an If-Modified-Since header.
  4. Twitter (presumably) recognises that the last status update was not after the browser’s cache timestamp. It sends a degenerate response entity: “[]“; an empty array in javascript, with a 200 OK status.

To test this from a shell:

url='http://twitter.com/statuses/user_timeline/p00ya.json?count=1&callback=f'
lm="$(wget --debug $url 2>&1 \
 |grep '^Last-Modified:' \
 |sed -e 's/Last-Modified/If-Modified-Since/' \
 |tr -d '\n\r')"
[ x != "x$lm" ] && \
  wget -nv --save-headers -O - --header="$lm" $url

The brokeness comes from the half-baked response. A 200 OK status code would be fine if the full JSON object was written out. A 304 status code with any kind of entity would be fine too. The empty array might even prevent breakage in user-agents that don’t handle 304s (but do send If-Modified-Since? wtf?). Sending a 200 response overwrites the correct cached entity, replacing it with the degenerate response.

Firefox seems to be unaffected since it doesn’t cache the document at all and so doesn’t send the If-Modified-Since header.

One workaround is to use something like jQuery’s cache breaking capability (where it adds some random tokens to the URL each time). I refuse. Just remember the widget breakage wasn’t my fault!

A separate issue is that it seemed to break Opera quite badly. Perhaps it was because I’m using jQuery’s ready event, but Opera hangs as if the XHR was synchronous. The document wouldn’t receive any events (no mouse-wheel scrolling). I’ve got no idea why though; my callback functions are robust enough to handle being passed the empty array twitter calls them with, and I wasn’t getting any exceptions.