Twitter not sending 304s

The twitter JSON API that I’ve been using for my status widget has a caching problem, which has caused it to be broken in Opera for a while now. Opera is quite aggressive in re-using its cache (which IMHO is a good thing). However, bad things happen when webservices deviate from the HTTP cache validation model. Twitter is recognising that the browser should be hitting its cache, but its response is broken.

Here’s how it goes:

  1. Load up the JSON document for the first time with an empty cache.
  2. Twitter sends a Last-Modified header and the expected JSON document.
  3. Refresh the document (in Opera, hit enter in the address bar as opposed to clicking the Reload button, since the latter forces a cache refresh). Opera sends an If-Modified-Since header.
  4. Twitter (presumably) recognises that the last status update was not after the browser’s cache timestamp. It sends a degenerate response entity: “[]“; an empty array in javascript, with a 200 OK status.

To test this from a shell:

url='http://twitter.com/statuses/user_timeline/p00ya.json?count=1&callback=f'
lm="$(wget --debug $url 2>&1 \
 |grep '^Last-Modified:' \
 |sed -e 's/Last-Modified/If-Modified-Since/' \
 |tr -d '\n\r')"
[ x != "x$lm" ] && \
  wget -nv --save-headers -O - --header="$lm" $url

The brokeness comes from the half-baked response. A 200 OK status code would be fine if the full JSON object was written out. A 304 status code with any kind of entity would be fine too. The empty array might even prevent breakage in user-agents that don’t handle 304s (but do send If-Modified-Since? wtf?). Sending a 200 response overwrites the correct cached entity, replacing it with the degenerate response.

Firefox seems to be unaffected since it doesn’t cache the document at all and so doesn’t send the If-Modified-Since header.

One workaround is to use something like jQuery’s cache breaking capability (where it adds some random tokens to the URL each time). I refuse. Just remember the widget breakage wasn’t my fault!

A separate issue is that it seemed to break Opera quite badly. Perhaps it was because I’m using jQuery’s ready event, but Opera hangs as if the XHR was synchronous. The document wouldn’t receive any events (no mouse-wheel scrolling). I’ve got no idea why though; my callback functions are robust enough to handle being passed the empty array twitter calls them with, and I wasn’t getting any exceptions.

miniupnpd stale EXTIP

Miniupnpd comes with the script iptables_init.sh, which among other things adds an iptables rule:

#adding the rule to MINIUPNPD
$IPTABLES -t nat -A PREROUTING -d $EXTIP -i $EXTIF -j MINIUPNPD

Unfortunately, this rule isn’t replaced when the connection is dropped (and the router gets a new dynamic IP). OpenWRT does have a hotplug script:

[ "$ACTION" = "ifup" -a "$INTERFACE" = "wan" ] && \
/etc/init.d/miniupnpd enabled && \
/etc/init.d/miniupnpd restart

Problem here is that /etc/init.d/miniupnpd enabled returns false! It’s a function defined in /etc/rc.common that checks for /etc/rc.d/S95miniupnpd. So the solution is:

ln -s /etc/init.d/miniupnpd ./S95miniupnpd

Realistically, the ipkg should come with this file (effectively enabling miniupnpd by default). A more robust solution would be to modify the hotplug script so that it restarts the daemon without the enabled check, and rely on the restart function not to start the daemon if it wasn’t already running.

PHP array references

Hit this one at work, where we use RHEL4’s PHP 4.3.9.

The online documentation doesn’t mention it any more, but there’s a history of PHP not dealing well with array references.

PHP segfaults. Unfortunately it’s quite hard to reproduce artificially; we don’t see the segfault until way after the reference was taken (presumably after the GC has done its rounds). It’s 100% reproducible in our program, but I’m still working on a minimal test case.