Monitoring Intel SSD lifetime with S.M.A.R.T.

The Internet is abuzz with talk about solid state reliability right now (see a recent article by Jeff Atwood). Random, catastrophic failures aside, how can you know how much life you’ve eaten into your SSD?

If you’ve an Intel SSD, it’s pretty easy; they export a S.M.A.R.T. attribute “Media Wearout Indicator”. Starting at 100 (new), the attribute decreases to, well, zero. Forget how to do that on Linux? It’s easy:

$ sudo smartctl -a /dev/sda | grep Media_Wearout_Indicator
233 Media_Wearout_Indicator 0x0032   098   098   000    Old_age   Always       -       0

The SSD in my laptop is at 98, and my oldest SSD in another system (from mid-2009) is at 97. Yours?

On to the real news: the OpenStreetMap project has switched their tile rendering server to an SSD (hopefully making tile renders much, much faster). A newer, consumer-grade MLC-based Intel 320 Series 600 GB SSD, in fact. Conveniently, OpenStreetMap monitors their servers with Munin, which by default graphs all S.M.A.R.T. attributes, including Media Wearout Indicator.

Other than the initial import of the tile rendering database, OSM tile rendering does not consume many write cycles. But it definitely hammers the disk to death on reads. Keep a lookout on these graphs to see how their SSD ages over time. Don’t forget to contribute to OpenStreetMap yourself so we can see that number go down a bit quicker (I’m pretty sure OSM doesn’t mind!).

Note: Toby mentioned to me that all the values appear to be pegged at 100. Most of these attributes are dummy values—only “Media Wearout Indicator” and “Available Reserved Space” appear to change with normal use.

What does the "BY" in Creative Commons' license names mean?

Have you ever wondered what the “BY” in Creative Commons’ licenses (e.g. CC-BY, CC-BY-SA, and CC-BY-SA NC) stands for?

The other bits in the license name are obvious, for the most part:

  • CC – Creative Commons
  • NC – Non-Commercial
  • ND – No Derivatives
  • SA – Share-Alike

BY” is confusing because it’s not an acronym, shortening of a word, or anything otherwise obvious. But buried in the Creative Commons FAQ, it’s mentioned it stands for attribution.

If all Creative Commons require attribution (except CC0), why is it included in the license names (especially the abbreviations) at all?

Increase file descriptors for Transmission on Linux

Have you run out of file descriptors for Transmission? The torrent will be stopped (for no apparent reason), and when you examine it, you’ll see an error similar to:

$ transmission-remote -t 1 -i | grep -i 'open files'
Unable to save resume file. Too many open files.

Time to increase the number of file descriptors available. This article is tailored towards Debian and Ubuntu.

It’s unlikely you’ll need to raise your system’s global limit. Check with:

$ cat /proc/sys/fs/file-max

The OS needs a couple thousand file descriptors for itself. Make sure to make space for them with whatever numbers to choose below. In my case, I’ve more than enough.

If you still need to raise your system’s limit, you can easily; to set it to a million (which will be remembered across reboots):

sudo sh -c "echo fs.file-max=$(dc -e '2 20 ^ p') > /etc/sysctl.d/file-descriptors-max.conf
sudo service procps restart

While you may not need to change your system’s global limit, you probably will need to change the limit for your users. Check that limit with:

$ ulimit -Sn
$ ulimit -Hn

If you’re working with hundreds of torrents (each with dozens to hundreds of files) with Transmission, this isn’t enough. To let a user have a few thousand (in the below example, 16,384, with 128 more for the hard limit), create a new file /etc/security/limits.d/debian-transmission.conf:

sudo sh -c "echo debian-transmission soft nofile $(dc -e '2 14 ^ p')" > /etc/security/limits.d/debian-transmission.conf
sudo sh -c "echo debian-transmission hard nofile  $(dc -e '2 14 ^ 2 7 ^ + p')" >> /etc/security/limits.d/debian-transmission.conf

Replace “debian-transmission” with the user that is running Transmission.

For the changes to go into effect, you need to logout completely (e.g. close multiplexed SSH connections, etc), and log back in again. Or to be sure, just reboot to make sure changes kick in. You’ll see you have many more file descriptors available:

$ ulimit -Sn
$ ulimit -Hn

Now, we need to configure Transmission to use this many. In /etc/transmission-daemon/settings.json, find the open-file-limit option and set to a larger number (e.g. 16000 or so). When done, restart transmission-daemon:

sudo service transmission-daemon restart

If you’re not running Transmission as a system user, edit the right settings.json and restart the daemon appropriately.

That’s it. Have fun!

Changes that caught my eye in Python 3.2

Python 3.2 was released a few days ago. Reading the What’s New document, some of the stuff that caught my eye…

argparse is a new command-line option parsing module that replaces optparse. I found optparse, now deprecated, a pain to use—argparse looks a lot better. argparse can be used in older programs today with a third-party module (E.g. argparse Debian package). I actively avoided adding command-line parsing to my applications because optparse was such a pain—something now much less of a problem.

The new concurrent.futures module seems to be a port of Java’s java.util.concurrent package. I haven’t looked into it yet, but with the GIL, how many people really care about Python threads? The simple examples given can easily be done with Python’s multiprocessing module, for both processes and threads, though without the Java paradigm. The threading module also has a new threading.Barrier class.

WSGI for Python 3 appears to be finalized. I’m looking forward to finally porting my WSGI web applications to Python 3. The email module also received the same makeover that WSGI needed for Python 3.

ElementTree has been updated to v1.3. Not that I care—lxml’s namespace handling is much better (which is to say, it’s just terrible as opposed to near unusable, as is with pure ElementTree).

Quite possibly the most badass addition to Python 3.2: the functools.lru_cache decorator. I use this pattern all the time, but have never gotten around to making it a generic class or decorator I could reuse across applications. With Python 3.2, I no longer need to. The idea: decorate a slow-performing function (where it makes sense) with functools.lru_cache, and Python will instantly and easily memoize the return values of that function making subsequent calls to the function faster.

html.escape() is a simple function that escapes HTML markup for you with the appropriate HTML/XML entities (I forget how I used to do this usually; something in one of the CGI utils, I think? Or maybe stuff in my template engine…).

This is just from a quick reading of the release notes—it’s very likely I missed something. What’d I miss? And what do you like that’s new in Python 3.2?


Subscribe to Samat Says RSS