My new year’s resolution for 2008 was to get my sleeping pattern into check. To help me do this I recorded data about my sleep on a daily basis throughout the year. Although it was clearly impossible to get precise measurements of things such as the time I fell asleep, I managed to make estimates for every single day of the year. As with a graph of my taste in music, graphs about my sleep tell a surprising amount about my life.

As before, these large graphs show seven day rolling averages of the variables, each of which should be self-explanatory. The smaller graphs concentrate on one variable each. The blue line represents the daily change. The red line is the seven day rolling average, while the grey straight line is a trendline.

The final quarter of the year didn’t start very well. After a relatively well-behaved summer, I began to slip into nocturnal habits at the beginning of October. At first this happened by accident, but I also encouraged it because at the time the following two Formula 1 races were “flyaways”, races in Asia that happen in the middle of night UK time.

What I didn’t think of, though, was that while it was perhaps, just about, feasible to stay up until about 6am to watch the Japanese Grand Prix, it was totally stark raving bonkers to stay up until about 9am watching the Chinese Grand Prix. Which is why the graph drops more quickly than it rises, as I changed my tactics for approaching the grands prix at the last minute.

In mid-November, my work situation changed. Instead of working primarily at nights, which I had done more or less since I started working at Woolworths, I was now working during the day (I don’t know why — must be the excellent customer service I provide!). This completely altered the path that my sleeping patterns took, and also led me to view the project in a new light.

The week I woke up the earliest all year was the seven day period leading up to 1 December, when I got up at an average clock time of approximately 09:13. At the same time, my “insomnia”, which was one of my primary concerns through the year, fell through the floor, lasting an average of just 21 minutes per night by the end of the year. The average “insomnia” for the whole year was 1 hour and 30 minutes per night.

All the while, the amount of time I was sleeping didn’t change too much, despite the earlier-than-normal starts. It fell to an average 7 hours and 1 minute per night on 18 December. That is pretty low, but it that was a one off (it normally hovered around 7½ hours) and it had been as low as 5 hours and 59 minutes in March.

Indeed, the fact that the amount of time I slept per night was pretty much stable at the recommended 8 hours per night was the pleasant surprise of the project. Maybe I had nothing to worry about all along.

All-in-all, my taste of what life in a 9–5 routine might be like left me with optimism about how I would cope. I adapted to regular early starts without a great deal of bother. I slept a bit less, but not uncomfortably so. And I was actually saving a lot of time by falling asleep much more quickly when I got to bed.

I came to realise that perhaps all of the sleep “problems” I had were actually down to a lack of routine rather than any genuine trouble with sleep. Of all the experiments I tried to help me sleep better, having a reason to get up early regularly was by far the best. And there’s no use in setting the alarm early if there’s no reason to get up, because I’ll only switch the alarm off and sleep in until at least the mid-morning. And why not? Maybe my new year’s resolution for 2009 should be to relax more about everything.

You might be asking, “what’s with that mad spike on most of the graphs on 31 December?” Well, I was hit by a dodgy winter-related disease. I went to bed at about 16:30 and more or less slept right through until 08:30 the following morning. Apart from making me feel rotten, the illness caused a right mess of my graphs! Mind you, the finishing point of the seven day averages looks fairly normal because I had been so indulgent during that Christmas week.

Overall, I am pleased with how the year-long experiment has gone. The trendlines for almost every variable I measured went in the right direction as the year progressed. The exception was ‘lazy’, to my shame. The experience of the final few weeks of the year have assured me that I probably don’t actually have much of a sleeping problem at all.

But I found the whole thing fascinating, and I already kind of miss logging every detail of my sleeping habits. I referred to the graphs often just for interest. But I have decided that I had better stop graphing elements of my life in minute detail, before it jeopardises any relationships.

All of the remaining sleep graphs are included below the fold.

My new year’s resolution for 2009 is to read more books. I have quite a daunting pile (actually, it’s a shelf) of books that have gone unread, some for several years. I will make my aim to reduce the existing pile to zero by the end of the year. I think my primary strategy will be to get more reading done in bed before I go to sleep. After all, now that I’m no longer worried about my sleep, it’s probably the easiest place to cram it in!

Click for more »

Around a year ago I wrote a post that analysed the year’s activity on Last.fm. For those who don’t know, Last.fm is a website that tracks your music listening habits. It produces lovely graphs and churns out recommendations as well as providing tailored radio stations for you to listen to at your leisure. I adore the site.

This year, instead of looking just at the past year’s statistics, I have decided to look at my entire Last.fm history. A couple of weeks ago, I hit my 100,000th scrobble (instance of listening to a track). It’s a suitably big landmark.

My 100,000th scrobble happened on 7 December, just over four years after my first scrobble on 18 November 2004. By that time I had listened to 730 different artists. Of these, 18 had 1,000 plays or more. 196 artists had over 100 plays.

My top thirty artists chart looked like this:

And my top thirty tracks were:

There are a few problems with this chart. The top track, ‘untitled (live)’ by Boards of Canada, is actually several different tracks from bootlegged gig recordings. I am quite sure that ‘Xmd 5a’ by AFX should not be that high, as one day I logged in and it showed many more plays than there should have been. It’s still a good track though.

John Cage tracks figure highly because I own three different recordings of Sonatas and Interludes for Prepared Piano. In fact, many of these tracks are high up simply because I own multiple versions, normally because I have the single as well as the album. Shining’s ‘To Be Proud of Crystal Colors is to Live Again’ is actually two different tracks with the same title. All of the Autechre tracks and most of the Jaga Jazzist tracks are here purely on their own steam.

It is obvious that, interesting though they are, Last.fm statistics are far from scientifically rigorous. For one thing, one track counts as one scrobble whether it’s 31 seconds long or 31 minutes long. One website, Last.fm Normaliser, attempts to get round this by weighting your artists by the average length of their tracks. This is also completely unscientific, but it is interesting nonetheless.

This table makes my obsession with Autechre even clearer. Even in the normal Last.fm table they have a huge lead. But by this measure I like Autechre twice as much as any other artist. The biggest climber in the top twenty is Steve Reich, who is number 20 in this table, but number 37 in my original Last.fm chart.

My favourite thing to do with Last.fm data, though, is to analyse it using LastGraph. I did this last year, looking at my Last.fm activity throughout 2007. This time, I am looking at my Last.fm activity as far back as the data goes — March 2005, just a few months after my first scrobble.

The graph is so huge that I can’t include a readable version on this page, but a miniaturised version appears below. Click on it to view it at its original size (Warning: It’s a large file).

I love looking at these graphs. They tell a story about my developing taste in music. But they also, in a way, tell a story about what is happening in my life at a certain point. I can glance at the graph and remember that I had exams during a certain period, or I was working lots in that summer, or whatever. It takes me back. I’m also quite surprised sometimes at which artists appear where on the graph. It appears that my memory was a bit out in a few places.

So there we have it. 100,000 scrobbles; four years of tracking my music listening habits.

I’m not sure how this post will be taken. The first update got 20 comments, the second one got zero. It does seem a bit self-indulgent. But I promised to keep you updated on the experiment so here it is.

If you missed the original posts, basically I made it my new years resolution to get my sleeping patterns in order. A big part of that is logging all sorts of information to do with my sleep so that I can see how things are progressing. I have already posted updates at the 3 month and 6 month marks. Today marks the end of the 9th month, and I am three quarters of the way through the experiment.

As in the previous posts, these graphs all depict 7-day rolling averages. Hopefully the labels are self-explanatory. As you can see, my sleep still has these annoying cycles which I’m trying to shake off at the moment. I am still experimenting a bit, seeing what I can do to make the situation better.

At the beginning of August I decided I would go to bed quite early regardless of how tired I felt, no matter whether I thought I would get to sleep or not. The result was that I was actually getting to sleep later than I had been (as the blue line heads down, the red line heads up). By the end of the week I felt awful and I knew that I had wasted a lot of time just lying awake in bed. I think it is probably better to tire yourself out before you retire to bed.

Looking towards the end of the graph, it also seems as though the lines are beginning to flat-line rather than go in these peaks and troughs that I am trying to get rid of. There is a slow but definite drift upwards though due to a lack of commitments giving me little incentive to get up early.

Just now I am going to bed later than I have for a number of months and I am not getting up until after noon. So in the short term I’ll be working to bring those times back earlier again.

In this graph you can quite clearly see the point where I tried going to bed early — the highest peak in the ‘insomnia’ graph all year. You can see the knock-on effect that had as a whole by the red and green peaks as well. All-in-all, I was spending almost 12 hours per day in bed. That’s not on as far as I’m concerned, and it’s good to see that the whole thing has calmed down a lot over the past couple of months, with a prolonged period where I was only spending 9 hours per day in bed. I’m aiming to keep this below 10 hours in the long run.

One annoying feature of my sleeping patterns that has crept in over the past few months is waking up early for no apparent reason. The graph to the right demonstrates this.

The blue line shows the difference between the time I set my alarm for and the time I wake up every day. The red line is a 7-day rolling average and there is a grey trendline. As you can see, the spikes below zero are becoming more frequent than they were at the start of the year.

I think like most people I have always woken up in the middle of the night, but normally I am able to go straight back to sleep. More and more, I find myself unable to simply turn over and fall asleep again and I actually end up staying awake until I get up. Strangely, this seems to happen most often on Saturdays. That used to be the day when I slept in the most.

What I also find amusing about the graphs is just how much you can tell about my life just by looking at them. Take the time I set my alarm for. You can see more or less the exact point where I finish university, when the regular early starts stop in March. You can even see that extra-evil exam when I had an extra-early start.

From then on, the time I set my alarm at more or less goes wherever my whim takes me. But you can still see little glimpses into my life. Those moments where I set my alarm at 0830 every two weeks or so? Formula 1 practice sessions. You can even see the point when it was the Canadian Grand Prix rather than a European race, the giveaway being that there is no early start for around four weeks in June. The line goes even further up because there was no early Sunday morning start for the GP2 race! The same happened when there was no F1 action for three weeks in August.

All-in-all, I’m pleased with my progress so far during the year. But the times have not become much earlier since I wrote my last update in July. Then I said my aim was to start getting up regularly at 1000. I have failed to do that and the trendline still rests above 1100. The times remain around an hour earlier than they were at the start of the year — the same as six months ago.

My target for the rest of the year now is to take all of the activity an hour earlier, then get rid of the ‘cycles’ so that I have a regular pattern. There are a couple of challenges ahead, namely the Japanese and Chinese Grands Prix which are coming up in a couple of weeks. That will inevitably cause havoc.

On my side, though, is the fact that the clocks change later this month. Essentially, my target therefore is to keep my sleeping patterns as they are. That might seem like a bit of a cop-out, but I’m sure most of this is psychological.

Perhaps unwisely, I didn’t take into account daylight savings time in my graphs. This means that instead of getting up (etc.) one hour earlier, I have really got up two hours earlier. But in a way I don’t think that’s the point. The point is, how much do I feel like I need to get up if I wake up at 1000? How different does that feel if I wake up at 1100? My aim is to start feeling like I should get up at 1000.

My first post about my sleeping patterns was a surprise hit. So I have decided to write a second update as I reach the halfway mark of my year-long experiment to keep data on my sleeping patterns.

The previous post ended on a bit of a cliffhanger as all my graphs were spiking up quite alarmingly. Since then I think progress has been quite good.

Here is graph 1 (data measured in clock times) updated to show the first six months (i.e. this year up to yesterday). As before, these are all seven day rolling averages.

As you can see, the broad trend for all of the lines is for them to go in the right direction. In fact, very recently the ‘alarm’ and ’slept until’ lines were at the lowest point they’ve been all year. However, since my sleeping patterns appear to be in cycles, that will be counterbalanced soon enough by a period where I wake up later. You can just see the start of that at the end of this graph.

The previous three months are very different to the first three months. The cut-off point for the last post came just after I had had my last class at university. Since then I have had far fewer regular engagements, but I have still had the odd activity to get up early for — exams, GP2 races, graduation ceremonies and what-have-you.

In general, I am still having a lot of trouble predicting how long I will sleep for. Choosing the right time to set the alarm for is the most difficult thing about getting my sleep under control. If I set it too late then that is useless, whereas if I set it too early I just go back to sleep, possibly not to be seen again until the afternoon!

The ‘morning’ lines (alarm, slept until, got up) have been much more unpredictable than the ‘night’ lines (bed at, slept from). In fact, the night time variables are remarkably flat, with only a little bulge a couple of weeks ago ruining an otherwise slow but relatively steady trend towards earlier times. It now feels weird to be up after, say, 0200 and I consciously try to avoid staying up beyond that time (which was otherwise commonplace for me).

Having said that, although they fluctuate a lot, the morning variables are also going in the right direction — but very slowly. At the start of the year I was most likely to wake up at midday. Nowadays I’m more likely to wake up at 1030. Considering we have also had the clocks changing in that period, I am effectively waking up two and a half hours earlier than I was at the start of the year. Assuming I end up with a normal job though I will be looking to get up three or four hours earlier than even this.

Here is graph 2 — variables measured as lengths of time.

This graph is still fluctuating quite a lot. As you can see, ‘insomnia’ is going down in general. But it is still causing me a headache. I seemingly can’t tell how tired I am, so sometimes I am unable to fall asleep for half an hour (which I consider to be normal), others for over four hours (as actually happened on one day and is distinctly abnormal).

Incidentally, the data for what I have called the ‘insomnia’ variable is slightly odd. The name is misleading. It measures the difference between the time when I go to bed and my estimate of when I fall asleep. But often I am sitting in bed reading a book before actually turning in. So perhaps you can knock, say, half an hour off the figures to get a better idea of my ‘insomnia’.

Another notable aspect of the graph is the fact that the area of green — which I have called ‘lazy’, the difference between the time when I wake up and when I get up — has increased. I think this is partly due to some advice I followed in the comments to the last post. Duncan2 and 4u1e both suggested putting my alarm at the other side of the room.

I had tried that trick before, but with little success. Now I have put it at the complete opposite side of the room, a good 15 or so yards from my bed, and in an awkward position. At first it certainly had me waking up earlier — but I felt so awful that I just stayed in bed for ages! Hence the increase in ‘laziness’.

As you can see on the ‘lazy’ graph, it is pretty easy to pinpoint the moment when I started putting the alarm at the other side of the room, with a massive spike in early April. Since then the spikes have still happened from time to time. But they are getting smaller, suggesting that I am coping better with the scheme now. However, the ‘lazy’ graph is disappointingly the one graph where the trendline is going in the wrong direction. So that’s something for me to work on over the coming months.

Another point to note from the comments is that I have now extended my caffeine curfew. Beforehand I just banned coffee after around 1800. Now I have banned tea as well. Green tea is banned from about 2000 onwards except for when I am working until 2100, in which case I have that final mug of caffeine at the first opportunity I get. I used to be sceptical about whether cutting out caffeine was actually working for me. But since I started cutting out tea as well I have found that I am getting to sleep earlier.

I think overall the year so far has been positive in terms of getting my sleep under control. Now what I am aiming for is to start waking up regularly at 1000 without feeling rotten and hauling myself out of bed at that time as well!

I had a new year’s resolution this year. As part of my current crisis (i.e. having to become responsible), I am trying to get my notoriously bad sleeping pattern in order. Amazingly, I have stuck to the first part of the resolution.

For the past three months, I have been keeping a log of my sleeping patterns. It’s quite detailed. Every morning I open up my big Excel spreadsheet and record the time I went to bed, when I think I fell asleep, when I woke up and when I actually got my lazy arse out of bed. I also note when I set my alarm for. From all of this I work out how long I am unable to sleep, how long I sleep in and… well, how lazy I am.

Obviously it’s not an exact science. I obviously can’t tell exactly what time I fall asleep at, but I think I have a good idea — often because I end up listening to the radio (with its regular half-hourly bulletins) because otherwise I just get frustrated at not being able to fall asleep which makes the problem even worse.

So I am three months in. My excellent maths skills tell me that this means I am a quarter of the way through the project. A good time to look over the data. That can mean only one thing: graphs.

This first graph shows rolling seven day averages for the five variables that are measured as clock times. The labels are probably self-explanatory enough.

This second graph shows the variables that are measured in lengths of time. ‘Insomnia’ is the length of time it takes for me to fall asleep (i.e. the difference between the time at which I fall asleep and the time at which I go to bed). ‘Asleep for’ is self-explanatory, and ‘Lazy’ is the length of time I spend in bed after waking up. Stacking these shows the amount of time I spend in bed per day, which you can read off the y-axis.

One thing that I have noticed is that my sleeping patterns appear to be in cycles. There are fairly distinct peaks and troughs and it doesn’t look quite as ‘random’ as you might expect.

I will spare you from the boring details that led to every peak and trough. What I have learned from generating these graphs is the fact that even the slightest disruption to my routine can have massive effects on my sleep.

For instance, the pronounced peak that happens in around week 3 of February came about because I had a reading week on just one of my courses. This small change led to me falling asleep one hour later than normal and sleeping a further hour longer than normal.

The broad trend, though, had been good. The lines were going in the right direction (of course I am trying to sleep earlier in the day). But since the middle of March it has all gone wrong.

It started when I travelled up to Dundee to attend a friend’s birthday outing and opted to take the first train home (so I didn’t get to sleep until a disgusting 8am). This was exacerbated by the Australian Grand Prix which, of course, I had to watch live (you have to get your priorities straight, you see). Later that week I (almost) inexplicably woke up hours earlier than I expected to.

All of this left me rather more sleep-deprived than normal. Ironically, of all the variables, the most important one — length of time spent asleep — is fairly stable at 8 hours, which is said to be the recommended amount of sleep. But this week it fell through the floor to a seven day average of 6 hours.

It felt okay at the time, but I know from experience that this situation can only last so long until it catches up with you. Combine this with the Malaysian Grand Prix (which I also had to watch live) and the fact that I no longer have any classes, and the result is the mess you see towards the end of the graph.

So now I am at a situation where the earliest I have to get up all week for the next few weeks is 4pm. With no incentive to get up, I just don’t. A hefty dose of self-discipline is clearly in order, but more than once in the past two weeks I have slept straight through four alarm clocks without having any recollection of switching them off.

The result is now that I am falling asleep at around 5am if I am lucky, and waking up at around 2 if I’m lucky. Smart alecs might point out that I should maybe try going to bed earlier. But if you look at graph 2 you will see that I am already spending an average of 3 hours in bed without being able to get to sleep.

These seven day average graphs are nothing though. I have also generated separate graphs for each of the variables showing daily changes (in blue, of course). The red line is the familiar seven day average, and there is a grey trendline (though in this particular graph is is almost indistinguishable from the gridline marking 8 hours).

As you can see, it fluctuates wildly during my ‘routine’ weeks. Despite the ‘overall’ consistency of the seven day average, and the reassuring fact that the trendline is almost parallel to the x-axis, the fluctuating blue line is alarming. I suspect that this is the root of most of my problems.

In extreme cases, I will get little more than 2 hours. This obviously has to be offset sometime, so a night soon afterwards I will be knocked out for 12 hours. Those two days alone would be enough to send my entire sleeping pattern askew, never mind having a similar pattern repeated several times in a matter of months.

The silver lining is that the end of this graph actually looks quite good. As you can see, I have not emerged from this peak yet, but it looks as though it is a peak to compensate for the previous trough. And the wild fluctuations have stopped — mostly because I don’t have any early starts for the time being.

The problem is that once it all settles down it will almost certainly be into a routine something along the lines of sleeping from 4am until 12 noon. This has been the way of it for years now. The difficult part is shifting this so that it is, say, midnight until 8am.

Why am I a nightowl? Well, one possibility is the fact that I haven’t had anything resembling regular early starts since the distant days of school. But even then I stayed up late and was seriously sleep deprived. According to my mother, I was even a nightowl when I was a baby. It looks like I was born this way.

Now, the task is working out how I can adapt this inconvenient personal trait into something that I can manage in my adult life. (Can you tell that this is a stressful period for me?…)