My first post about my sleeping patterns was a surprise hit. So I have decided to write a second update as I reach the halfway mark of my year-long experiment to keep data on my sleeping patterns.

The previous post ended on a bit of a cliffhanger as all my graphs were spiking up quite alarmingly. Since then I think progress has been quite good.

Here is graph 1 (data measured in clock times) updated to show the first six months (i.e. this year up to yesterday). As before, these are all seven day rolling averages.

As you can see, the broad trend for all of the lines is for them to go in the right direction. In fact, very recently the ‘alarm’ and ’slept until’ lines were at the lowest point they’ve been all year. However, since my sleeping patterns appear to be in cycles, that will be counterbalanced soon enough by a period where I wake up later. You can just see the start of that at the end of this graph.

The previous three months are very different to the first three months. The cut-off point for the last post came just after I had had my last class at university. Since then I have had far fewer regular engagements, but I have still had the odd activity to get up early for — exams, GP2 races, graduation ceremonies and what-have-you.

In general, I am still having a lot of trouble predicting how long I will sleep for. Choosing the right time to set the alarm for is the most difficult thing about getting my sleep under control. If I set it too late then that is useless, whereas if I set it too early I just go back to sleep, possibly not to be seen again until the afternoon!

The ‘morning’ lines (alarm, slept until, got up) have been much more unpredictable than the ‘night’ lines (bed at, slept from). In fact, the night time variables are remarkably flat, with only a little bulge a couple of weeks ago ruining an otherwise slow but relatively steady trend towards earlier times. It now feels weird to be up after, say, 0200 and I consciously try to avoid staying up beyond that time (which was otherwise commonplace for me).

Having said that, although they fluctuate a lot, the morning variables are also going in the right direction — but very slowly. At the start of the year I was most likely to wake up at midday. Nowadays I’m more likely to wake up at 1030. Considering we have also had the clocks changing in that period, I am effectively waking up two and a half hours earlier than I was at the start of the year. Assuming I end up with a normal job though I will be looking to get up three or four hours earlier than even this.

Here is graph 2 — variables measured as lengths of time.

This graph is still fluctuating quite a lot. As you can see, ‘insomnia’ is going down in general. But it is still causing me a headache. I seemingly can’t tell how tired I am, so sometimes I am unable to fall asleep for half an hour (which I consider to be normal), others for over four hours (as actually happened on one day and is distinctly abnormal).

Incidentally, the data for what I have called the ‘insomnia’ variable is slightly odd. The name is misleading. It measures the difference between the time when I go to bed and my estimate of when I fall asleep. But often I am sitting in bed reading a book before actually turning in. So perhaps you can knock, say, half an hour off the figures to get a better idea of my ‘insomnia’.

Another notable aspect of the graph is the fact that the area of green — which I have called ‘lazy’, the difference between the time when I wake up and when I get up — has increased. I think this is partly due to some advice I followed in the comments to the last post. Duncan2 and 4u1e both suggested putting my alarm at the other side of the room.

I had tried that trick before, but with little success. Now I have put it at the complete opposite side of the room, a good 15 or so yards from my bed, and in an awkward position. At first it certainly had me waking up earlier — but I felt so awful that I just stayed in bed for ages! Hence the increase in ‘laziness’.

As you can see on the ‘lazy’ graph, it is pretty easy to pinpoint the moment when I started putting the alarm at the other side of the room, with a massive spike in early April. Since then the spikes have still happened from time to time. But they are getting smaller, suggesting that I am coping better with the scheme now. However, the ‘lazy’ graph is disappointingly the one graph where the trendline is going in the wrong direction. So that’s something for me to work on over the coming months.

Another point to note from the comments is that I have now extended my caffeine curfew. Beforehand I just banned coffee after around 1800. Now I have banned tea as well. Green tea is banned from about 2000 onwards except for when I am working until 2100, in which case I have that final mug of caffeine at the first opportunity I get. I used to be sceptical about whether cutting out caffeine was actually working for me. But since I started cutting out tea as well I have found that I am getting to sleep earlier.

I think overall the year so far has been positive in terms of getting my sleep under control. Now what I am aiming for is to start waking up regularly at 1000 without feeling rotten and hauling myself out of bed at that time as well!

I had a new year’s resolution this year. As part of my current crisis (i.e. having to become responsible), I am trying to get my notoriously bad sleeping pattern in order. Amazingly, I have stuck to the first part of the resolution.

For the past three months, I have been keeping a log of my sleeping patterns. It’s quite detailed. Every morning I open up my big Excel spreadsheet and record the time I went to bed, when I think I fell asleep, when I woke up and when I actually got my lazy arse out of bed. I also note when I set my alarm for. From all of this I work out how long I am unable to sleep, how long I sleep in and… well, how lazy I am.

Obviously it’s not an exact science. I obviously can’t tell exactly what time I fall asleep at, but I think I have a good idea — often because I end up listening to the radio (with its regular half-hourly bulletins) because otherwise I just get frustrated at not being able to fall asleep which makes the problem even worse.

So I am three months in. My excellent maths skills tell me that this means I am a quarter of the way through the project. A good time to look over the data. That can mean only one thing: graphs.

This first graph shows rolling seven day averages for the five variables that are measured as clock times. The labels are probably self-explanatory enough.

This second graph shows the variables that are measured in lengths of time. ‘Insomnia’ is the length of time it takes for me to fall asleep (i.e. the difference between the time at which I fall asleep and the time at which I go to bed). ‘Asleep for’ is self-explanatory, and ‘Lazy’ is the length of time I spend in bed after waking up. Stacking these shows the amount of time I spend in bed per day, which you can read off the y-axis.

One thing that I have noticed is that my sleeping patterns appear to be in cycles. There are fairly distinct peaks and troughs and it doesn’t look quite as ‘random’ as you might expect.

I will spare you from the boring details that led to every peak and trough. What I have learned from generating these graphs is the fact that even the slightest disruption to my routine can have massive effects on my sleep.

For instance, the pronounced peak that happens in around week 3 of February came about because I had a reading week on just one of my courses. This small change led to me falling asleep one hour later than normal and sleeping a further hour longer than normal.

The broad trend, though, had been good. The lines were going in the right direction (of course I am trying to sleep earlier in the day). But since the middle of March it has all gone wrong.

It started when I travelled up to Dundee to attend a friend’s birthday outing and opted to take the first train home (so I didn’t get to sleep until a disgusting 8am). This was exacerbated by the Australian Grand Prix which, of course, I had to watch live (you have to get your priorities straight, you see). Later that week I (almost) inexplicably woke up hours earlier than I expected to.

All of this left me rather more sleep-deprived than normal. Ironically, of all the variables, the most important one — length of time spent asleep — is fairly stable at 8 hours, which is said to be the recommended amount of sleep. But this week it fell through the floor to a seven day average of 6 hours.

It felt okay at the time, but I know from experience that this situation can only last so long until it catches up with you. Combine this with the Malaysian Grand Prix (which I also had to watch live) and the fact that I no longer have any classes, and the result is the mess you see towards the end of the graph.

So now I am at a situation where the earliest I have to get up all week for the next few weeks is 4pm. With no incentive to get up, I just don’t. A hefty dose of self-discipline is clearly in order, but more than once in the past two weeks I have slept straight through four alarm clocks without having any recollection of switching them off.

The result is now that I am falling asleep at around 5am if I am lucky, and waking up at around 2 if I’m lucky. Smart alecs might point out that I should maybe try going to bed earlier. But if you look at graph 2 you will see that I am already spending an average of 3 hours in bed without being able to get to sleep.

These seven day average graphs are nothing though. I have also generated separate graphs for each of the variables showing daily changes (in blue, of course). The red line is the familiar seven day average, and there is a grey trendline (though in this particular graph is is almost indistinguishable from the gridline marking 8 hours).

As you can see, it fluctuates wildly during my ‘routine’ weeks. Despite the ‘overall’ consistency of the seven day average, and the reassuring fact that the trendline is almost parallel to the x-axis, the fluctuating blue line is alarming. I suspect that this is the root of most of my problems.

In extreme cases, I will get little more than 2 hours. This obviously has to be offset sometime, so a night soon afterwards I will be knocked out for 12 hours. Those two days alone would be enough to send my entire sleeping pattern askew, never mind having a similar pattern repeated several times in a matter of months.

The silver lining is that the end of this graph actually looks quite good. As you can see, I have not emerged from this peak yet, but it looks as though it is a peak to compensate for the previous trough. And the wild fluctuations have stopped — mostly because I don’t have any early starts for the time being.

The problem is that once it all settles down it will almost certainly be into a routine something along the lines of sleeping from 4am until 12 noon. This has been the way of it for years now. The difficult part is shifting this so that it is, say, midnight until 8am.

Why am I a nightowl? Well, one possibility is the fact that I haven’t had anything resembling regular early starts since the distant days of school. But even then I stayed up late and was seriously sleep deprived. According to my mother, I was even a nightowl when I was a baby. It looks like I was born this way.

Now, the task is working out how I can adapt this inconvenient personal trait into something that I can manage in my adult life. (Can you tell that this is a stressful period for me?…)

I recently had to write an essay for university about changes in Scotland’s population since 1945. While I was writing that I happened, almost by chance, upon The Registrar General’s Annual Review of Demographic Trends 2004.

What’s so special about 2004? It was the 150^th anniversary of civil registration (which began in 1855, in case your arithmetic isn’t too hot). So the Registrar General took the opportunity to delve into the statistics and produce lots of interesting analysis on this historical trends of Scotland’s population as far back as records go.

While I should have been writing my essay, I found myself perusing the graphs. I’m that sort of person. Obsessed with graphs. I’ll share a few of the most interesting ones with you.

Sorry about the illegibility of some of these. I have to confess that I am stealing the Registrar General’s bandwidth (although this does not vex me because the public is paying for that bandwidth, and something tells me they won’t get me with the goatse treatment). The original images are huge (much bigger than they appear on the PDF), so I have had to crudely reduce them in size to fit in these pages.

Immigration

Immigrants flooding this country! Er, or not.

Literacy

The Registrar General used the number of people signing by mark while marrying as a crude measure of literacy up until 1915.

Marriages in Gretna

They are a much more modern phenomenon than you might imagine.

Divorce

I bet if you got divorced in the nineteenth century it was national news.

Death

My favourite topic! You can see the general long-term decline in the number of deaths. But more interestingly, the peaks a troughs become much less extreme, signifying improved medicinal technology and ability to cope with epidemics.

These are just a few of my favourites, but I could have included twice as many (to be honest, you’re lucky I didn’t). But if you’re interested in Scotland’s modern history and demography I’d definitely take a look at the full document.

I am a big fan of the music discovery and social networking site Last.fm. I am also a lover of beautiful pointless graphs. So LastGraph was always going to be a winner for me.

The existence of LastGraph makes me so happy. Like Andrew Godwin, who wrote LastGraph, I read this webpage a while ago and thought to myself, “I want.” But I thought it was a bit of a long shot to expect anyone to provide such a service.

Obviously I was wrong! What’s more, there are plenty of different options to choose from, so once you’ve made one graph you can make another and look at it differently.

Unfortunately, it only goes back as far as the beginning of 2005, so the early part of my Last.fm history. But what remains is fascinating to look at. The snapshot I have posted above is from May–July 2007. For some reason I always get more nostalgic about music that I listened to during summer, so I’m pretty sure that looking back I will get a little bit misty-eyed about Justice, Simian Mobile Disco and Can (incidentally, they were all albums that I bought on my last trip to the pre-HMV Fopp).

If, for some reason, you are interested in seeing the entire graph, it is available to download here. But be warned — the PDF is quite a large file!

The colours represent how early I first listened to a band. Reds and oranges were the first bands I listened to; purples are the most recent and greens are in the middle. Glancing at my graph, it is clear how much more eclectic my listening habits are. As time goes on there are fewer large patches, and the reds have been squeezed out by a larger variety of colours.

Surprisingly, it is completely impossible to follow any band all the way through my graph. I think this is down to the fact that I tend to just put iTunes in shuffle mode and let it select albums itself. I suspect the same bands would crop up week after week if I tended to choose for myself.

Even so, I am amazed that I can only pick out Battles three times on my graph. I have been flat-out obsessed with them for over a year (and particularly since the release of Mirrored), but seemingly I have not listened to them as much as I had thought.

My theory is that when I am out and about (and therefore not scrobbling), I choose what album to listen to more often, so this is where I have listened to Battles the most. (I think this happened to Polar Bear as well, who I listened to a lot, but are nowhere to be seen on my Last.fm profile.)

Funnily, I can see plenty of patches where I listened to Eels a lot. I never understood why Eels appear so high up on my Last.fm profile (not just overall, but for the past 12 months as well). I’m not sure the graph has aided my understanding of this, but I have clearly listened to Eels a lot more often than I thought.

I just got an iPod, so I will be able to scrobble while I’m out and about as well. I predict an increase in both the amount of music and large patches appearing on the graph. I hope LastGraph sticks around because it will be really interesting to create more of these graphs to compare over time.

H/T lots of people including Plasticbag and Somefool.

I am not qualified to talk about Chris Lightfoot. I have never come close to even meeting him, and I only know of him through his blog and his work on certain mySociety projects and the like. But the news of his unexpected death has made me quite sad.

As MatGB notes, if you have never heard of Chris Lightfoot, you will probably at least have used a website that he helped build. For instance, the Downing Street e-petitions website which has been making the news recently. Or WriteToThem. Or PledgeBank.

Whenever I read anything that Chris Lightfoot wrote, he always came across as incredibly clever. I didn’t always agree with him. For instance, I didn’t take his rants about Chip and Pin seriously, particularly as he acknowledged that Chip and Pin is more secure than old fashioned signatures.

However, I recognise now that he was right to be wary. We have seen the banks use it more as a mechanism for shifting the burden of fraud onto the consumer more than anything else. (Not to mention stories like this.)

I usually feel equipped to fight my corner and defend whatever I have written on this blog. The very few times that Chris Lightfoot left a comment (on my old site; his comments were on the Haloscan system, so will have disappeared into the internet black hole) were the exception. He changed my mind.

His blog was always well worth a read. I would almost always learn something from his posts which were frequently backed by a mind-boggling graph. He might not have updated his “real” blog for a long time, but I always kept my eye on his linklog. Many posts on his linklog were decent enough blog posts in their own right. There were plenty of very interesting links posted there. He was still updating it quite recently.

I also enjoyed his novel take on those online political surveys. Having two axes instead of one is kind of old news. But what he plotted on the actual axes caused people to do a double-take. But he did it on the basis of actual data. It made sense and allows us to understand the political landscape a little bit better. I was looking forward to any political surveys he might have done in the future.

From the various tributes that have been written over the past day or two by those who knew him considerably better than I did, it is clear that Chris Lightfoot was entirely selfless and helpful. He was also innovative and very clever as well as highly principled and passionate in the causes he believed in. Most importantly, he believed in the right causes.

I can’t help but wonder if the world without Chris Lightfoot is a slightly more dangerous place.

• Bloggerheads
• The Devil’s Kitchen
• Chicken Yoghurt

Update: Forgot to mention also that yesterday the first campaign leaflet for the Scottish elections plopped on the doormat yesterday. Every time I come across a piece of election literature I think of Chris Lightfoot’s brilliant blog post on how he decided who to vote for:

read each manifesto until you encounter something really offensive or stupid, then stop and reject that party.