Posts tagged with statistics

Managing Input

October 18th, 2009

You know, I’m pretty lazy. I don’t want to do anything. I’d be happy to just read cracked.com and tvtropes.org all day. But I’m also a megalomaniac. I wanna know everything. And I mean everything. I wanna understand about the Great Vowel Shift, the colonization of Australia, the evolution of the influenza virus, the performance of text matching… everything. Unfortunately, for now at least, the human brain can’t know everything. You can’t just pour stuff in until you run out of it. Upload Wikipedia and be done with it. You need to actually work for it and study. Slowly, painfully and did I mention slowly?

Sure, you can use a SRS to manage information so that you don’t forget the stuff again. You only learn it once and then do your daily repetitions so it stays in your brain. That’s pretty cool already, but there are two drawbacks: it’s a little boring and, worse, tiring. It’s in fact so tiring that I had to give up an otherwise pretty neat sleeping schedule. I read a few articles and forum posts on this and everyone seems to agree – you can do about 200 repetitions per day, maybe 300. More than that is too tiring, too time-consuming, too boring and maybe just plain impossible. Sure, for a few days you can do more, but I never saw anyone maintain this. You just burn out. (But you can study more, just not via SRS.)

Of course, old repetitions and new facts compete for resources. You can’t just add and add facts. Soon your daily repetitions surpass 200 and you will forget them anyway. But what is the optimal course of action? Is it better to add facts slowly to minimize the risk of burn-out? Soon, university starts again and I’ll have to study for exams. That means adding facts I don’t really wanna learn that much (at least right now), but I’m also learning other things (2 languages, for example). So on one hand, I want to learn as comfortable as possible, but on the other hand, I need to get done in time, so I’d better be fast.

What do you do in such a situation? You run a simulation, of course! I expanded my handy SRS simulator and tested a few possible configurations. I’m just going to show you the 3 most interesting ones:

amount of daily repetitions for 200 days

amount of daily repetitions for 200 days

new facts remaining each day

new facts remaining each day

A few short notes first.

I set the amount of facts to learn to 3400. That’s my current number of new, unseen facts. Don’t ask how I got that many (*shame*, *shame*), but it’s not unusual for many learners. Of course, if you are learning basically forever, there won’t be a “last new fact” – you’ll constantly add new ones. In this case, just look at the first graph and find the plateau for each curve. You will never drop below this.

The 3 configurations are:

  • A -> a maximum of 100 repetitions per day; add up to 50 new facts
  • B -> a maximum of 200 repetitions per day; add up to 50 new facts
  • G -> a maximum of 200 repetitions per day; add up to 200 new facts

The maximum is only respected when adding new facts, not when doing due facts.

Ok, now let’s have a look at the actual graphs.

The first and most obvious thing is that G totally fails at it’s goal. After about a week, the due cards explode right in your face and you’ll have to face up to 350 repetitions per day. And this continues to happen all the time, so you are pretty likely, at some point, to just give up. You feel so bad about the extreme and unexpected workload you’re facing that it becomes counter-productive. So to just “fill up” until you hit your maximum number of comfortable repetitions a day is a really bad idea.

Furthermore, we can see that both A and B are pretty good at keeping the maximum under control, A is pretty inefficient at it. You have lazy and busy days in a pretty regular pattern, but you never max out. Not really that good, but workable nonetheless. However, what you can see is that if you are working through a fixed amount of facts, B gets you into the “lazy phase” much sooner. A has many lazy days, but on average it is actually more work than B, not less.

The second graph shows us how fast we are progressing. It takes G about 50 days to work through all facts, B needs about 80 days and A about 150. Sure, G is pretty fast, but actually not much faster than B, at only about 40% less time. However, it is quite clear that A really is slow. It needs at least twice as long to work through the same amount of facts than the other two approaches.

Conclusion

I think the data allows us to make two important conclusions.

First, do not add facts like crazy. Trying to just add facts until you fall asleep might work for a few days, but very soon you hit the point where all those repetitions burn you out. They would demand up to twice the amount of work you were capable of just a week ago, so most likely, you will just fail to do them and forget everything again. Basically, you end up really tired, demotivated and not much smarter than before. A good waste of your time.

Second, to be lazy, work harder. It might sound counter-intuitive at first, but it makes sense. If you try to be lazy early on by working below your actual threshold, you will, on most days, actually have to work more than you expected. You will learn much slower and spend a lot more time on your repetitions. The reason for this is that you are spreading out your early repetitions, which are by far the hardest. If you work a bit more and get those early repetitions over with, it will get much easier later on. If you work consistently at your threshold, it wil be easier to make a routine out of it and your progress will be faster.

Finally, the simulation allows me to pick some more useful values. If you have another look at the second graph again, you can see that A and G look a bit like stairs, but B is smooth. This means there is an optimal value of new facts per day to pick that consistently maxes out your workload by looking at the slope of the curve. This optimal value, interestingly enough, is about 42. I do not believe this to be a coincidence.

Eating My Own Shit

October 15th, 2009

If you start with a bad assumption, you will invariably reach bad conclusions and constantly delude yourself about that fact. The only way to fix this is to regularly question your own basic assumptions about things. The scientific method provides a neat way (in fact, the only way) to do just this. Ask yourself: Is there a different explanation? On what data do you base your decision? Does another interpretation fit, too?

I don’t just claim this, but actually live this way. I want to demonstrate this by changing my opinion about sleep. Frustrated with polyphasic sleep, I have reëvaluated my own assumptions and checked the data. I read more studies and biology texts, looked through my own records and re-read a few polyphasic blogs. And I must conclude that polyphasic sleep, by and large, doesn’t work.

Let’s start at the opposite end – what does work? Well, polyphasic sleep is the best (known) option you have when you can’t have more than 2-4 hours per day of sleep. If you must sleep that little, for example because you are into solo sailing or your newborn child and 2 jobs keep you up all day, than polyphasic sleep is right for you. It minimizes the damage this kind of life will do, but you will still be worse off. You will still be sleep deprived.

Ok, having acknowledged that, let’s start with the criticism. In fact, it’s a very simple criticism because it only involves one point.

Polyphasic sleep destroys your memory.

Sure, you are awake more (if you are lucky, most people aren’t and delude themselves to the fact), but you can’t use the time in any meaningful way. You can’t learn more, in fact, you’ll learn less. All existing studies show that performance is slightly below normal levels, which means you have 4-6 more hours of waketime, but you are actually performing worse than if you had slept them all. Great job. That’s like taking a shortcut, only to drive slower so that you arrive even later.

Why is there not a single polyphasic scientist? No, Tesla was not polyphasic, he crashed regularly. Edison lied about his schedule and, while being mostly polyphasic, didn’t save any time (and he was not a scientist). Buckminster Fuller only slept polyphasically when touring, for the reason I mentioned above.

Why is there not a single polyphasic polyglot? You’d think that someone who is learning multiple languages at the same time would be glad over every single hour per day he can get. Yet, not a single one of them is documented to be polyphasic. Some have tried (mostly early polyglots), no one was happy with it.

Why does no military or space agency advocate polyphasic sleep? There are several studies researching it, but they all document a severe loss of performance and they all advise against it, except when external circumstances force you to be polyphasic, as mentioned earlier.

Why does all data collected via SRS, like for example Supermemo, show that sleeping in big chunks correlates with good performance? If there are working examples of polyphasic sleepers, no one of them has ever demonstrated this via their SRS statistics, and Supermemo captures a lot of those. There isn’t a single example of someone sleeping 4 hours or less per day and still getting a normal retention rate for the same amount of data learned.

There is a simple answer to these questions: Because polyphasic sleep doesn’t work. It’s bullshit. For all the claims of “superhuman” feats, there hasn’t been a single bit of evidence for it. Proponents have made all kinds of claims and assurances, yet have presented nothing. Most of them don’t even seem to be capable of grasping the importance of empirical evidence. It is pseudoscience.

Conclusion

If you don’t care about your memory and you don’t care about being able to learn, sure, go right ahead. If you also keep in mind that the majority of people drop out of polyphasic sleep after a month or less, I would recommend a better alternative: Amphetamines. It has exactly the same amount of advantages (awake at all costs), is easier to use and fucks you up just the same.

I’m now recanting all my previous posts and claims about polyphasic sleep. They are wrong. I have marked the posts accordingly. Don’t sleep polyphasically, yo. If you still think that it works, prove it. And no, “I’ve been doing this for months and I’m fine!!1!” isn’t proof. Get some real data. A SRS is a good starting point. Show that learning for 8 hours or more per day works as least as well as normally and doesn’t destroy your sleep. You won’t be able to, but try all the same. Hint: just log your exact sleeping times and do a few standard performance tests. This alone will probably demonstrate that you are deluding yourself.

So what’s the real alternative? This.

Inevitable Patterns

September 30th, 2009

(Yeah, sleep again. Didn’t I say I’d write about something else for a change?)

I highly recommend writing down your exact sleep times and level of tiredness every day. It will help you see patterns and prepare for tight spots in advance. However, occasionally, it will also show you when things are not working.

Like with my sleep. For the last 3 months, there is always the same pattern. I adopt a strict sleep pattern. It works great for exactly 6 days, with maybe a slight amount of tiredness here and there. Then, on day 7, I suddenly get hit with all that missing tiredness at once. Once, I was still able to go another day, but most of the time, I just crash on the spot and sleep at least 8 hours. Of course, this is anything but practical. There is nothing I can do to avoid the crash. I have done uberman and it was easier.

I noticed this patterns before, but I’m now confident it will always be there. So it’s not my own incompetence of adapting, but something my brain just does. I never read about another polyphasic sleeper noticing that, though chronically undersleeping and then catching up on the weekend is pretty common. Yet I don’t feel like I’m getting too little sleep – I can function perfectly fine for 20 hours a day, with no caffeine and no concentration problems at all. It’s as if constantly delaying sleep during school made my brain unable to not crash once a week.

So, solutions. I mentioned 2 a few weeks ago, namely incorporating the crash into the schedule or trying to get more sleep per day. I know of several polyphasic sleepers how do crash regularly, generally on the weekend, but they do this once a month, not once a week. So I don’t think this is a good solution. I’m pretty sure crashing that often is a sign of deeper problems. Also, when doing uberman and uberman with a core, I didn’t notice this pattern. Those two worked just fine.
I also get vivid dreams everytime I sleep and feel refreshed afterwards. I make sure it’s dark, avoid sugar or anything else that screws with your awareness and so on. Really, if sleep quality is the problem than it is beyond my powers anyway.

Conclusion? It is possible to be severely sleep deprived without noticing it at all for about a week. Do not feel safe after you had several good days – you might still be doing it wrong. Reassuring, isn’t it? :) Anyway, I’ll try to make the core longer again, hoping to average it out.

ashuku – a personal statistics tool

September 16th, 2009

Statistics. I love statistics. And graphs. Graphs are cool, too.

It’s funny, actually. I really suck at statistics. I have a hard time understanding probabilities and statistics is probably the one mathematical field I understand the least. But I still love it. I track a lot of data and love reading tables. I have several books full of yearly death statistics, broken down by age, gender, cause, region and so on. Some of the greatest stuff I ever read. Crime statistics are really cool as well.

Anyway, it might come as no surprise to you then that I like correlating personal data. If I do this change in my life, how does it affect me? Is their a correlation between sleep time and happiness? What about nutritional supplements? So I wrote a tool to track and analyze just this. [0]

Enter ashuku. I’m lazy, so let’s just quote the readme:

ashuku is a tool to track a multitude of daily statistics, like mood and
health. Its design goals are simplicity and fast usage.
ashuku can draw graphs [citation needed] and analyze data for correlation.
Data is stored in plain text files in YAML. It’s easy to read for both humans
and machines.

ashuku is named after one of the 5 Wisdom Buddhas, 阿閦如来 (ashuku nyorai).
He is immovable and reflects all emotions like a mirror, showing things as they
really are.

ashuku is strongly influenced by todo.txt.

Dependencies
============

* Python 3 (although the code is probably compatible with Python 2.6)
* PyYAML

Here’s a screenshot. It’s fully customizable, so don’t be afraid of the Japanese UI. It’s in English by default and you can change it however you want. :)

a screenshot

a screenshot

I’ve been using it since 9/12. The data before that is from a different tool and partially incomplete, so there.

You can grab it here: http://github.com/muflax/ashuku

[0] Well, the second one, actually. The first one was a Perl script and… you know what they say about Perl code. It’s  all true, unfortunately.