Something is broken in the world of research and knowledge. According to my research:

Subscribing to at all Elsevier journals would consume the majority of the total budget of 97% of US colleges and universities.

Let me explain how I arrived at this conclusion. First, Elsevier is rather protective of its prices. It is common for them to include a confidentiality clause in their contracts with libraries which prohibits the library from disclosing Elsevier’s prices. Despite this, the folks behind journalprices.com managed to legally discover some pricing information, although Elsevier sued at least one university in an effort to keep their prices secret.

Thanks to journalprices.com, I was able to find this great summary of academic journal prices. The prices I quote below are all from this document, which has been produced with pricing information from over 100 different institutions across all publishers, and from at least 26 contracts specific to Elsevier.

The average price of an Elsevier journal was $2248 per year. According to Elsevier’s journal index, they have 2638 different journals (1360 of which were sampled in the above price summary). So a full-cost purchase of all journals would be about $6 million per year.

In practice, I understand that libraries actually purchase bundles of journals at a reduced price. I was unable to find bundle pricing information, but sources have told me that bundle prices are set based on the cumulative full-price cost of the subset of journals that a library actually wants to buy. For example, if a library cares about the best 15% of journals (about 400) from Elsevier, then it’s very likely that they’ll cost more than the mean $2248/journal. The most expensive journal from Elsevier I’ve found so far cost over $20,000. With such high-end costs, it’s quite possible that the top 400 journals might run close to $3 million, so that a bundle deal including those journals could be labeled as a good value at this price. (This $3 million figure is an educated guess, and my 97% statistic above does not depend on it.)

In other words, although I’m going to talk about full-price journals when bundles exist, we have good reason to believe that bundle pricing is similar to the full-price cost of the journals that libraries actually care about.

What is a typical library budget?

Let’s start with a relatively big budget. In 2009, Cornell’s library had a budget of about $50 million, most of which ($30m) went to salaries and operating expenses — not books and journals. When it comes to buying journals, we care about the collections budget of a library. Cornell’s 2009 collection budget was $14m. With this budget, Elsevier’s $6m full-price tag would chop a measly 42% off of their collections budget.

But Cornell’s endowment in 2009 (about $4b) was 18th in the nation, according to wikipedia. In 1999, the US census reported there were 2,363 4-year institutions in the country. (There are probably more now, so 2636 is my conservative estimate on number of US colleges and universities in 2010.) There are currently 73 US colleges and universities with an endowment over $1b (from wikipedia again).

This means 2563 US colleges and universities have less than 1/4th the endowment of Cornell. That’s 97% of all US colleges and universities. As another sample point, consider the University of Iowa, which had a $12m library collections budget in 2009/2010, and around $1b endowment. It seams reasonable to guess that schools with smaller endowments (97% of them) will have at most a $12m collections budget. If that’s true, this completes the argument that 97% of schools would lose the majority of their budget buying Elsevier’s full collection.

For the sake of completeness, let’s look at another library budget outside of this top-endowment list. UC Santa Cruz had a collections budget of about $4.5m in 2008/2009. In other words, the full-price cost of Elsevier’s journals exceeds their entire collections budget for all books and journals by about $1.5 million. My impression is that this budget is much more common than the $12 million figure I saw for the top-endowment schools.

Elsevier is a multi-billion dollar company based on these egregiously expensive journals. Non-profit academic journals tend to cost about 75% less (based on all-fields, all-publisher journal prices, for-profit vs non-profit from here). Elsevier made about $3b revenue in the 2010-11 fiscal year (source); it seems reasonable, combined with the above analysis, that most of this $3b came from our colleges and universities. All of this together means, as an educated guess from these figures:

Switching from Elsevier to non-profit journals would save US schools over $2b per year, or about $1m per school per year.

I’ve focused on US institutions because I found it easier to get the relevant statistics, but this is a global problem.

What can we do about it?

We can decrease the need for Elsevier journals by refraining from submitting our work to them (it’s worth another article to mention that Elsevier doesn’t pay its authors). This is the motivation behind thecostofknowledge.com, a grass-roots campaign inspired by Tim Gowers’s stand against Elsevier. This effort has already gathered, in under two weeks, over 3,000 individual pledges to withhold future work from Elsevier, many from scholars eminent in their field.

At the same time, we can give researchers more publication options by supporting open access and free-to-copy publishing initiatives. Some examples are Scholastica, OpenRePub, and peer evaluation — all sites created to support freedom of knowledge for research. Publishing in the arxiv is a great way to share work. The arxiv model — similar to self-publishing — would benefit from the prestige and quality assurance of a peer review process. I personally believe the most effective step in this direction is the creation of more open access journals of high prestige. In particular, editorial boards need to move away from for-profit publishers and into open access.

This article has been about high prices, which are a major problem with our current research system. There are a number of other problems with the system that I haven’t discussed here; I should mention at least these two: First, that Elsevier is not alone in its behavior. There are several other for-profit academic publishers, such as Springer and Wiley, which would not fare much better on a good-for-research score. Secondly, and perhaps even more importantly, is the question of author’s rights. It is standard practice for an author to receive no money for their articles to these publishers, and to sacrifice essentially all their rights to the material they submit. With very limited exceptions, the article is now behind a paywall. It would take at least another post to fully outline the details of author rights. The sad summary is that loss-of-copyright is taken for granted to such an extent that price may feel more compelling an argument to many than ownership of the work itself.

On Friday (30 Sept 2011), US drones purposefully killed two US citizens without a trial. We can be confident that one of the men killed was active in supporting acts of violence against the US (and others); about the second man the situation is less clear.

Is this legal? Absolutely not. The fifth amendment states that “No person .. shall be deprived of life, liberty, or property, without due process of law.” Rising above actions like this are part of the bedrock of our country, explicitly vilified since the bill of rights was effected in 1791.

In practice, we have reached a point where sticking the label “terrorist” on a person unilaterally denies their fundamental civil protections. We look back on the days of the “unamerican” slur and laugh, yet here we are again. Black-box labels like this are dangerous, and have always been. Once someone is called an X, you’re not supposed to ask, “how do we know they’re an X?” or “don’t they still have rights?”

This is a slippery slope, one we may already be sliding down. Earlier today (2 Oct 2011), Aaron Bassler was shot and killed on suspicion of murder. The manner of his death seems questionable to me. Reading articles such as this one, it sounds as if he was shot without warning, and that he did nothing at the moment to provoke being killed. Use-of-deadly-force standards are tricky, but my impression is that they were violated here.

There’s something called the Garner standard, named after a supreme court case (Tennessee vs Garner 487 US 1, 1985), listing three major conditions in order to allow use of deadly force. One of these is that the suspect must be given warning if feasible. It sounds like Aaron Bassler was given no warning before being shot.

I have little sympathy for the individual targets of these attacks because there is strong evidence for their guilt (though I worry about the drone’s bystander victim). But we should not confuse personal judgment with justice. Something has gone wrong here. The due process of law is part of what make the US the US. It is part of civilization itself. I’m glad I’m not the only one to take note of this, and I hope we have the maturity to begin a course correction.

Download the widget!

This post is about a mac os x dashboard widget I wrote.  It shows the calendar date in a new measurement system: download it here.  (Unzip & double-click the widget to install it.)

I use a different calendar system than most people, one that I made up, but that I think is extremely practical.  I call it the 7date because it’s essentially the number of days passed in a given year, written in base 7.  Any single number-of-days system is much better than the current month/day system, which is much worse than, say, the imperial metric system (inches, feet, etc) versus the SI metric system (centimeters, meters, etc).  Because, for example, a single month is not a standardized unit of measure!!  WHAT?!?!  In the future, people will either laugh or just scratch their heads in bewildered amusement at our archaic measurement standards with respect to time.

I think it makes sense to use base 7 to measure days of the year.  Why?  Because weeks have 7 days.  Weeks are very deeply ingrained into our culture.  We are used to working Monday through Friday, and have many traditions of what to do on Saturdays or Sundays, etc.  If I were a pure scientist trying to devise a universal system of time measurement for use on any planet, by any society, I would not use weeks.  But I’m not being that abstract.  I’m trying to be very, very practical.  If we use a base 7 calendar, then we always know what day of the week a day is by the date itself, since the last digit is 0-6, each one corresponding to a given day.  In the current system, the mapping changes at the start of each year, but we could alter the first day of the week to always be a Saturday, so that days ending in 0/1 are Saturday/Sunday, and then we would immediately know the day of the week of any date – for any year – based on this system.  We could have a slightly altered idea of a month as 7 weeks (call it a 7month), and we would immediately know the 7month (and week-within-month) by looking at a single date — the month would be any digits excluding the rightmost two digits (e.g. May 3rd is 233., so we know 2 full 7months have passed).

The notation also becomes less ambiguous than current notation.  Consider the date 03/04/02, which could by written by someone in Europe thinking of April 2nd, 2003, vs 116.2003.

This system has the added bonus of giving us slightly more weekend time every year.

There are some difficulties with using any new system of measure.  Primarily, conversion.  No strongly superior system of calendar measurement will be easy to convert with the current system, because the current system is so crazy.  So, to be realistic, I admit there’s little chance of many people actually using the new system.  However, I think that nerds would enjoy it, and I can personally enjoy the new system.  I’ve used it for almost a decade now, and it works well.

However, partially since no one else uses the system, I sometimes find it hard to remember the date (I think most people forget the Gregorian date from time to time, so I think this is just human nature, not a shortcoming in the calendar system).  It’s useful to have some tools to work with any calendar system, and so this post introduces a free dashboard widget for mac users.  Just download from the link at the top of the post and double-click to install.  It’s easy to remove from your dashboard if you don’t like it.  It displays the 7date for today, like this:

PS In case any readers remember my old posts about 7date, I actually changed one small thing.  I used to consider the first day of the year as 1.2011 (e.g.), but I changed it to 0.2011.  The first minute of every hour is 00, the first hour of every day is equivalent to 00, the first second of every minute is 00, so it just made sense.  There are a lot of other benefits of starting at 0, which I won’t go into here.

This idea started with a disagreement.  Last Christmas, a good friend of mine professed his belief in a single universal system of morality, and this felt wrong to me, although I could not articulate a counterargument.  After a little thought, this post is my answer: I present a perspective on morality by which I conclude that there’s something fishy about the idea of a single universal system of morals.

Consider traffic laws.  Cars drive on the right side of the road in the US.  Some intersections have stop lights, some have stop signs, others have yield signs, or are built as on-ramps and off-ramps without any need to stop.  We have a set of rules for dealing with virtually any situation that may arise.  And when something goes wrong, there is often a sense that someone has made a mistake to cause the problem, such as a driver being drunk.

Traffic laws are designed with the global good in mind.  They are designed to be fair, and to help everyone get where they’re going.  Emergency vehicles receive special treatment, since some travel is more important than others.  Cars on a highway receive treatment so that they are never forced to stop while on the highway, unlike smaller roads.  So the rules are designed to maximize a certain function of transportation bliss, and this function takes many factors into account.

These laws have changed throughout history.  They evolve to better serve traffic as it changes.  A small intersection starts with a 2-way stop, and may one day graduate to a 4-way stop, then a stop light, and perhaps one day an overpass with on-ramps and off-ramps.  If many accidents happen at a certain spot, things change to avoid future accidents there — perhaps some warning signs, some yellow flashing lights, or even a change in the physical layout of the road itself.  So traffic laws are built from a history of experience in seeing what works and what doesn’t.

In all of these ways, traffic laws shadow moral laws.  Moral rules tend to be thought of as maximizing a nuanced sense of the global good.  When something goes wrong, we often feel that someone has done something morally wrong.  And we can see that specific (moral) laws have a history – they are created as a reaction to something that has previously gone wrong (to avoid it), or something that has gone right (to encourage it).

Now imagine an ultimate, perfect traffic bible – a single set of traffic laws which, when applied universally, exactly maximizes some absolute sense of transportation bliss.  This idea seems a bit silly to me.  It seems silly for a number of reasons:

1. It does not seem that there really is a single, objective function of transportation bliss to be maximized.  Different people will want different choices to be made.  Should ambulances take precedence over fire trucks or vice versa?  Should mothers in labor going to the hospital get a way to travel more quickly, without having to wait for an ambulance?  It seems that, no matter what choices are written down, there will always be room for reasonable argument.

2. Some laws seem quite arbitrary.  It seems necessary to decide that either all cars travel on the right or on the left side of the road by default.  But who is to say which side is better?  In practice, there does not seem to be a huge difference (between, say, the UK-left and the US-right).  Consistency is important, but we still have a sense that many decisions like this could be made either way, as long as they are made, and everything will be essentially the same either way.

3. Things change.  Once there were no cars on the road.  If the traffic bible existed then, it would have to somehow anticipate cars.  It seems inevitable that the nature of traffic will be so different in 500 years that the traffic laws will also need to change.  It does not make sense that any single set of rules could really anticipate all possible forms of practical transportation.  Even if we do not pretend that this traffic bible will ever be a reality – rather, we think of it as an ideal toward which we aspire – even then, it feels as if only a small fraction of it could ever apply to any one time period.

4. It violates the principle of mediocrity.  And this I personally find to be the single most compelling line of thought: the principle of mediocrity states that there is nothing special about humanity.  The Sun does not rotate around the Earth, we evolved from other animals, and so forth.  This principle can be seen behind many great advances in human understanding.  It is a useful principle precisely because it anticipates the type of mistake we are apt to make: considering ourselves special.  If we thought of traffic as an eternal and omnipresent issue facing all known life in the universe, it feels much easier to embrase the idea of an ultimate traffic bible.  And I do admit that traffic is quite possibly a universal problem, given some flexibility for different contexts.  But I’m guessing the reader will agree that, despite being a common problem, traffic does not feel profound.  It is a mediocre phenomena, and a bible feels out of place.

And this is exactly why I think there’s utility in comparing traffic and morality.  It lets us consider morality without the pomp.  Traffic is one version of morality minus profundity.

So I argue that it makes more sense to see morality from a perspective of mediocrity: we are building a very practical system out of a hodgepodge of past experience.  We have a feeling of universal good, but it is amorphous, and most likely induced by our human tendency to think of ourselves and our actions as special.

None of this is meant to detract from the significance of moral questions.  Indeed, traffic laws are very important in our lives.  Perhaps a bit boring by comparison, but very meaningful.  So we see a way in which a system of laws can be vitally important, have a vague sense of a general good, and be constantly improving — all this without the existence of an ultimate perfection.  Each of the reasons I’ve listed above against a traffic bible can be equally well applied to reasoning about a universal system of morals.  The difficulty is in fighting our own feeling of profundity and superhuman depth we easily attach to moral questions.  If we can see that moral questions are traffic questions in another light, our thinking becomes clear.

Tidiness is nice, but the world is not so tidy; there is no universal system of morals.

Break it up into smaller companies.

Companies are a lot like people.  They have values, direction, vision, work ethics, relationships, desires, and on some level even emotions.  They apologize when they mess up, they brag when they succeed, they occasionally make jokes.  And this is good, because people have evolved to be creative specialists, many of whom make the world a much better place.

But there are some key differences.  One of those is that almost every company gets boring with age.  It’s hard to relate to now, but IBM used to be a very cool company.  Electronic Arts used to be innovative and risky.  Microsoft used to be adroit and unpredictable.  And Google used to be awesome.

A lot of companies are built around one or two people.  Bill Gates made Microsoft, Larry Ellison made Oracle, Steve Jobs made Apple.  As far as these people are truly controlling their companies, they tend to create success.  This is especially clear when we look at how well Microsoft or Apple has done with or without Gates/Jobs — the correlation is clear.

Why do companies get boring?  I can tell you firsthand.  I’ve worked at Microsoft and Google, and I have many friends with a lot of combined work experience at most of these places.

These places get boring because they own more than they use.  Think of all the crap you have at home that you never use.  The pile of unread magazines, the orphaned remote controls, mostly-blank notebooks, the unused gadgets you paid too much for to throw away, the presents from friends you feel guilty getting rid of.  This stuff builds up, and adds constraints to your life.  Everyone does it, so it doesn’t feel like anything’s wrong.  It’s just human nature that this happens.

In software companies, they get security reviews.  They get deeper hierarchies, everyone with veto power.  They get more PM’s and managers.  They get more designers, more hardware, more tools, more buildings, more partnerships, more half-done projects, more directions.  More of everything.  And everything has an expectation pricetag, a cost on the future.  If you’re Google, and you have a mail site and a photo-sharing site and a social infrastructure, and if you want to compete with Instagram/Picplz, you can’t just build an iPhone app.  You have to integrate with your internal social stuff, you have to integrate with gmail and picasa and convince your PM’s and your manager and his/her manager and a few VP’s that this is a good idea.  You have to convince security you know what you’re doing, and follow login protocols.  You have to wait for three other projects to finish, each of which change their specs and deadlines continuously.  The list goes on.

For software companies, age is a blanket of barnacles.  The ones that survive have figured out how to keep moving despite the barnacles.  The best ones figure out how to keep the barnacles to a minimum.

Which brings me back to fixing Google.  I care about Google because the core of the company is pure gold.  They have a brilliant, community-constructive attitude.  They love to make things, and they love to make things in a way that has historically vastly improved our perspective on what is possible with technology.

Yet they are mired in cruft.  They have far more resources than they can cohesively use.

The solution is simple yet daunting.  No one person or company should own all that stuff.  I’m not saying throw it away.  I’m saying, break the company up into sister companies.  A conglomerate of smaller companies with unusually-friendly terms amongst them.  Imagine you and your ten best friends all have separate startups, with full profit sharing and mutual open source.  You don’t have any obligations to your friends to do work for them, but you will naturally look for self-benefiting ways to work with them.

How small should the pieces be?  Here’s an acid test: can one person keep in mind a single cohesive idea that encompasses everything the company does, even at a detailed level?  This person should be able to describe in an hour how every single project in the company fits into the main idea of that company.  If you can do this, then the company has returned to a low-barnacle state.  The obligations are minimized, and the energy can go toward creativity instead of constraint-grappling.

That’s the idea.

-

I want to add a note about why anyone should listen to me.  I’m not a Bill Gates or Steve Jobs.  I haven’t led a smashingly successful startup.  But I have been paying close attention to the software industry for about 20 years, and I’ve been working within it (counting school internships) for close to 15 years.  I am running my own company now, and I’ve been thinking about software entrepreneurship since I was a kid.  I care deeply about the success of the industry as a whole, how it affects the world, how well Google does in particular, and how myself and my friends can make a difference in all of this.  A lot of people care about Google, but most of those either work there, or are basically fans.  I don’t fit cleanly into these categories – my personality is averse to opinions formed out of sentiment.  So I offer a combination of detailed technical knowledge of how these companies work, what they do, who works there, aspirations for the projects at these companies, and a penchant for analytic honesty.  I’m calling it like I see it, and the cruft is killing a lot of the awesome Google can still have.

I recently found out about a simple and fun type of puzzle called a crazy cut, first made famous by Martin Gardner.

The general idea is to take a given shape, and cut it into two pieces with the same shape.  The cut does not have to be a straight line, and the shapes are allowed to be “flipped,” i.e., one shape can be a reflection of the other.  Here’s an example puzzle, with its solution:

Can you find the solutions for each of these four puzzles?

Puzzle number 4 is much harder than the rest.  I’ll post solutions in a few days.

Friends seem to find this puzzle pretty engaging, so I’m thinking about making an iPhone/iPad game with a bunch of these.  Do you guys have any thoughts/ideas on what would make such an app fun?

In this post I’m going to prove that there cannot exist a good notation system for talking about sizes of infinity.

It’s fun to think about big numbers.  And by big numbers, I mean different sizes of infinity.

You can define different sizes of infinity in many ways – there are the cardinal numbers, the more specific ordinal numbers, and then the even more specific (and my personal favorite), surreal numbers.  There are also the hyperreals, but those are not as cool as the surreals.

When you start talking about sizes of infinity, you have your first size of infinity – let’s call it omega – and you get numbers like omega + 1, 2 * omega, omega squared, and so forth.  In the surreals, you can even talk about crazy-sounding things like x = the square root of omega, and this number really does exist (in the surreals), and has nice properties like x * x = omega (we would be offended otherwise!).  The important thing is, I’m not just making up nonsense about some vague idea of sizes of infinity.  These are well-defined mathematical constructs that very smart people have thought about, and nodded sagely in solemn agreement.  For the sake of using something specific, when I say “size of infinity,” or “infinite number,” I’ll mean an infinite ordinal number.

So far so good.  Now for the fun stuff.

A little while ago, I sat down with pencil and paper, and decided to invent a system of notation that would capture the notion of how big different sizes of infinity are.  Having some basic math background I was already aware of Impossible Fact #1:

Impossible Fact #1: No system of notation can express every size of infinity.

Let me be more precise.  By a “system of notation” I mean a way that we can turn a human-writeable string into a fixed ordinal number.  The input string must be from a fixed-size alphabet, since otherwise humans couldn’t practically learn it.  You could suggest analog parameters, such as “the length of this line I just drew,” but then you would be assuming we have infinite-precision measuring capacity, which we don’t.  So a system of notation is any definable function from strings of a fixed alphabet into ordinal numbers.  This is an extraordinarily general definition, and probably includes any practical system any human will ever come up with.

But the first thing to notice (as a mathematician), is that the set of possible outputs from this system of notation is necessarily countable.  And there are an uncountable number of sizes of infinity.  This was one of the first big surprises when set theory was being born – a fact that can be seen in Georg Cantor’s diagonal argument.  So right away, there will be some specific size of infinity that is impossible to write down.  And I don’t mean impossible like some number with a billion kazillion digits is “impossible;” I mean literally, logically impossible.  As in, if you wrote down every number you could possibly write down, you would still be missing a number.

This will not suprise any mathematicians who are reading this, because it is a well-known idea in the math-world.  However, the next fact is something I had to think about a little before understanding.

We know we have to give up on being able to write down every size of infinity – ok.  So let’s compromise.  How about we come up with a system of notation where, for any infinite number x, we can write down some number y with x < y.  That’s a pretty big compromise.  We’re giving up on being able to write down an infinite, arbitrarily-large portion of all numbers.  But, no, infinity has no mercy.

Impossible Fact #2: Any one system of notation will have all its numbers < most numbers.

In other words, no matter what system of notation you invent, there will always be an infinite number y so that x < y for every single number x in the notation.  In fact, it’s even worse than that.  If you take the union of all numbers less than any number in the notation, the set you arrive at will still be smaller (as a set) than the numbers remaining.  In other words, even in this compromised setting, we still are forced to miss so many numbers that our “captured” set looks infinitely small in comparison.

This last point is not completely obvious to see, although the proof from the ZFC axioms (the most commonly-used axioms of set theory) is easy to understand.  Because ZFC works easily with ordinals, I’ll give the proof for ordinals as the sizes of infinity, but surreals are easy to understand as an order-preserving superset of ordinals, so the proof carries over to the surreals as well.

We think of the notation system as a definable function F(x) where x is a natural number (N = natural numbers), and F(x) is an ordinal.

1. By the axiom schema of replacement, there is a set A = {F(x) | x in N}.

Each ordinal number is itself a set, so every element a in A is a set.  Every ordinal o can be defined as the set of all smaller ordinals o = {p : p < o}.  So the union of A is the union of all elements smaller than any number F(x) that we can write in the notation.  We’ll call this union B.

2. By the axiom of union, B = union(A), given by b in B iff b in a for some a in A, is also a set.

Let’s call the rest of the ordinals R.  All ordinals O taken together form a proper class — essentially, they are too big to be considered a set.  (We get paradoxes if we do – see Russell’s paradox.)  As a class, we have R = O – B.

3. B is a set, O is not, so R cannot also be a set.

4. If there were a injective function f:R -> B, then we could use the axiom scheme of replacement (via f inverse) to show that R is a set.  This would be a contradiction, so there is no such f, meaning that R can be thought of as having more elements than B.

We could informally write card(R) > card(B), but some might object since usually we only allow taking card(x) when x is a set.  But we can safely say that R is infinitely larger than B.

QED

In fact, we’ve proven something even stronger than Impossible Fact #2, because even if we pretend that the input for our notation function F(x) is any ordinal, the same proof works to show that we still miss most infinite numbers R.  So, for example, if we allow any real number as an input into our notation, it would still fail.

Alas, there is no system of notation that does what we want.  But at least we know to stop looking.

Final score

Humanity: epsilon (for knowing to give up)

Infinity: omega

This article caught my eye today:

http://www.newsweek.com/2010/08/05/the-limits-of-reason.html

It’s interesting for a number of reasons.  First, the content itself is kind of cool – why are humans irrational?  It’s a good question.  Second, they mention specific examples – including some actual logic analysis – of things people get wrong, and maybe why they get it wrong.  Finally, and perhaps most importantly, it’s interesting for what it doesn’t do.  It doesn’t do a good job of analyzing how the conclusion was reached.

1. Why are humans irrational?

This is a great question, because it is a major contradiction in our day-to-day lives that has a huge impact on how we live and our success as individuals and societies.  And looking at it from the evolutionary perspective is insightful, because it strikes at the core of the contradiction — if our reasoning is flawed, wouldn’t it follow that our faulty conclusions would diminish our survival?

For these reasons, I feel the work being discussed is very relevant and worthwhile.  It is a great question, and a good context within which to ask that question.

The article claims (or rather, cites other work which claims) that we tend to make reasoning mistakes which can help us win arguments.

I’m not sure I really agree with this conclusion.  The main problem is: Wouldn’t it just be better for humans if we were swayed by more logically-correct arguments?

In other words, the article’s conclusion seems to simply defer the contradiction of bad human reasoning to a new problem of bad human argument-listening.  This is still a huge, looming problem that hasn’t even been attempted here.

Here’s another approach we could take to tackling this problem: categorize the types of mistakes humans most often make (I think this has already been worked on), and then look for situations in which the mechanism behind the “faulty” reasoning is actually helpful.  This method – finding good properties of evolved traits – makes much more sense to me.

If you want to understand why something evolved, you must understand how it helps survival.

2. Logic analysis

As a mathematician, I can’t avoid commenting on the logic analysis of the article.  Specifically, the article says:

Consider the syllogism “No C are B; all B are A; therefore some A are not C.” Is it true? Fewer than 10 percent of us figure out that it is, says [Hugo] Mercier.

Actually, this line of reasoning is not generally correct.  To be more precise, if any B exist, it is correct, but if no B exist, then the conclusion may or may not be correct.  If you know what a Venn diagram is, then that’s probably the simplest way to visualize the problem:

The first statement omits anything from being in C ∩ B.  The second omits anything in B outside of A.  The result is the Venn diagram on the right.  Clearly, anything in B must be in A but not C, which would verify the conclusion.  The possible mistake is that B and A outside of C could be empty!

Let me illustrate with an example.  Fermat’s Last Theorem states that there is no integer solution to the equation x^n + y^n = z^n with x,y,z,n > 0 and n > 2.

Let’s plug in some values to the above argument.

Let A = (quadruples (x,y,z,n) of positive integers with n > 2).

Let B = (quadruples (x,y,z,n) of positive integers with n > 2 and x^n + y^n = z^n).

Let C = (quadruples (x,y,z,n) of positive integers with n > 2 and x^n + y^n ≠ z^n).

We can apply the above argument:

First, No C are B.  Yes, that’s true.

Second, all B are A.  Yes, that’s also obviously true.

Therefore, some A are not C.  But anything in A and not C is a counterexample to Fermat’s Last Theorem!  According to the article, I’ve just proven the theorem false.  But of couse I haven’t.  I’ve just pointed out that this line of reasoning can fail.  So the 90% of humans who “fail” to confirm the syllogism aren’t so stupid, after all.

3. The missing analysis

I’m about to criticize this article, but I want to preface the criticism by saying that I’m glad the article exists.  There is a gaping hole in journalism around science.  We need more science journalism because we need a culture that celebrates research and progress.

The best kind of scientific journalism is both appreciative and critical at once – this article is not critical enough.  It doesn’t sufficiently explore the key question of any good investigation: why do the researchers think the new claim is true?  Sadly, this is the norm in mainstream so-called scientific journalism.  The articles are so caught up in trying to decipher the complexities of the new ideas that they forget to ask why we need the complexities in the first place.  Why don’t the simpler answers suffice?

It is a bit ironic that an article decrying the carelessness of human reasoning fails to reason carefully.

The phrase the scientific method implies that there is some universal, automated process that investigators blindly follow in order to do science. In truth, there is a great deal of improvisation and creativity required for the doing of good science. Great leaps forward, such as general relativity or the complex (as in complex numbers) proof of the prime number theorem, often rely on bold, inspired insights into the nature of an unsolved problem.

However, there are a few common principles that unite the rational attitudes of modern research. I want to highlight a few that I feel are somewhat neglected. They are:

• experimental candor,
• easily reproducible experiments, and
• induced correlation.

Experimental candor

Here’s a nice way to get great results: suppose you think that drug A will help people lose weight. Conduct a thousand studies on small groups of test subjects. Suppose one of those studies shows good results – publish those good results, and throw away the rest of the results.

This may sound a bit unrealistic, but something like this can happen much more easily in computer science. In this case, there is a growing field of algorithms which are both probabilistic and approximate – very similar to experimental drugs in medicine. If they do pretty well most of the time, that’s good enough. Yet with an algorithm, it’s incredibly easy to run a million trials of your code, and only publish the best subset of that. Even if the quality of your results are completely random, it’s just a matter of time before one small subset of the test results look good.

Hence the need for experimental candor. It’s important to reveal all the relevant experiments performed, including the negative or inconclusive ones. The web is the perfect platform for this kind of data disclosure – you can pre-publish your intended experiments and hypotheses before you actually run the experiments. This way, good results look better, and other researchers won’t waste time on previously failed experiments. Of course, it’s always possible that an experiment failed for unaccounted-for parameters (including human error), which is why experimental reproducibility is also crucial to good research.

Easily reproducible experiments

This scientific tenet is well agreed upon, but poorly executed. In practice, I know of very few experiments which can be very easily reproduced at the research level. In some cases, one may wish to build upon the work of another, such as by augmenting a biochemical procedure with a new step. Articles involving experimental lab work do indeed contain careful procedural explanations meant just for this purpose, which is great. But in many cases, even this is not enough for other researchers – in my days as a grad student, I would see other grad students emailing or calling other investigators (often ones who were considered serious competitors) to ask for critical clarifications in procedure.

We can do better than that.

I’m going to pick on computer scientists for a moment, because they’re the worst offenders. An algorithmic experiment has the most potential to be easily reproducible. Ironically, it seems typical to leave out necessary parameters to perform the experiments used in many papers. In order to reproduce a certain graph of time complexity versus input size on a certain real-world dataset, for example, a reader will often have to code up the algorithm based on very vague pseudocode and hand-wavy explanations, guess at parameter values, and separately download the dataset. I’ve even seen code used which was nowhere available in either pseudocode or executable code – the reference given was by personal communication with another researcher (who won’t answer my emails).

There is no excuse for this. Any good algorithmic experiment can be reproducible at the click of a button. The experimenters have already written the code – it is simply a matter of adding a link to this code to a website. It would be friendly to add a little documentation; or better yet, to follow a pattern of operation for the field, in much the same way that some software installation procedures have become standardized.

Induced correlation

This point is a call for the conscious recognition of an idea that’s been implicitly used for some time.

Certain experiments have the goal of looking for something like a causal relationship. If a drug company is testing a weight-loss drug, they want to know that their drug causes the weight loss, as opposed to it causing something else, or something else causing the weight loss.

Unfortunately, there’s no fool-proof way to experimentally test causality. This is a well-known problem. It’s also interesting to note that, philosophically, causality itself is subjective in nature, although that is the matter of another post.

Here’s the trouble: Let’s hypothesize that chemical X causes weight gain. As an experiment, get a large group of people together. We randomly select some folks as the control – they won’t change their diets, and we randomly select some others to change their diet to no longer consume chemical X. We see the desired results: the control group gains a little weight on average, but the experimental group (no chemical X) actually loses some.

Does that mean anyone can prevent weight gain by avoiding chemical X? Absolutely not. Here is one possible explanation: Suppose that the vast majority of foods contain both chemicals X and Y together, or not at all. So when the experimental group avoided X, they were also avoiding Y without knowing it. Now you unleash your study on the world, and everyone starts avoiding X. But there are some foods with chemical Y in it, without X. It could happen that those foods become more popular, or that certain people subconsciously crave Y. In either case, we have people consuming Y, not X, and gaining weight.

Is there anything we can do to experimentally show something stronger than mere correlation? A little bit, yes – we can show induced correlation. This is a correlation between parameters which was observed specifically by either turning on or off the cause in each trial, and purposefully leaving all other known parameters the same. Let’s use the term natural correlation to indicate experiments where the cause was either present or absent without any control by the experimenters. Induced correlation gives more evidence of causality than natural correlation since there is more evidence that we can control the effect by controlling the cause.

I think this general idea has been understood already, but I’m not sure that it has been explicitly recognized. My goal throughout this post has been to encourage the codification and emulation of a few good core principles of scientific investigation. There are definitely more key principles, although I’ve been reminded many times that at least these three could use a little more awareness and observation.

It’s interesting to think of DNA as the source code for life. A lot of ideas fall into place nicely with this analogy.

You need some sort of compiler or interpreter; this role is given to RNA. You need a basic set of atomic instructions, and something like labels to certain parts of the code base – pointers into memory. Codons are the instruction set, with start codons helping to act as labels. A central processing unit executes the commands – ribosomes turn the codon sequences into proteins, and the proteins interact to achieve various goals. Chemistry itself is the ultimate processor, but it takes more focused form in the complex interaction of the enzymes produced by the DNA. Some of the proteins act as inhibitors, decreasing the activity of enzymes; others are activators, doing the opposite. These constructed molecules are capable of effecting or halting the production of still other amino acid complexes. The end result is a logically sophisticated dance worthy of the millennia of evolution which produced it.

As I write code on my own, in an experimental fashion, I sometimes don’t worry about the readability of the code. It is in this scenario that the evolution of source code best matches that of DNA. There is a small cost to having extra/old code, yes, but it is far outweighed by the raw functionality created.

Looking at some source which has grown up just a little bit, mostly unsupervised, offers a few suggestions about bits of information that may, at first glance, appear non-functional (aka junk DNA):

• Old functions which are never or rarely ever called

  As code evolves, some functions become less useful, or replaced by newer ones. It would make sense that some codon sequences would become obsolete, and the encoding would remain in the DNA.

• Literal strings and other initialization data

  There might be a bit of initialization data in DNA – information not obviously functional, yet still used. For example, some DNA may only be active for a very short time when an embryo is first developing, or triggered temporarily at certain key development stages. An even more interesting hypothesis is the possibility that some instincts, or primal knowledge, are somehow encoded in DNA, in a manner somewhat different than traditional protein transcription.

• Debug code

  Debug code is useful for figuring out what part of a process has failed. Although there may not be a conscious debugger to check the output, we could still hypothesize that a little extra information about each step in a procedure could give enough information to locate and react to a failure or attack in the system. In this case, the usually non-functional code would be rarely and temporarily activated as a defensive mechanism.