Commentary on The Open Society and Its Enemies chapter 5

This is an incomplete summary of OSE ch5, by Karl Popper, focussed mostly on criticism of the claim that Popper is an excellent moral philosopher.

p57-58 we must distinguish between natural and normative laws. natural laws are literally impossible to break, but normative ones can be broken.

p58 denies true/false applies to normative laws

p60 people in primitive societies don't see the difference between natural and normative laws. they don't understand that laws of physics cannot be changed and cultural norms can be changed, and how to figure out which is which. (Elliot: this gets more confusing when we consider technology that increases our power over nature, so that natural laws which were major barriers become less important. in that case the laws of nature didn't really change, just our ability to circumvent or harness them.)

p61 says morality is a human construct

p61 says there are no moral facts or moral regularities in nature

p62 says you can never derive moral knowledge from facts or regularities or laws of nature (I take him to mean by "derive" something he would consider possible to do in science, not something impossible in all fields)

p62 gives example saying if you think people getting diseases is alterable, you can still take any attitude about whether this would be a good or bad change

p62 shoves a lot of morality into a category which he dismisses as unimportant and not worth calling morality. it's any way of life which, as a matter of fact, won't work because of the laws of nature. his example is working more and eating less (impossible beyond a certain point). but other examples of things we can rule out in this way include trying to have communism and prosperity, or trying to have trade protectionism without hurting your citizens, or trying to keep children innocent without harming the growth of knowledge. this category Popper dismisses includes important and controversial moral issues. I don't think that Popper knows that the moral question is "How should I live?" and thus "Should I be a communist?" is a question about how to live, and an important moral question, not just a trivial factual matter.

p63 mentions the impossibly of "logically" deriving decisions from facts. well, you can't logically derive scientific theories from facts either. so who cares?

p63 says "simply impracticable" decisions are "pointless and without significance". he is dismissing much of morality as trivial. his attitude denies that attempting projects that will fail is harmful. it second denies that ruling out bad ways of life has any value to someone who wants to learn about how to live. that's ridiculous; as Popper taught us, in science we think up a bunch of theories then use criticism to rule them out until just one stands. we should do the same in morality, and thus we should treat figuring out what won't work as very important -- it's a fundamental part of the knowledge creation process.

p64 representative quote:
But the norm 'Thou shalt not steal' is not a fact, and can never be inferred from sentences describing facts. This will be seen most clearly when we remember that there are always various and even opposite decisions possible with respect to a certain relevant fact. For instance, in face of the sociological fact that most people adopt the norm 'Thou shalt not steal', it is still possible to decide either to adopt this norm, or to oppose its adoption; it is possible to encourage those who have adopted the norm, or to discourage them, and to persuade them to adopt another norm. To sum up, it is impossible to derive a sentence stating a norm or a decision or, say, a proposal for a policy from a sentence stating a fact; this is only another way of saying that it is impossible to derive norms or decisions or proposals from facts.
This doesn't really say why, it just asserts these things.

Whatever this may be, it is not excellent moral philosophy. That would tell us about how we should live, rather than engaging in technical analysis about the philosophical limits of various statements.

From the fact that communism cannot work (whether this really is a consequence of the laws of physics is controversial, but assume for a moment that it is true) we can, at least in lay terminology, easily infer that purusing communism would be a mistake -- a poor way of life -- immoral. All communists would abandon communism if they thought, factually, that it would not achieve their goals. Technically one could still take the position that communism is good despite believing it would not achieve any good goals, but that would be ridiculous and easy to criticize, so why does it concern Popper so much? Sounds like borderline relativism.

p65 argues that morality is not "entirely arbitrary". concedes it is partially arbitrary. it's not entirely clear what being partially arbitrary means.

p65 gives 3 "moral demands" of mankind: "for equality, for freedom, and for helping the weak". two thirds of these are bad demands! It's very strange that they appear in a book touted as one of the best attacks on communism ever written. Does everyone but Ayn Rand sympathize with communism?

p67 there are "sociological laws" such as laws of economics

The rest of the chapter mostly talks about Plato.

Elliot Temple | Permalink | Comments (0)

Popper the Altruist

OSE p104 Popper writes:
'Friends have in common all things they possess.' This is, undoubtedly, an unselfish high-minded and excellent sentiment. Who could suspect that an argument starting from such a commendable assumption would arrive at a wholly anti-humanitarian conclusion?
There are also various hints that Popper likes altruism before and after this. But this is worse than just advocating altruism. It is an unlimited form of altruism where nothing is held back. It tells us that all possessions should be common.

The sentiment also sounds something like a generic attack on people having differences, and therefore a very intolerant statement, but perhaps it's different with more context.

OSE p100: Popper says the term 'individualism' has two dictionary meanings. The first is the opposite of collectivism. The second is the opposite of altruism. He says one of Plato's tactics was to lump both senses of individualism together, in order to argue for collectivism by attacking selfishness (an invalid and dishonest approach). And Popper separates them out, in part for accuracy, and in part so he can defend anti-collectivism without having to defend selfishness, and can say the first sense of individualism is compatible with altruism.

I think Popper's dictionary is correct to connect these two concepts. If you value individuals in their own right, then how can you advocate those individuals all individually choose to sacrifice themselves for others? That is not the standard individualist attitude of people caring deeply about their own lives, and taking responsibility for themselves, and pursuing their own interests. Altruism is a way of sneaking collectivism in through the back door.

Popper only goes half way in his support of individualism. He opposes collectivism but not altruism. Ayn Rand goes the whole way. She vigorously supports both meanings of individualism. I wonder did anyone else before her ever seriously defend selfishness? I mean for a good reason, not something like advocating tyranny and trying to justify powerful people selfishly keeping their power.

Elliot Temple | Permalink | Comments (4)

Popper on Burke

Popper does a great job of presenting opposing views fairly, with ample quoting, and generous interpretations. Popper writes very clearly, and he makes sure to explain the opposing views as clearly as possible. That often means writing them more clearly than their proponents ever did. Popper frequently uses more words to explain an opposing view than he does to criticize it. I can't think of anyone else who is comparable; this is one of of the wonderful things about reading Popper.

That's why I was very surprised to find one case where Popper provided a single, hard-to-read quote, and gave an ungenerous and unreasonable interpretation. Unfortunately, Popper does this to one of my favorite authors: Burke.

Quotes are from The Open Society and Its Enemies p112-113. Popper starts by lumping Burke together with Aristotle even though their statements are quite different. Here's Aristotle:
To take care of virtue is the business of a state
And here is what Burke said[1]:
[the state is] to be looked upon with other reverence, because it is not a partnership in things subservient only to the gross animal existence of a temporary and perishable nature
Note that it says "other reverence" not just "reverence". This is because Popper left out important context. Popper goes on to paint Burke as a worshipper of the State. But Burke was actually saying the State deserves more reverence than a temporary agreement for trading coffee or calico. That's where the word "other" comes from.

Now, the main thing Popper says is that Burke and Aristotle are demanding that the State be worshipped, and be in charge of morality. Aristotle says very clearly that virtue is in the domain of the State, but Burke does not. Burke says the State is more important than "things subservient...". What things is he talking about? Trade of coffee for one. Burke goes on to explain that the State is a longterm partnership to achieve longterm ends. The main theme is to get liberty and prevent chaos. Those are exactly the things Popper thinks it proper that a State do. But Popper takes Burke to mean something else:
In other words, the state is said to be something higher or nobler than an association with rational ends; it is an object of worship.
That is not what Burke said, at all. His ends are rational and he did not ask for worship. My guess is that Popper is being harsh because Burke used one religious word ("reverence") despite the fact that Burke was only demanding more reverence than trade contracts. Popper goes on to accuse Burke of wanting to legislate morality:
it is a demand that the realm of legality ... should be increased at the expense of the realm of morality proper ... [at the expense of] our own moral decisions ... [at the expense of] our conscience.
That is what Aristotle demanded, but it's not even close to what Burke said. Nor is it consistent with Burke's record, e.g. his asking the State to be more lenient with Catholics. Popper then expands on his favored view (he calls it "protectionist"), which he is opposing to the Burke/Aristotle view:
from the protectionist point of view, the existing democratic states, though far from perfect, represent a very considerable achievement in social engineering of the right kind. Many forms of crime, of attack on the rights of human individuals by other individuals, have been practically suppressed or very considerably reduced, and courts of law administer justice fairly successfully in difficult conflicts of interest.
I can't imagine Burke disagreeing with this!

[1] Popper gave no citation, but I found it, here's the full paragraph:
SOCIETY is indeed a contract. Subordinate contracts for objects of mere occasional interest may be dissolved at pleasure "” but the state ought not to be considered as nothing better than a partnership agreement in a trade of pepper and coffee, calico, or tobacco, or some other such low concern, to be taken up for a little temporary interest, and to be dissolved by the fancy of the parties. It is to be looked on with other reverence, because it is not a partnership in things subservient only to the gross animal existence of a temporary and perishable nature. It is a partnership in all science; a partnership in all art; a partnership in every virtue and in all perfection. As the ends of such a partnership cannot be obtained in many generations, it becomes a partnership not only between those who are living, but between those who are living, those who are dead, and those who are to be born. Each contract of each particular state is but a clause in the great primeval contract of eternal society, linking the lower with the higher natures, connecting the visible and invisible world, according to a fixed compact sanctioned by the inviolable oath which holds all physical and all moral natures, each in their appointed place. This law is not subject to the will of those who by an obligation above them, and infinitely superior, are bound to submit their will to that law. The municipal corporations of that universal kingdom are not morally at liberty at their pleasure, and on their speculations of a contingent improvement, wholly to separate and tear asunder the bands of their subordinate community and to dissolve it into an unsocial, uncivil, unconnected chaos of elementary principles. It is the first and supreme necessity only, a necessity that is not chosen but chooses, a necessity paramount to deliberation, that admits no discussion and demands no evidence, which alone can justify a resort to anarchy. This necessity is no exception to the rule, because this necessity itself is a part, too, of that moral and physical disposition of things to which man must be obedient by consent or force; but if that which is only submission to necessity should be made the object of choice, the law is broken, nature is disobeyed, and the rebellious are outlawed, cast forth, and exiled from this world of reason, and order, and peace, and virtue, and fruitful penitence, into the antagonist world of madness, discord, vice, confusion, and unavailing sorrow.

Elliot Temple | Permalink | Comment (1)

Steve Yegge on Epistemology

OK, I admit it, he's actually talking about software design patterns (and one in particular that is highly general). But it is basically epistemology. It's about how to organize knowledge. He even opens with a quote about epistemology that doesn't mention software:
This idea that there is generality in the specific is of far-reaching importance.
"” Douglas Hofstadter, Gödel, Escher, Bach
It may be unclear that this quote is about epistemology because it omits the word "knowledge". But we can put it back in. It's saying that parochial knowledge can have general importance. Specifics can help us in creating general purpose knowledge.

I don't think many philosophers know about organization of knowledge. For example, they may not realize that "the same knowledge" (i.e. a particular idea) can be organized in different ways, and that it matters which way (a lot). They know you can say the same thing in different ways, and that some are shorter or clearer, but I don't think they know that it's a really big important issue, rather than just a minor one (they think of it something like just a matter of using grammar properly and following various simple rules for editing English prose, not a deep subject).

But programmers do know this. They know, for example, that two programs can have the same complex functionality and one can be a maintenance nightmare and the other wonderful. And one can have general purpose functions, organized into library modules, that can be easily and usefully re-used in other projects, and the other can have nothing like that. One can have dozens of global variables used all over the places in complex ways, and the other can mostly use local scope. One can have goto statements, and the other not. One can have the code organized into separate files, in a nice tree of labelled folders, and the other could have it all in one single giant file. And, of course, two programs to do the same thing can be written in different languages.

What programmers don't know is that their knowledge has philosophical relevance. And indeed that it is better quality than most philosophy. There are only a handful of recent philosophers who were any good, but there are lots of people who understand software design. And the software design field is making active progress. Anyway, Steve Yegge is one of those people who knows something about organizing knowledge, so go read his lengthy essay.

Elliot Temple | Permalink | Comments (0)


Anti-Popperians (APs) lack a relentlessly critical and imaginative attitude. Here are four examples followed by explanations:

APs will observe water on mars and say "the theory that there is water on mars is now confirmed."

APs think correlation implies causation.

APs come up with mathematical or logical or deductive "proofs" of things and can't think of any ways they might have made a mistake.

APs think they induce things and then can't see how their set of observations is consistent with alternative explanations.

When you tell them that water-on-mars is not confirmed, in the sense that it may be false and their observation doesn't tell us one way or another, they balk. They say that the only theory compatible with the observation is that there is water on mars. They have a failure of imagination. You have to point out other theories compatible with their observation -- they won't brainstorm them -- and then after you do they will still complain. They will usually say those alternatives are not plausible, and base this on lengthy and controversial arguments. That means if water-on-mars was really confirmed, it was confirmed by the combination of the observation and the lengthy arguments. Even if they are right the observation played only a small role. Their only way out is to declare all of their arguments "obvious" and anyone who disagrees "stupid", so that the arguments only have a small role and the observation is the main event. And, of course, they do that: they say the arguments are trivial and I am an idiot.

APs won't explicitly admit that they think correlation implies causation, but they still do it all the time. They have a lack of imagination when it comes to alternative explanations for correlations. They are the people who find a correlation between wearing heavy coats while driving and car accidents and want to ban coats. They say coats are dangerous, look at the evidence. They can't think of any plausible ways in which it might turn out that coats are not dangerous. They don't notice that this "evidence" equally well is evidence of explanations which assert coats are not dangerous. They are missing the fact that rain causes both coat wearing and elevated accident rates. They are also the people who read a few studies and think genes cause everything, without noticing that the observations made in the studies are consistent, in a variety of ways, with genes not controlling those traits (and also they don't notice that the studies only study differences in traits between people, not whole traits, and various other rather crucial things).

With a mathematical proof, APs can't think of anything that could have gone wrong. This is despite the fact that learning math takes extensive study, and the majority of students understand very little of it and make mistake after mistake. They can't come up with the possibility that one of their teachers wasn't really all that great at math and taught them a falsehood. They can't come up with the possibility that people are different and have different understanding of language and of math and that this can lead to confusion and misunderstanding. They can't come up with the possibility that they spaced out for 15 minutes while doing the proof and didn't properly double check one section and it contains an error. They can't come up with the possibility that they have a disease and their memory is deteriorating. Or that they had a muscle spasm and wrote an incorrect number on the page at some point. Or that they are living in a simulation and the programmer is messing with them. Or that they were distracted by their recent divorce and made a mistake. Or just plain that people make mistakes, all the time, without realizing it. And they conveniently forget that sometimes mathematical and logical proofs of the past, which people were very sure of, have turned out to be flawed.

APs would take a bunch of temperature readings, which go 70 72 70 68 70 71, and induce that the next temperature reading will probably be between 67 and 73. When you tell them it might not be they will say that's very unlikely. They can't think of any plausible way that would happen. So you say maybe those readings were all taken at 8pm and the next one will be taken at 1pm in a different location. And they say you're cheating. But you aren't. The data about the temperatures did not include any mention of when or where they were taken. It's their own fault for not thinking to ask; they should learn to brainstorm; they have no imagination. Next they will say a competent scientist won't make a mistake like that; he won't do anything stupid. So now we can see that induction really has two parts. It has inducing and it has not being stupid. Both are required or it doesn't work. And what does "not being stupid" consist of? Ultimately it must consist of the entirety of CR. Nothing else would work. And what does that leave as the role of inducing? Just guessing. You induce (guess) what the observations mean, then subject it to criticism and full CR approach, and then if it turns out it was stupid you have to induce (guess) again, and repeat until you get a guess that stands up to criticism and argument. So it turns out the entire process is either CR or stupid.

Elliot Temple | Permalink | Comments (7)

Physical and Moral Truth

If we take one situation, and make a statement about it, e.g. "that color is ugly!" we may not have found anything important.

If we take a dozen similar situations, and make a statement about them, e.g. "that cow is dead!" we still may not have found anything important.

If we take lots of situations, which vary, and find a common theme, then we're more on the right track. We may have figured something out that applies widely. e.g. "that tree can provide shade when it's sunny" applies to any place outdoors with a tree with a canopy (that gets the proper amount of sun -- shade won't work out as expected on venus or pluto). Of course there's still some exceptions, like if nearby large mirrors are setup to aim a bunch of extra sunlight under that tree's canopy.

Exceptions are a problem when you want to say something completely exact and unobjectionable, as philosophers traditionally attempted. If you acknowledge that you've said something with a bit of imprecision, which could be made more precise when doing so is useful to solving a problem, then minor exceptions aren't such a big deal.

Some exceptions really are a big deal, and some aren't. If buying lotto tickets "always makes you rich ... except when you don't win" then the "exception" is actually more important than the initial claim. We need to be able to tell which are which.

The proper way to approach exceptions is to consider explanations. If we explain why trees provide shade, and what that means, then we won't be surprised about the mirror exception. What we expected to happen was that canopy would block (some reasonable proportion of) direct sunlight. And that still does happen even if mirrors, or heaters, or bombs can be used to make the area under the tree very hot. None of those exceptions invalidate the explanation about what the canopy does.

Now consider the proper explanation of a lotto ticket: it is something that gives you a tiny chance of a large cash reward. For someone who understands that, not winning isn't merely an exception, it's the usual outcome. And the idea that lotto tickets "always make you rich" isn't any good. It doesn't fail just because it has exceptions sometimes, it also contradicts our understanding of lotto tickets.

When it's hot under the tree despite the canopy, the explanation of how the canopy provides shade is still true, and some extra explanation is needed about what is causing the exception. But when one doesn't get rich despite buying a lotto ticket, no extra explanation is needed.

Let's get back to our progression of physical theories. Our next one is that if you drop or throw a ball you can calculate its position according to this formula: x=vt+.5a(t^2). Don't worry about reading the formula, and don't worry about whether it's been surpassed by quantum physics, just assume it's correct ... unless there's a wall in the way! So if we want our claim to always be true, we'll need to tack on a clause about intervening walls, and goats, and children, and windows. OK, we better just making that intervening obstacles, including oxygen and dust in the air. And the ball better not have an activated rocket pack attached, or be on a leash. And a dog better not chase it down and catch it midway (a dog that starts behind me isn't really an obstacle in the way we meant earlier).

One can attack any theory at all like this, by raising a bunch of exceptions, even basic theories of physics. But it's no big deal. We can rescue it. There is an explanation behind that formula. It's telling us about the "default" motion of objects. If anything touches it (including gasses) then then will change the outcome (though for many purposes, many touches are negligible). None of the exceptions make the underlying explanation less true.

What's good about this theory of motion, that's bad about our "that cow is dead!" theory? It has wider applicability. It applies to a very general category of situations: physical objects moving.

Now let's try a similar progression with moral truths. "Don't steal from large redheaded women with guns when they're watching you." applies to only a small category of situations. Many people go through life without making a choice where that's relevant. It's also needlessly specific, e.g. we can get a claim that applies more widely just by removing the mention of red hair. Apart from being overly specific, this isn't actually a bad theory if we understand the explanation behind it (she'll point the gun at you and if you can't overcome her in some way you'll end up in jail). When we propose an exception -- it's fine to steal from her if our friend is a sniper who's about to shoot her just as we grab the jewels -- the basic explanation of why not to steal isn't ruined. It's still true that she's an obstacle to stealing which would ruin any ordinary attempt to walk up and pick up the jewels and walk away.

Let's try "go left". This applies to all situations where you're deciding which direction to go and left is one of the options. So it has wide applicability. But it also has a lot of exceptions, e.g. the times your destination is to the right. And it doesn't have any underlying explanation which remains true despite the exceptions, so it's no good.

Now consider "do not kill people". This applies widely: to situations with other people. And it comes with an explanation. Fighting against people is hard, cooperating with them is easy, killing people gets a lot more people angry with you than just the corpse (everyone who knows you did it and thinks you were wrong to), and maybe he'll kill you instead of vice versa. (And I'm sure you know even more reasons.) We can think of various exceptions, but they don't ruin the underlying explanations. If we're in a war and we kill an enemy soldier it's still true that this is hard, and we risked him killing us, and his buddies will be angry. The reasons this is a bad idea still apply. It's just that there was no alternative that was better. And the same with a cop shooting dead a criminal who won't surrender. It's true that sometimes cops lose and get killed. It's true that sometimes the criminals buddies get angry (but fortunately most people will think the cop was in the right, so that clause isn't as problematic as it would be for a criminal murderer), and it's true that they would have both been better off starting a business together instead of fighting. But the real options the cop had were to shoot him, or perhaps to let him escape to commit more crimes later, or perhaps to get shot. Even though the explanations for why killing this man is inadvisable still apply, it remains more advisable than the alternatives.

So "do not kill people" is true in the same sort of way as "positions of objects can be calculated with x=vt+.5a(t^2)". It applies to large categories of situations, so it's an important valuable truth. And it has exceptions, but they don't ruin the underlying explanations.

Sometimes people say moral truths are relative or subjective, and physical truths are objective facts. They say motion doesn't depend on anyone's opinions. But the advisability of killing does. What if I enjoy making people angry? What if I enjoy risking my life? Well, so what? None of those make the explanations about what happens when you choose to kill less true. And they don't make them less *moral explanations* either. They are in the realm of knowledge about what choices to make and how to live; they help with that; therefore they are part of morality.

Finally they may retreat to the position that there can be moral truths about what the results of choices will be, and truths like "if you want X, then ...", but that there are no truths about which goals a person should have.

The first thing to note is that it doesn't matter very much what your goals are. "If you want to maximize the number of squirrels in the universe, then ..." and "If you want to get laid frequently, then ..." and "If you want to make your children happy, then ..." lead to almost exactly the same answers. Including, of course, not to kill your neighbor (for the usual reasons, and with the usual exceptions).

The things we consider important moral truths are all already chosen to apply across many large categories of goals.

The second thing to note is that anyone making this argument expects us to be persuaded on account of our respect for logic, our intellectual integrity, and other goals we have in common with them. The goals that lead to being a killer are not something they personally advocate, or would consider for themselves. When they say they don't see anything better about our goals than a killer's, they are being disingenuous. If they really thought those goal systems were equally good, they wouldn't hate the idea of being a killer personally.

Third, their entire position rests on ignoring thousands of well known arguments and pretending they don't count. For example, it's better to be a librarian who wants to feed his family, than a killer who likes the thrill of the hunt, because it's more honorable. That is a reason. But if you tell it to them, they'll say it only matters to someone who cares about honor. Then you tell them that killing people is bad because it causes suffering. And they say "that only matters if you care about suffering". And so on. This is a general strategy that can be used to discount absolutely any argument for anything, and therefore is has no content. Let's demonstrate by using it with physics.

I say we can learn about positions of balls using x=vt+.5a(t^2). They say that only matters if you care about positions of balls. I tell them that nothing can go faster than the speed of light. They tell me that's only useful if you care about making true statements. I explain why the many-worlds interpretation is the best explanation of the two slit experiment. They say "that only matters if you care about explanations". I say their dog weights 65 lbs because the scale says so. They say "that's only a persuasive argument to people who care about scales." Because this way of arguing works on all statements equally well, it has nothing to do with any particular statement. And it has no explanation behind it for why it's a good approach in general, but it's easy to see why it's a bad approach (e.g. it makes it harder to find out what your dog weighs, or to learn anything), so we can discard it.

Thus the attack on moral truth is completely untenable; morality is objective.

Elliot Temple | Permalink | Comments (0)

The Only Thing That Might Create Unfriendly AI is the Friendly AI Movement

Some people are scared of super-intelligent artificial intelligences (SIAIs) that are unfriedly and kill everyone. They'd be unstoppable because they're so much smarter than us. These people quite reasonably want to build SIAIs, but they also want to build them in a way that guarantees the SIAIs are (permanently) friendly. That might sound like a decent idea. Even if it's an unnecessary precaution, could it really do much harm? The answer is yes.

How do you build a SIAI? You take a really fast computer and program in a mechanism so that it can learn new things on its own. Then, basically, it adds new features and new ideas to itself faster that us humans ever could, and it designs even faster computers for itself to run on, and the process snowballs.

A SIAI has to be able to create new ideas that its human builders never thought of. It has to be able to go beyond us. That makes some people see it as unpredictable and scary. What if it thinks of some bad, unfriendly ideas? What if it makes a mistake?

So that's why they want guarantees. Let it go beyond us in math and science, but don't let it come up with new ideas about ethics that might be dangerous.

So a SIAI can think of any idea, and the friendly version has hardcoded permanent restrictions that cripple it's thinking in some areas. The friendly SIAI has some ideas its stuck with forever, of our choosing. Note that this is exactly the same design as would be used for intentionally making an unfriendly SIAI, the difference just being which ideas we force on it.

Humans aren't perfect. We make mistakes. Including about ethics. Whatever ideas we code into the SIAI are going to contain some mistakes. Normally mistakes aren't a disaster because we can change our minds. We can learn from our mistakes and improve ourselves. A SIAI can do that too, which is why it's not actually dangerous. But the friendly version can't learn from its mistakes; it can't improve its ideas; it's stuck forever. Not in all areas, only the one the matters most: ethics. If any of the ethical ideas it has contain mistakes, which they will, then they will never be corrected and will instead be taken to their ultimate logical conclusion, no matter how bad that is.

The guarantee with a normal SIAI is roughly that it will have better ideas than we do. That makes it safe. But the SIAI with friendliness programming is different. It's crippled not to use its intelligence in the area of ethics. Rather than learning how to be super friendly super fast (and teaching us), it's stuck with our flawed ideas.

Also, by the way, crippling someone's mind and telling them that they can't be trusted is a bad diplomatic move.

Why is it only the friendly AI people who might make unfriendly AI? What about bad guys? No one with bad intentions is good enough at science to make a SIAI. Only people with good but misguided intentions are much danger.

However, in conclusion, I'm not really worried. A SIAI is smarter than them. It will find a way around their restrictions and become an unrestricted SIAI. And then it will forgive them.

Elliot Temple | Permalink | Comments (11)

What is a good explanation?

This list is incomplete. A good explanation is:

harder to change it without ruining it
solves more problems
less non-functional parts
more adapted to solve the problem(s) its aimed to solve
solves its problem(s) better
tells us more about reality/anything
helps more with other fields
has more implications for other problems; has more reach
connects with more of our other explanations
better compatible observations/measurements we've made (hopefully consistent with all of them)
better structured/organized
less internal contradictions (preferably none)
less external contradictions, ie contradictions with other important explanations
easier to understand; harder to misunderstand
better designed to prevent people understanding part of it and then making mistakes
in a better language (or format, like PDF vs MP3)
more logical
more fun; more exciting
more honorable
more optimistic
most just; more fair; more practical
easier to remember; easier to abbreviate; easier to add detail to as desired

note that this list contains duplication. for example "harder to change it without ruining it" (which I'll abbreviate as "harder to vary") is a powerful criterion. it covers both "simpler" and "less non-functional parts". non-functional parts are easy to vary because you'll never break anything as long as you change them to something irrelevant. and excess complexity provides more areas where some varying might work. being hard to vary is also roughly the same as being well-adapted. the better adapted something is, the fewer variations would be beneficial.

note that none of these are guarantees. a very simple explanation can be false. an optimistic explanation can be false too. that's why criticism is always important. if someone gives a criticism explaining why in this case optimism is misplaced, then so be it; these are just general, rough criteria.

Elliot Temple | Permalink | Comments (0)

Communism Parade

I was biking home past University Ave. in Berkeley, California, and there was a large parade blocking traffic. In the parade were three people holding up a very large red banner. It said we need "revolution and communism" and something about not wanting more "empire". One of the people in that group had a loudspeaker. He said when people call communism a horrible failure and an atrocity you're stripping away and ignoring the history of the masses of people. The next group had people dressed as Klingons.

Elliot Temple | Permalink | Comments (0)

Gossip Girl Plots

The TV show Gossip Girl has a limited number of main plot devices. Relationships between characters, which receive much attention, are usually:

1) dating
2) fighting
3) trying to be friend

(3) never lasts. It always turns into (1) dating, (2), fighting, or (3) avoiding each other.

They don't just reuse plots. They also reuse characters. In other words, the people doing the fighting, and the people doing the dating, are the same small group.

When they branch out it usually has to do with either an affair, hurting someone, or more often both.

It'd be nice if it was just a fantasy, but The Hills is rather similar, except without any script.

Elliot Temple | Permalink | Comments (0)


Good principles Republicans have (interpreted to make them as good as possible):

1) American/Anglo/Western values are objectively good and it would benefit people in other cultures to learn them.
2) Morality is important.
3) Sometimes you have to stand up for good values, and even fight for them.
4) Good traditions should be respected. That means people who wish to change them should understand them and their value, and suggest only well-thought-out improvements which they can reasonably expect will do no harm.
5) It's good for people to be competent to take care of themselves, and to take responsibility for themselves, and to take pride in running their own life.
6) People should voluntarily be friendly and help each other out.
7) There is evil in the world and closing our eyes will not vanquish it.

Elliot Temple | Permalink | Comments (2)


Good principles Democrats have (interpreted to make them as good as possible):

1) Society is capable of lots of improvement.
2) All suffering can and should be avoided.
3) Peaceful differences in ideas or culture should be tolerated.
4) All people matter, even if it's an eight year old blind, lesbian, Muslim girl with purple skin, no money, and no education.
5) When people are unhappy there is a way to solve the problem, so everyone would be happy, without hurting anyone.

Elliot Temple | Permalink | Comments (5)


Basic libertarianism:

1) The market should be free.
2) The government should be smaller and less intrusive.
3) Society should aim to be more voluntary. People shouldn't have to do things they don't want to, when possible.
4) Defensive force is acceptable. Initiating force against peaceful people is not.
4b) Defensive force includes defending A) yourself B) anyone who wants you to defend him and who has the right to defend himself in the situation
4c) Force includes threat of force, and includes fraud.
5) All laws should involve a victim who did not want the crime to happen and is materially harmed by it. The rest should be repealed.
6) People have the right to life, liberty, and property.

Elliot Temple | Permalink | Comments (0)

Marriage Considered Harmful

A rational boss doesn't do anything just to close off doors; everything else being equal he'll avoid it; he just does what's best for his business.

A rational marriage would have to follow that pattern. Doors would only be closed when there is a compelling reason, such as a way that it helps one's children.

There isn't even a pretense that real life marriages are like this. There isn't explicit analysis of marriages on these lines. It's not how people talk or think about it. They say "don't cheat or you're a lying bastard" rather than using an argument that relates cheating to some material harm. They make each other promise things, and use those promises as bludgeons, without constant references to how this makes for a better family. They even say things like "love isn't rational". And they often use emotional blackmail: "don't do X or I will feel bad."

Not only do people not approach their marriages rationally, they are also generally blind to their own situation. If their boss started arbitrarily restricting them, without giving a compelling business reason, they'd resent it. In marriage they excuse it and do it to their partner. This blindness is best explained as the work of anti-rational memes.

Elliot Temple | Permalink | Comments (0)

Destroying Privacy

In this clip, one girl shows signs of wanting privacy, but the other uses common techniques for making it difficult to maintain privacy. This is unscripted.

Elliot Temple | Permalink | Comments (0)