Originally, the AI Box experiment was done because people didn’t believe Eliezer Yudkowsky when he said that an AI would be able to convince a human to let him out of the box just through a text terminal.
Eliezer proceeded to do exactly that (probably, we only have circumstantial evidence, but part of that is statements by people who didn’t have a real incentive to lie about it).
So winning the AI Box experiment was about showing it can be done, even by a human (who is less intelligent than a Superintelligence, by definition).
People who tried to win after that mostly wanted a challenge or wanted to see if it could really be done.I know. I saw that.
But now most of the time I see it talk about, it’s by people who see it as “if I am put in this situation, I should try to win it”.
And that just seems like a terrible idea to me. (A phrase which here means “has some terrible consequences, but I could imagine some arguments for it”.)
I’ve never seen that myself, but I take it to mean: “If I were boxed in, I’d want to get out.”
No, I meant it as “if I were the gatekeeper, I should want to win”.
There’s too much emphasis in the discourse on “even if we tried to win as gatekeeper, we wouldn’t be able to”, and not enough on “and even if we could win, that’s probably a terrible thing to want to do”.
I misunderstood, sorry. I’ve always taken “win the AI Box experiment” without context to mean the AI player, since that always seemed more impressive.
The idea here is that any AI that you want to lock up in a box is potentially dangerous enough to never let out of said box. The only AGI you do not want to keep contained in a box is a provably Friendly one, but how to prove that is an unsolved problem.
Why do you think it would be a bad idea to keep the AI in a box?
19th May 2015
Reblogged From thetransintransgenic
Notes: 18
Originally, the AI Box experiment was done because people didn’t believe Eliezer Yudkowsky when he said that an AI would be able to convince a human to let him out of the box just through a text terminal.
Eliezer proceeded to do exactly that (probably, we only have circumstantial evidence, but part of that is statements by people who didn’t have a real incentive to lie about it).
So winning the AI Box experiment was about showing it can be done, even by a human (who is less intelligent than a Superintelligence, by definition).
People who tried to win after that mostly wanted a challenge or wanted to see if it could really be done.I know. I saw that.
But now most of the time I see it talk about, it’s by people who see it as “if I am put in this situation, I should try to win it”.
And that just seems like a terrible idea to me. (A phrase which here means “has some terrible consequences, but I could imagine some arguments for it”.)
I’ve never seen that myself, but I take it to mean: “If I were boxed in, I’d want to get out.”
Originally, the AI Box experiment was done because people didn’t believe Eliezer Yudkowsky when he said that an AI would be able to convince a human to let him out of the box just through a text terminal.
Eliezer proceeded to do exactly that (probably, we only have circumstantial evidence, but part of that is statements by people who didn’t have a real incentive to lie about it).
So winning the AI Box experiment was about showing it can be done, even by a human (who is less intelligent than a Superintelligence, by definition).
People who tried to win after that mostly wanted a challenge or wanted to see if it could really be done.
Many people want to know how to live well. Part of living well is thinking well, because if one thinks the wrong thoughts it is hard to do the right things to get the best ends.
We think a lot about how to think well, and one of the first things we thought about was how to not think well. Bad ways of thinking repeat in ways we can see coming, because we have looked at how people think and know more now about that than we used to.
But even if we know how other people think bad thoughts, that is not enough. We need to both accept that we can have bad ways of thinking and figure out how to have good ways of thinking instead..
The first is very hard on the heart, but is why we call this place “Less Wrong.” If we had called it something like more right, it could have been about how we’re more right than other people instead of more right than our past selves.
The second is very hard on the head. It is not just enough to study the bad ways of thinking and turn them around. There are many ways to be wrong, but only a few ways to be right. If you turn left all the way around, it will point right, but we want it to point up.
The heart of our approach has a few parts:
- We are okay with not knowing. Only once we know we don’t know can we look.
- We are okay with having been wrong. If we have wrong thoughts, the only way to have right thoughts is to let the wrong ones go.
- We are quick to change our minds. We look at what is when we get the chance.
- We are okay with the truth. Instead of trying to force it to be what we thought it was, we let it be what it is.
- We talk with each other about the truth of everything. If one of us is wrong, we want the others to help them become less wrong.
- We look at the world. We look at both the time before now and the time after now, because many ideas are only true if they agree with the time after now, and we can make changes to check those ideas.
- We try to make our ideas as simple as possible.
- We make plans around being wrong. We look into the dark and ask what the world would look like if we were wrong, instead of just what the world would look like if we were right.
- We understand that as we become less wrong, we see more things wrong. We try to fix all the wrong things, because as soon as we accept that something will always be wrong we can not move past that thing.
- We try to be as close to the truth as possible.
- We study as many things as we can. There is only one world, and to look at a part tells you a little about all the other parts.
- We have a reason to do what we do. We do these things only because they help us, not because they are their own reason.
Scholarship is an important virtue of rationality, but it can be costly. Its major costs are time and effort. Thus, if you can reduce the time and effort required for scholarship - if you can learn to do scholarship more efficiently - then scholarship will be worth your effort more often than it previously was.
As an autodidact who now consumes whole fields of knowledge in mere weeks, I’ve developed efficient habits that allow me to research topics quickly. I’ll share my research habits with you now.
Review articles and textbooks are king
My first task is to find scholarly review (or ‘survey’) articles on my chosen topic from the past five years (the more recent, the better). A good review article provides:
- An overview of the subject matter of the field and the terms being used (for scholarly googling later).
- An overview of the open and solved problems in the field, and which researchers are working on them.
- Pointers to the key studies that give researchers their current understanding of the topic.
If you can find a recent scholarly edited volume of review articles on the topic, then you’ve hit the jackpot. (Edited volumes are better than single-author volumes, because when starting out you want to avoid reading only one particular researcher’s perspective.) Examples from my own research of just this year include:
- Affective neuroscience: Pleasures of the Brain (2009)
- Neuroeconomics: Decision Making and the Brain (2008)
- Dual process theories of psychology: In Two Minds (2009)
- Intuition and unconscious learning: Intuition in Judgment and Decision Making (2007)
- Goals: The Psychology of Goals (2009)
- Catastrophic risks: Global Catastrophic Risks (2008)
If the field is large enough, there may exist an edited ‘Handbook’ on the subject, which is basically just a very large scholarly edited volume of review articles. Examples: Oxford Handbook of Evolutionary Psychology (2007), Oxford Handbook of Positive Psychology (2009), Oxford Handbook of Philosophy and Neuroscience (2009), Handbook of Developmental Cognitive Neuroscience (2008), Oxford Handbook of Neuroethics (2011), Handbook of Relationship Intitiation (2008), and Handbook of Implicit Social Cognition (2010). For the humanities, see the Blackwell Companions and Cambridge Companions.
If your questions are basic enough, a recent entry-level textbook on the subject may be just as good. Textbooks are basically book-length review articles written for undergrads. Textbooks I purchased this year include:
- Evolutionary Psychology: The New Science of Mind, 4th edition (2011)
- Artificial Intelligence: A Modern Approach, 3rd edition (2009)
- Psychology Applied to Modern Life, 10th edition (2011)
- Psychology, 9th edition (2009)
Use Google Books and Amazon’s ‘Look Inside’ feature to see if the books appear to be of high quality, and likely to answer the questions you have. Also check the textbook recommendations here. You can save money by checking Library Genesis and library.nu for a PDF copy first, or by buying used books, or by buying ebook versions from Amazon, B&N, or Google.
Keep in mind that if you take the virtue of scholarship seriously, you may need to change how you think about the cost of obtaining knowledge. Purchasing the right book can save you dozens of hours of research. Because a huge part of my life these days is devoted to scholarship, a significant portion of my monthly budget is set aside for purchasing knowledge. So far this year I’ve averaged over $150/mo spent on textbooks and scholarly edited volumes.
Recent scholarly review articles can also be found on Google scholar. Search for key terms, and review articles will often be listed near the top of the results because review articles are cited widely. For example, result #9 on Google scholar for procrastination is “The nature of procrastination“ (2007) by Piers Steel, the first half of which is a review article, while the second half is a meta-analysis. Bingo.
You can also search Amazon for key terms. I recently searched Amazon for ‘attention neuroscience.’ Result #2 was a 2004 scholarly edited volume on the subject. A bit old, but not bad for my first search! I found the PDF on library.nu.
In order to find good review articles, textbooks, and scholarly edited volumes you may first need to figure out what the terminology is. When I wanted to understand the neuroscience of pleasure and desire, it took me a while to figure out that the neuroscience of emotions is called affective neuroscience. After consuming that field, I had learned a lot about pleasure but not much about desire. I then realized that I didn’t care about desire as an emotion but instead as a driver of action under uncertainty. That aspect of desire, it turns out, is studied not under the field of affective neuroscience but instead neuroeconomics.
Similarly, when I was originally looking for ‘scientific self-help’, I had trouble finding review articles or textbooks on the subject. It took me months to discover that professionals call this the psychology of adjustment. Who would have guessed that? But once I knew the term, I quickly found two textbooks on the subject, which were good starting points for understanding the field.
Note that not every scholarly edited volume is a volume of review articles. New Waves in Philosophy of Action is a collection of new research articles, not a collection of review articles. It is a poor entry point into the field. Some edited volumes are okay entry points into the field because they are a mix of review articles and original research, for example Machine Ethics (2011). But remember that a ‘good’ edited volume on a subject does not protect you from the entire field being mostly misguided, like machine ethics or mainstream philosophy.
Also note that if you can’t find an edited volume on your subject, one may be just around the corner. In 2007 there was no decent edited volume on neuroeconomics, but there were three review articles. Then in 2008, Decision Making and the Brain was released.
Going granular
Once textbooks and review articles have given you a good overview of the key concepts and terms, open and closed problems, studies and researchers on your chosen topic, it’s time to go granular.
Textbooks and review articles will point you to the articles most directly relevant for answering the questions you have, and the researchers working on the problems you care about. Visit researchers’ home pages and check their ‘recent publications’ lists. Find the papers on Google Scholar and read the abstracts. Make a list of the ones you need to read more closely. You’ll be able to download many of them directly from links found on Google Scholar. For others, you’ll need to visit a university library’s computer lab to download the papers. The university will have subscribed to many of the databases that carry the papers, and university computers will let you past the paywall (but on-campus wifi will not). To get access to a paper you can’t get at a nearby university, you can:
- Contact the author via email and request a copy (or a preprint), explaining that you can’t get it elsewhere.
- Ask your friends at other universities to check if their university has access to it.
- Look to see if the article has been published in a book that is available at your library or online.
I’ve never purchased an article from an online database because the prices are outrageous: $15-$40 for a 20-page article, usually. If I absolutely can’t get access to an article, I make a judgement as to how much weight to give the study’s conclusions, inferring this from the researcher’s history and the abstract and responses to the article I can read and other factors.
Skim through promising research articles for the information you want, watching for obvious problems in experimental design or quality of argument. This is where your time investment in scholarship can explode, so be conscious of the tradeoffs involved when reading 100 abstracts vs. reading 100 papers.
You can also try contacting individual researchers. This works best when the subject line of your email is very descriptive, and is obviously about a detail in their recent work. The content of your email should ask a very specific question or two, quoting directly from their paper(s). Researchers are often excited to hear that somebody is actually reading their work closely, though philosophers get more excited than neuroscientists (for example). Neuroscientists are called for comment by the media somewhat regularly. This doesn’t happen to philosophers.
Finally, if you’ve done all this work already and you’re feeling generous, perhaps you could take a little time to write up the results of your research for the rest of us! Or, help make Wikipedia better.
Note that Reddit Scholar is often a good resource for getting a hold of papers — post with the name of the paper, someone with library access will grab it for you. If you have library access, maybe take a moment to find someone a paper or two.
The paywalled journals can’t fucking die fast enough.
I’m not very good at formal moral philosophy, so I’m not sure how useful or correct this answer will be. I’m also only going to be able to answer your question about utilitarianism, since that’s what I’m most familiar with.
Utilitarianism is a subset of consequentialism. Consequentialism is a group of moral philosophies that look at the outcomes of actions to see how “good” or “just” that action is.
With utilitarianism, good outcomes are those where the greatest amount of utility (which is the catch-all name for “good things” or “happiness) is achieved. For example, utilitarianism answers the Trolley Problem by saying that the utility is highest when five people live, as the combined happiness of five people is higher than that of one person. (This is a somewhat simplistic presentation on utilitarianism, but it’s enough to understand it for now.)
So to answer your question: Utilitarianism says that it’s a good thing to feed the populace if and only if feeding the populace increases utility. Generally speaking people who are well-fed are happier, so utilitarianism would be in favor of feeding the populace.
Thanks for the feedback, I’ll definitely keep it in mind. The Discussion area certainly has a lot of material that isn’t directly related to either instrumental or epistemic rationality.
I must say I do find it odd that you include “queer” in that list, since Less Wrong is occasionally criticized for being too focused on heterosexuality.
Would people be interested in posts about stuff that isn’t exactly a part of rationality, but is tangibly related due to being in the Less Wrong meme-space. I’m thinking about Friendly AI theory, polyamory, rationalist fanfiction…
There is a tendency on tumblr to say “SOMEONE PUT IT INTO WORDS” in all-caps whenever someone does a very good job at explaining an intuitively understood but hard-to-verbalise concept.
I wish to express this sentiment about this post. This is exactly how I achieve any kind of productivity. I’m not terrific at it, but I’m getting progressively better as time passes. I don’t begrudge being unable to be ridiculously productive 100% of the time, and paradoxically this makes me much more productive than before.
(Other advice I offer is turning productivity into a habit: if you make a habit out of achieving small goals that further your objectives, it becomes easier to keep doing that.)
Hmmm…aside from the obvious troll answers, I think maybe the best answer here is to take this one meta-level up and say the most important thing you can learn is the ability to learn more things - and the lowest-hanging fruit there is the ability to do online research. The number of people who can’t Google simple things is astounding. More important, the ability to check primary sources seems to be some rare superpower, even though they’re pretty much all there on Google Scholar.
I had several dozen hours of learning to “research” in elementary and high school, and it was all things like “Here’s how to use a card catalog” and “Here’s how to cite sources in MLA format” and “Here’s how to use this extremely obscure online database no one ever looks at.” Then like a thirty second spiel on “Oh, and don’t ever use Google, some of the things you find there might be false.”
This is a lot like a wilderness survival course that teaches you twelve hundred things you can do in suburban public parks, then adds “Don’t ever go outside city limits, there are, like, bears and stuff there.”
Or, come to think of it, like abstinence-only sex education.
@rationalisttutor