While I think there is value in the high-level discussions of what caused the financial mess and ensuing economic contraction, not enough attention has been given to the specifics. While I agree with those who point the finger at government policies (see previous blog entry), I also agree that the market economy does experience swings. These are not necessarily bad, but smoothing them out a bit is probably good -- making economic coordination easier and reducing the costs of misallocated resources.
I don’t mean heavy-handed government intervention that acts in a way that prevents the circuit breaker from blowing. On the contrary, many of the most important ways of moderating the swings consist of removing and preventing government interventions of the kinds I listed in my previous entry.
More relevant are means of helping us learn more quickly, thereby reducing the magnitude of the problems resulting from failure. Designing institutions and learning processes to learn from “fast failure” through many modest experiments (as well as developing better means of anticipation) seems to be a promising approach. This is really just a practical implementation of pancritical rationalism, and was nicely described in some detail by Stefan H. Thomke in his book Experimentation Matters.
We won’t really make major progress in moderating the business cycle until we can find better ways of reducing the endemic biases in human thinking. We also need to continue improving our understanding of feedback systems and problems resulting from imitative behavior. (Imitation may be why all major mortgage debt rating agencies used the same flawed ratings models for poorly-understood derivatives, though that may have more to do with SEC regulations.)
One factor that no doubt contributed to the problems is the way executive compensation has been incentivizing executives to take on excessive risk in pursuit of short-term gains. That is not inherent in the market system; it's a result of the specific compensation schemes used. Four authors have recently published a working paper suggesting a better compensation scheme. My review of “Dynamic Incentive Accounts” is here.
Sunday, May 31, 2009
Saturday, May 30, 2009
Stress Testing Government Regulations
Given that the economy will only grow more complex in the future, I find it disturbing how so many of us still run to Great God Government for top-down solutions to the intricacies of complex economic systems. I do believe there are many opportunities for improving the functioning of markets in the economy – for turning markets-as-we-find-them into what I call “smart markets” (or designer markets). But we continue to turn too quickly to poorly thought-out regulation to solve problems, often unaware of how previous poorly designed regulations created or contributed to the problems.
There’s been talk recently about “stress testing” the banks to determine their financial strength. It seems odd to me that we don’t say much at all about the need to stress test government regulations and institutions. Two of the tenets of the Proactionary Principle is to take a comprehensive and maximally objective look at proposed actions, policies, regulations, and institutions. Perhaps we need a constitutional amendment to require the stress testing of proposed regulations. They should be carefully tested under widely varying assumptions and scenarios.
Those looking for easy and centralized answers to current financial and economic problems have renewed the ideological attack on free markets… or on anything remotely close to free markets in our very heavily regulated economy. I have long since repudiated the “libertarian” label as inadequate to describe my economic and political views. Even so, I think the best answers to economic matters almost always reside in the smart design and use of markets rather than in direct government intervention. Since I don’t want to be taken to support the latter in the current situation, I feel compelled to list here some of the ways the government has caused or strongly contributed to the financial and economic problems.
I find ludicrous the claim that our problems result from a lack of regulation. The real situation is one of continuing heavy regulation but with decreased effectiveness and ever less accountability. As economist Tyler Cowen put it, “That’s dysfunctional governance, not laissez-faire.” He points out that, just in the regulatory category of finance and banking, inflation-adjusted expenditures have risen 43.5 percent from 1990 to 2008. The Federal Register puts out something like 70,000 pages of new regulations each year.
Between 1980 and 2007, the highest growth rate in regulation was in "homeland security". The second-largest growth rate was in regulation of finance and banking, where spending almost tripled, rising from $725 million to $2.07 billion. (See this for more details.)
Some of the worst things happened in the highly regulated housing and bank mortgage lending sectors, including among the government-sponsored mortgage agencies. Banks are regulated by rules and agencies including the Office of the Comptroller of the Currency, the international Basel accords on capital standards, state authorities, the Federal Reserve the Federal Deposit Insurance Corporation, and particular laws such as the Sarbanes-Oxley Act.
I don’t doubt that problems can arise in financial markets through poor decision making and herd behavior. But that doesn’t mean that regulation is the answer in most cases. In genuinely free markets, or anything close to them, problems will usually reveal themselves before they grow as large as the recent Western financial problems. They only grow monstrous if the government won’t allow the fuse to blow.
Some of the specific problematic regulations and institutions, in my view:
The Federal Reserve, formed by the government, played a central role in the financial crisis with its insistence on keeping interests too low for too long. The government thereby contributed to what economists call moral hazard.
The longstanding mortgage interest deduction encouraged overinvestment in real estate.
Fannie Mae and Freddie Mac were formed by the government and given a legally-enforced monopoly over “conforming loans.” These institutions contributed to the credit crisis by pushing money at borrowers who wouldn’t otherwise have received loans.
The Community Reinvestment Act (CRA) reinforced this problem when the government pressured banks to loan lots more money to people with bad credit. The mortgage market collapsed when many of those people could no longer repay the loans. The CRA, passed in 1977 and strengthen in 1995, compelled banks to extend loans in high-risk areas. If they refused to do so, they would be liable for fines and would find it harder to get approval for mergers and branch expansions.
The federal government added to the subprime problem through a change in regulations by the comptroller of the currency in December 2005. This triggered some mortgage borrowers to default.
In 1975, the SEC created a credit rating cartel by mandating that debt be rated by a Nationally Recognized Statistical Rating Organization (NRSRO). By establishing the NRSRO, the government raised barriers to entry, leaving those in the favored group protected from competition in the ratings business. It also spurred the inflation of debt ratings. How? Before the NRSRO, it was the debt buyers who had to go to the ratings agencies to evaluate what they were buying. After the NRSRO, it was the issuers of debt who sought out the ratings. Naturally they sought out the highest rating possible. [See this.]
Those are just a few of the bad moves originating in government regulations and institutions. I could also point to increased uncertainty created by inconsistent actions, such as the government bailing out AIG but not Lehman. The government even spurred the use of securitized mortgages through federal regulations allowing the banks to hold much smaller loan loss reserves on the condition that they used securitized mortgages.
The point here is not that the market works perfectly. Nor is it that all regulations necessarily make things worse. It is that regulations have unintended consequences and that therefore we should be applying much smarter and more critical thinking to how we design and evaluate them. I believe that the most promising role for regulation is in helping markets work better, that is, in creating smart markets. But the regulations listed above are of a different kind: they attempt to directly force the highly complex system that is the economy to produce outcomes desired by politicians and interest groups in the name of the public interest.
There’s been talk recently about “stress testing” the banks to determine their financial strength. It seems odd to me that we don’t say much at all about the need to stress test government regulations and institutions. Two of the tenets of the Proactionary Principle is to take a comprehensive and maximally objective look at proposed actions, policies, regulations, and institutions. Perhaps we need a constitutional amendment to require the stress testing of proposed regulations. They should be carefully tested under widely varying assumptions and scenarios.
Those looking for easy and centralized answers to current financial and economic problems have renewed the ideological attack on free markets… or on anything remotely close to free markets in our very heavily regulated economy. I have long since repudiated the “libertarian” label as inadequate to describe my economic and political views. Even so, I think the best answers to economic matters almost always reside in the smart design and use of markets rather than in direct government intervention. Since I don’t want to be taken to support the latter in the current situation, I feel compelled to list here some of the ways the government has caused or strongly contributed to the financial and economic problems.
I find ludicrous the claim that our problems result from a lack of regulation. The real situation is one of continuing heavy regulation but with decreased effectiveness and ever less accountability. As economist Tyler Cowen put it, “That’s dysfunctional governance, not laissez-faire.” He points out that, just in the regulatory category of finance and banking, inflation-adjusted expenditures have risen 43.5 percent from 1990 to 2008. The Federal Register puts out something like 70,000 pages of new regulations each year.
Between 1980 and 2007, the highest growth rate in regulation was in "homeland security". The second-largest growth rate was in regulation of finance and banking, where spending almost tripled, rising from $725 million to $2.07 billion. (See this for more details.)
Some of the worst things happened in the highly regulated housing and bank mortgage lending sectors, including among the government-sponsored mortgage agencies. Banks are regulated by rules and agencies including the Office of the Comptroller of the Currency, the international Basel accords on capital standards, state authorities, the Federal Reserve the Federal Deposit Insurance Corporation, and particular laws such as the Sarbanes-Oxley Act.
I don’t doubt that problems can arise in financial markets through poor decision making and herd behavior. But that doesn’t mean that regulation is the answer in most cases. In genuinely free markets, or anything close to them, problems will usually reveal themselves before they grow as large as the recent Western financial problems. They only grow monstrous if the government won’t allow the fuse to blow.
Some of the specific problematic regulations and institutions, in my view:
The Federal Reserve, formed by the government, played a central role in the financial crisis with its insistence on keeping interests too low for too long. The government thereby contributed to what economists call moral hazard.
The longstanding mortgage interest deduction encouraged overinvestment in real estate.
Fannie Mae and Freddie Mac were formed by the government and given a legally-enforced monopoly over “conforming loans.” These institutions contributed to the credit crisis by pushing money at borrowers who wouldn’t otherwise have received loans.
The Community Reinvestment Act (CRA) reinforced this problem when the government pressured banks to loan lots more money to people with bad credit. The mortgage market collapsed when many of those people could no longer repay the loans. The CRA, passed in 1977 and strengthen in 1995, compelled banks to extend loans in high-risk areas. If they refused to do so, they would be liable for fines and would find it harder to get approval for mergers and branch expansions.
The federal government added to the subprime problem through a change in regulations by the comptroller of the currency in December 2005. This triggered some mortgage borrowers to default.
In 1975, the SEC created a credit rating cartel by mandating that debt be rated by a Nationally Recognized Statistical Rating Organization (NRSRO). By establishing the NRSRO, the government raised barriers to entry, leaving those in the favored group protected from competition in the ratings business. It also spurred the inflation of debt ratings. How? Before the NRSRO, it was the debt buyers who had to go to the ratings agencies to evaluate what they were buying. After the NRSRO, it was the issuers of debt who sought out the ratings. Naturally they sought out the highest rating possible. [See this.]
Those are just a few of the bad moves originating in government regulations and institutions. I could also point to increased uncertainty created by inconsistent actions, such as the government bailing out AIG but not Lehman. The government even spurred the use of securitized mortgages through federal regulations allowing the banks to hold much smaller loan loss reserves on the condition that they used securitized mortgages.
The point here is not that the market works perfectly. Nor is it that all regulations necessarily make things worse. It is that regulations have unintended consequences and that therefore we should be applying much smarter and more critical thinking to how we design and evaluate them. I believe that the most promising role for regulation is in helping markets work better, that is, in creating smart markets. But the regulations listed above are of a different kind: they attempt to directly force the highly complex system that is the economy to produce outcomes desired by politicians and interest groups in the name of the public interest.
Monday, May 25, 2009
A Letter to Mother Nature: Amendments to the Human Constitution
It's about ten years since I wrote the following piece (and read at the EXTRO 4: Biotech Futures conference in Berkeley, California). I'm thinking of including it in the book I'm working on (or possibly another book to follow right after that, focused on transhumanism). If you have any feedback on what you think works and what doesn't work as well as it might, I'd like to hear from you.
A Letter to Mother Nature
From Max More
August 1999 (undergoing revision May 2009)
Dear Mother Nature:
Sorry to disturb you, but we humans—your offspring—come to you with some things to say. (Perhaps you could pass this on to Father, since we never seem to see him around.) We want to thank you for the many wonderful qualities you have bestowed on us with your slow but massive, distributed intelligence. You have raised us from simple self-replicating chemicals to trillion-celled mammals. You have given us free rein of the planet. You have given us a life span longer than that of almost any other animal. You have endowed us with a complex brain giving us the capacity for language, reason, foresight, curiosity, and creativity. You have given us the capacity for self-understanding as well as empathy for others.
Mother Nature, truly we are grateful for what you have made us. No doubt you did the best you could. However, with all due respect, we must say that you have in many ways done a poor job with the human constitution. You have made us vulnerable to disease and damage. You compel us to age and die—just as we’re beginning to attain wisdom. You were miserly in the extent to which you gave us awareness of our somatic, cognitive, and emotional processes. You held out on us by giving the sharpest senses to other animals. You made us functional only under narrow environmental conditions. You gave us limited memory, poor impulse control, and tribalistic, xenophobic urges. And, you forgot to give us the operating manual for ourselves!
What you have made us is glorious, yet deeply flawed. You seem to have lost interest in our further evolution some 100,000 years ago. Or perhaps you have been biding your time, waiting for us to take the next step ourselves. Either way, we have reached our childhood’s end.
We have decided that it is time to amend the human constitution.
We do not do this lightly, carelessly, or disrespectfully, but cautiously, intelligently, and in pursuit of excellence. We intend to make you proud of us. Over the coming decades we will pursue a series of changes to our own constitution, initiated with the tools of biotechnology guided by critical and creative thinking. In particular, we declare the following seven amendments to the human constitution:
Amendment No.1: We will no longer tolerate the tyranny of aging and death. Through genetic alterations, cellular manipulations, synthetic organs, and any necessary means, we will endow ourselves with enduring vitality and remove our expiration date. We will each decide for ourselves how long we shall live.
Amendment No.2: We will expand our perceptual range through biotechnological and computational means. We seek to exceed the perceptual abilities of any other creature and to devise novel senses to expand our appreciation and understanding of the world around us.
Amendment No.3: We will improve on our neural organization and capacity, expanding our working memory, and enhancing our intelligence.
Amendment No.4: We will supplement the neocortex with a “metabrain”. This distributed network of sensors, information processors, and intelligence will increase our degree of self-awareness and allow us to modulate our emotions.
Amendment No. 5: We will no longer be slaves to our genes. We will take charge over our genetic programming and achieve mastery over our biological, and neurological processes. We will fix all individual and species defects left over from evolution by natural selection. Not content with that, we will seek complete choice of our bodily form and function, refining and augmenting our physical and intellectual abilities beyond those of any human in history.
Amendment No.6: We will cautiously yet boldly reshape our motivational patterns and emotional responses in ways we, as individuals, deem healthy. We will seek to improve upon typical human emotional excesses, bringing about refined emotions. We will strengthen ourselves so we can let go of unhealthy needs for dogmatic certainty, removing emotional barriers to rational self-correction.
Amendment No.7: We recognize your genius in using carbon-based compounds to develop us. Yet we will not limit our physical, intellectual, or emotional capacities by remaining purely biological organisms. While we pursue mastery of our own biochemistry, we will increasingly integrate our advancing technologies into our selves.
These amendments to our constitution will move us from a human to an transhuman condition as individuals. We believe that individual transhumanizing will also allow us to form relationships, cultures, and polities of unprecedented innovation, richness, freedom, and responsibility.
We reserve the right to make further amendments collectively and individually. Rather than seeking a state of final perfection, we will continue to pursue new forms of excellence according to our own values, and as technology allows.
Your ambitious human offspring.
A Letter to Mother Nature
From Max More
August 1999 (undergoing revision May 2009)
Dear Mother Nature:
Sorry to disturb you, but we humans—your offspring—come to you with some things to say. (Perhaps you could pass this on to Father, since we never seem to see him around.) We want to thank you for the many wonderful qualities you have bestowed on us with your slow but massive, distributed intelligence. You have raised us from simple self-replicating chemicals to trillion-celled mammals. You have given us free rein of the planet. You have given us a life span longer than that of almost any other animal. You have endowed us with a complex brain giving us the capacity for language, reason, foresight, curiosity, and creativity. You have given us the capacity for self-understanding as well as empathy for others.
Mother Nature, truly we are grateful for what you have made us. No doubt you did the best you could. However, with all due respect, we must say that you have in many ways done a poor job with the human constitution. You have made us vulnerable to disease and damage. You compel us to age and die—just as we’re beginning to attain wisdom. You were miserly in the extent to which you gave us awareness of our somatic, cognitive, and emotional processes. You held out on us by giving the sharpest senses to other animals. You made us functional only under narrow environmental conditions. You gave us limited memory, poor impulse control, and tribalistic, xenophobic urges. And, you forgot to give us the operating manual for ourselves!
What you have made us is glorious, yet deeply flawed. You seem to have lost interest in our further evolution some 100,000 years ago. Or perhaps you have been biding your time, waiting for us to take the next step ourselves. Either way, we have reached our childhood’s end.
We have decided that it is time to amend the human constitution.
We do not do this lightly, carelessly, or disrespectfully, but cautiously, intelligently, and in pursuit of excellence. We intend to make you proud of us. Over the coming decades we will pursue a series of changes to our own constitution, initiated with the tools of biotechnology guided by critical and creative thinking. In particular, we declare the following seven amendments to the human constitution:
Amendment No.1: We will no longer tolerate the tyranny of aging and death. Through genetic alterations, cellular manipulations, synthetic organs, and any necessary means, we will endow ourselves with enduring vitality and remove our expiration date. We will each decide for ourselves how long we shall live.
Amendment No.2: We will expand our perceptual range through biotechnological and computational means. We seek to exceed the perceptual abilities of any other creature and to devise novel senses to expand our appreciation and understanding of the world around us.
Amendment No.3: We will improve on our neural organization and capacity, expanding our working memory, and enhancing our intelligence.
Amendment No.4: We will supplement the neocortex with a “metabrain”. This distributed network of sensors, information processors, and intelligence will increase our degree of self-awareness and allow us to modulate our emotions.
Amendment No. 5: We will no longer be slaves to our genes. We will take charge over our genetic programming and achieve mastery over our biological, and neurological processes. We will fix all individual and species defects left over from evolution by natural selection. Not content with that, we will seek complete choice of our bodily form and function, refining and augmenting our physical and intellectual abilities beyond those of any human in history.
Amendment No.6: We will cautiously yet boldly reshape our motivational patterns and emotional responses in ways we, as individuals, deem healthy. We will seek to improve upon typical human emotional excesses, bringing about refined emotions. We will strengthen ourselves so we can let go of unhealthy needs for dogmatic certainty, removing emotional barriers to rational self-correction.
Amendment No.7: We recognize your genius in using carbon-based compounds to develop us. Yet we will not limit our physical, intellectual, or emotional capacities by remaining purely biological organisms. While we pursue mastery of our own biochemistry, we will increasingly integrate our advancing technologies into our selves.
These amendments to our constitution will move us from a human to an transhuman condition as individuals. We believe that individual transhumanizing will also allow us to form relationships, cultures, and polities of unprecedented innovation, richness, freedom, and responsibility.
We reserve the right to make further amendments collectively and individually. Rather than seeking a state of final perfection, we will continue to pursue new forms of excellence according to our own values, and as technology allows.
Your ambitious human offspring.
Oxford University Entrance Exams 1983
Looking through old diaries and memorabilia recently, I came across the exam papers for my Oxford University Entrance Exams. I took these in November 1983, and heard that I was accepted on December 16, 1983. On the exam papers, I marked the questions that I answered. I was interested to see these; perhaps some of you will be too.
GENERAL PAPER
Section I
The passage printed below [from J.S. Mill’s On Liberty] contains nearly 1200 words. Summarise it in not more than 250 words, and answer ONE of the questions following it.
4. Should there be any limits to “absolute freedom of opinion and sentiment on all subjects”?
Section II
10. Is the existence of evil irreconcilable with that of a loving, omnipotent God?
23. When, if ever, is it justifiable to break a law?
ECONOMICS
1. Why is unemployment in Britain so high?
14. Does the effect of a budget deficit depend on whether it is financed by printing money or borrowing from the public?
16. What is meant by “supply side” policies? Would they be effective?
22. Does the “natural monopoly” argument justify the present degree or nationalization in the UK economy?
POLITICS
1. “From each according to his ability, to each according to his means.” Is this a good principle by which to allocate rewards and burdens in society?
3. Is there a sense in which democratic government is “government by consent”?
11. Has the modern Conservative party justified its claim to be the heir to the Liberal tradition?
16. “How far a Prime Minister dominates the cabinet is more a matter of personality than of the institutional resources as his or her disposal.” Do you agree?
GENERAL PAPER
Section I
The passage printed below [from J.S. Mill’s On Liberty] contains nearly 1200 words. Summarise it in not more than 250 words, and answer ONE of the questions following it.
4. Should there be any limits to “absolute freedom of opinion and sentiment on all subjects”?
Section II
10. Is the existence of evil irreconcilable with that of a loving, omnipotent God?
23. When, if ever, is it justifiable to break a law?
ECONOMICS
1. Why is unemployment in Britain so high?
14. Does the effect of a budget deficit depend on whether it is financed by printing money or borrowing from the public?
16. What is meant by “supply side” policies? Would they be effective?
22. Does the “natural monopoly” argument justify the present degree or nationalization in the UK economy?
POLITICS
1. “From each according to his ability, to each according to his means.” Is this a good principle by which to allocate rewards and burdens in society?
3. Is there a sense in which democratic government is “government by consent”?
11. Has the modern Conservative party justified its claim to be the heir to the Liberal tradition?
16. “How far a Prime Minister dominates the cabinet is more a matter of personality than of the institutional resources as his or her disposal.” Do you agree?
Sunday, May 17, 2009
Comics of Transhumanist Interest
You're a transhumanist, or you're fascinated by the possibilities of a future grander than the past. You're really smart. You may be uneasy about asking which comics/graphics novels are worth reading. It's for you that I present: "Everything you always wanted to know about transhumanist comics but were afraid to ask."
Lucifer (volumes 1 to 11 by Mike Carey). An unconventional and engaging take on the Lord of Hell, who resigns that post."
Transhuman, by Jonathan Hickman and J.M. Ringuet. Not great, but worth reading as a very recent work.
Transmetropolitan, by Warren Ellis. A dozen volumes following the gonzo future journalist Spider Jerusalem. The early issues include great treatments of cryonics and uploading.
Squadron Supreme, by Mark Gruenwald. Explores what might happen if a super-powered group (modeled on DC’s Justice League of
Lots of stuff by Warren Ellis, including
--the Planetary series, superbly illustrated by John Cassaday. Each story draws on a classic myth or literary figure or other classic trope.
--Ministry of Space: An alternate-reality Britain goes into space. Superbly illustrated by Chris Weston.
--Ocean: “Sometime in the future, the UN sends weapons inspector Nathan Kane to a space station above Jupiter, where an exploratory team has made an alarming and ominous discovery: beneath the icy exterior of the planet's ocean moon, Europa, are coffins containing members of a sleeping alien race and guns capable of destroying an entire planet. As Kane and the station crew investigate, they are threatened by the sinister representative of a powerful software conglomerate seeking to exploit the discovery for its own purposes.” Among other fun aspects of this story are the “corporate humans” with company-designed personality templates installed for the duration of their contract.
--Orbiter: Dedicated to the astronauts of the ill-fated Columbia on mission STS-107, this is the story of a space shuttle missing for a decade that mysteriously returns… covered in a strange skin, with Martian soil in its landing gear. If you love space and are unhappy at the end of manned space missions, you’ll enjoy this.
--Doktor Sleepless (mad scientist who uses mildly future-tech to disrupt the social order).
--Stormwatch: A cynical, clever modern take on technologically-advanced super-teams who take a ruthless approach to solving problems. Volume 1: A Force of Nature. Volume 2: Lightning Strikes. In Volume 3, Change or Die, a Superman-level, ageless man known as The High leads a team of metahumans determined to radically change the world, not just to solve problems as they arise. This includes using nanotechnology to create the Nevada Garden (where treelike devices grow anything you want) and the abolition of government and war. The second story in this volume involves a conflict with a radical offshoot of an American Cyborg religion, the Church of Gort—cyborg fundamentalists with shared minds. In Volume 4: Final Orbit, Stormwatch fights off the aliens from the Alien movie.
--The Authority. This is a continuation of Stormwatch, with some of the same characters, but with the concepts and settings turned up higher. The team includes the tough, gay couple The Midnighter (an enhanced Batman-like character) and Apollo, as well as The Engineer, whose powers are self-created through nanotechnology. Every super-group has their HQ, but the Authority’s is the most amazing. It’s an alien space-city-spaceship 50 miles long and 35 miles deep and powered by a caged baby universe.
From the introduction by Grant Morrison to the first volume, Relentless: “Because traditional superheroes always put the flag back on top of the White House, don’t they? They always dust the statues and repair the highways and everything ends up just the way it was before… But what “IF”? What if the superheroes decided to make a few changes according to a “higher moral authority”? What if they started to act the way WE might act faced with impossible problems? What if every problem was a solution in disguise? What if WE began to think like superhumans, on a scale we never imagined before?”
Warren Ellis engages in plenty of big concept adventures in his Authority adventures, including an invasion from an alternate universe and the return of Earth’s creator (who wants the planet back, minus all life on it). His writing is beautifully complemented by Bryan Hitch’s penciling, Paul Neary’s inking, and Laura Depuy’s coloring. It could have been a disaster when he left the title, but happily he was replaced by Mark Millar and artist Frank Quitely who set out on their controversial (and occasionally censored run). The transition is in the middle of the volume appropriately titled, Under New Management (starting with #13 in the single issues). Even more so under Millar, The Authority set out to change the world drastically, not just to save it from threats.
Book 3 is Earth Inferno and Other Stories, followed by Book 4: Transfer of Power in which the governments of the world attack The Authority, in fear of losing their own power to abuse their citizens.
--Ultimate Human (the modern “Ultimate” universe versions of Iron Man/Tony Stark and the Hulk/Bruce Banner). Here they represent the triumphs of biotechnology and nanotechnology. The somewhat narcissistic but brilliant technology-inventor Stark gets the better deal; Banner is cursed with an insanely strong and id-driven alter ego whose physiology adapts to any environment.
Paradise
Ultimates 1 and 2. For a thoroughly modern take on super-powered teams.
Ultimate Iron Man (written by Orson Scott Card; the Ultimate line is Marvel’s more recent line (starting in 2000), unencumbered by decades of continuity). This is a technologically-enhanced Tony Stark, whose body is genetically-enhanced before he even puts on his advanced armor.
Ultimate X-Men: Volumes 1 to 6, written by Mark Millar, and 7 and 8 by Bendis. These entertaining issues (the series goes downhill from issue 46/volume 9) has some transhumanistically interesting aspects, including Magneto’s war on homo sapiens. Some quotes: Prof. X: “But there aren’t nicknames, Storm. You’ve just been rebaptized as a post-human being.” “Post-human problems require post-human solutions, Peter. I teach rehabilitation at my school, not revenge.” Seville at the Hellfire Club on an article given him by Prof X.: “Oh, just some stupid article from the London Times. It’s a piece by Stephen Hawking about mutants being man’s last hope against the evolution of artificial intelligence.
Magneto (mutant, anti-human terrorist): “Do not be afraid. Evolution is merely taking place. Just as man replaced ape, so now must you give way to your evolutionary masters… We are not murderers, we are not terrorists and our attacks upon human decadence are far from evil. The Brotherhood of Mutants is simply here to take our place at the top of nature’s food chain. I will keep this message brief because I disliking speaking to you. It feels ridiculous, like conversing with a toad or a common earthworm… you have six calendar months to surrender your world to Homo Sapien Superior. During this time, we will prepare your new society and decide which of your races should be kept as slaves, which should be fuel, and which should be saved for our larder. Magneto has spoken.”
The Surrogates, by Robert Venditti & Brett Weldele <http://www.amazon.com/Surrogates-Graphic-Novels/dp/1891830872/ref=sr_1_1?ie=UTF8&s=books&qid=1265598580&sr=1-1> I found the movie mildly entertaining, but not terribly engaging or intellectually stimulating. The original graphic novel is a little more interesting. I didn’t much like the illustration by Weldele, though it might be more to your taste (too lacking in detail for my liking). The strongest parts, for me, were the fictional ads for surrogate bodies (which seemed to have very much in common with Natasha Vita-More’s earlier “Primo Posthuman”) and related text on the ad campaign. We’re all extremely familiar with the idea of virtual bodies in virtual space.
Surrogates differs from the standard by envisioning a world of physical surrogate bodies that often look like de-aged and enhanced versions of people’s “real” physical bodies. The tone is lightly anti-transhumanist, alas. In reality, transhumanists might like to have such surrogate bodies, but surely they would also prefer to enhance their primary bodies, rather than to leave their sluggish, slobbish physical primaries stacked ungainly in the closet.
On religion and myth:
Garth Ennis’ Preacher series. 9 volumes; deeply offensive to traditional religions. “A tale out of Ireland, dragged through Texas with a bloody hard-on, wrapped in barbed wire and rose thorns. And it’s out to get you.” (From the introduction.)
Lucifer (volumes 1 to 11 by Mike Carey). An unconventional and engaging take on the Lord of Hell, who resigns that post."
Sandman (Neil Gaiman, books 1 to 10)
Other good comics/graphics novels, but not of specifically transhumanist interest:
Animal Man (by Grant Morrison--several volumes; lots of metaphysical fun)
Promethea, by Alan Moore. Delves deeply into magical and Kabbalistic symbols and systems and play with levels of mythic reality. I found the stories more turgid that other
V for Vendetta, by Alan Moore.
League of Extraordinary Gentlemen (Alan Moore). Far more than the later movie,
Irredeemable, written by Mark Waid, illustrated by Peter Krause: This is a new title, starting in April 2009. It’s the utterly convincing and terrifying tale of a Superman-like, extremely powerful superhero who goes bad in reaction to the ungrateful criticisms of those he saves. “Let me tell you the kind of world I live in. It is a world of miserable paramecium who lash out at you in a state of perpetual rage for not solving their problems fast enough.”
Alias (great characters, written by Brian Michael Bendis).
Powers, by Bendis and Oeming (super-powered detective thriller series with excellent dialogue).
Fell, by Warren Ellis. (The first issues just came out in trade paperback). A dark but remarkably fascinating series of detective tales set in Snowtown, featuring the exiled Detective Richard Fell.
Batman: The Dark Knight Returns (Frank Miller -- most people like this more than I do, but it’s a classic)
Daredevil (about ten volumes written by Bendis). A sophisticated take on Matt Murdock/Daredevil.
John Constantine: Hellblazer. Lots of volumes, many by good writers, especially the Garth Ennis issues. For instance, the collection Fear and Loathing (#62-67). The grim runs by Jamie Delano are often excellent. Some good ones by Warren Ellis (Haunted, and Setting Sun) and Mike Carey. From Tainted Love: " You don’t accept death! You don’t humanize it!! And it’s not your frigging friend. “Old man death”! Listen to yourself! …I saw death. It’s not your friend. It’s a twisted, coiled, ugly little length of dogshit and you fight it to your last bloody drop- -- otherwise you’re nothing. Otherwise… why did you even bother in the first place?"
Starman (issues/collections by James Robinson, currently being put out in Omnibus editions)
All Star Superman (Grant Morrison and Frank Quitely). A 12-issue limited series that has lots of fun with the central themes of the Superman mythos.
Ex Machina and The Last Man (both Brian K. Vaughn). The first concerns the Mayor of New York, who can “talk” to machines. The second is about the last man alive in a world of women.
Fables. A nicely-done translation of dozens of classic myths into the modern world.
From Hell (brilliant story of Jack the Ripper by Alan Moore)
Joker (Brian Azzarello & Lee Bermejo). A beautifully-drawn graphic novel focused on Batman’s central nemesis.
Any Punisher collection by Garth Ennis (assuming you enjoy hilarious-but-violent), e.g. Welcome Back Frank.
The Walking Dead, by Robert Kirkman. An engaging apocalyptic zombie series that focuses not on the zombies but on the drama of the few remaining humans struggling to survive.
Red Son (what if Superman as a baby landed in the
Fantastic Four: World’s Greatest (Millar/Hitch).
We3 (Grant Morrison; fantastic illustration by Frank Quitely). About a dog, a cat, and a rabbit, all hugely enhanced physically and cognitively for military purposes, who escape confinement and try to find home.
Doom Patrol (Grant Morrison). A bizarre, highly post-modern super-team with strange characters and entertainingly weird stories.
Amazon is a good place to get collections of these issues at low prices. If Amazon doesn't have an item, I can recommend this source: http://www.mycomicshop.com/
Tuesday, May 20, 2008
Doctor Who: Deontologist or Consequentialist?
First, for those who aren’t familiar with the good Doctor: Doctor Who is a British TV series, originally beginning in 1963 and re-started a few years ago after a long absence. It’s about the adventures of a Time Lord (who is rather fond of Earth and humans, and often takes a human companion on his adventures). Early on, the series established his most persistent adversary and arch-villain as the only other surviving Time Lord, known as The Master.
In my view, the third season of the revived series is the best so far. Even so, there are some episodes I loathe, especially the one in which a (human) scientist finds a way to achieve physical immortality. Of course, he is portrayed as evil and power mad.
The last two episodes of Season Three revolved around a conflict with The Master. I found myself thinking about the actions and inactions of the Doctor in terms of two moral paradigms that have historically been popular among philosophers: the deontological and consequentialist approaches. (I have long preferred a virtue-based ethics, but it’s hard to get very far away from elements of the other two models of morality.)
In case you don’t remember those Moral Philosophy classes you took years ago, here’s an exceedingly brief recap:
Deontological ethics: The right is prior to the good. In other words, actions are only good if they are the right kind of actions. That typically means that good actions are those that conform to some moral rule (such as Kant’s categorical imperative). The outcomes of an action do not (directly) determine its goodness.
http://en.wikipedia.org/wiki/Deontological_ethics
Consequentialist ethics: The good is prior to the right. In other words, the right action is the one that brings about the most good. (In the utilitarian version, that means bringing about the greatest happiness for the greatest number of people, or sentient beings.) For consequentialists, actions must be judged morally good or bad solely according to their outcomes, not according to motives or the character of the person carrying out the act.
http://en.wikipedia.org/wiki/Consequentialist
Doctor Who has always been portrayed as an unyieldingly moral person. In particular, he doesn’t kill, no matter how sensible a move that seems to be in even the direst of circumstances.
[SPOILER WARNING]
In the last two episodes of Season Three, The Master uses advanced alien technology to mesmerize the population, getting himself elected Prime Minister of England, then essentially taking control of the entire world. From this base, he intends to expand outward and conquer other races on other planets. With enormous help from his companion, Martha Jones, the Doctor succeeds in toppling The Master after a year of merciless, murderous, dictatorial rule. Everyone in the vicinity (all of whom have suffered badly) desperately urge the Doctor to kill the master. (Being a Time Lord, the Master can regenerate after being killed. But the Doctor could overcome that ability.)
The Doctor will not kill The Master, even though that might seem an eminently sensible thing to do, to prevent future evils should The Master escape. Instead, the Doctor forgives The Master, and plans to watch over him, imprisoned in a (relatively) secure location. Once again, the Doctor displays an absolute refusal to take (intelligent) life. Given the enormous genius and driven psyche of The Master, it’s entirely possible he will escape and cause enormous misery and destruction, but this seems not to weigh at all in the Doctor’s decision.
This suggests that the Doctor lives by a clearly Kantian or deontological moral code. His actions are limited according to what Robert Nozick termed “side-constraints”. Yet, the Doctor also appears to be something of a consequentialist, in that he travels the universe and time eagerly engaging with trouble and righting wrongs. He’s also clearly a universalist, in that his moral concerns transcend species boundaries. Arguably, this is more a feature of consequentialism that of deontology. Utilitarian consequentialists count the pain and pleasure of all feeling beings equally in their moral calculations. Deontologists such as Kant limit the realm of moral concern to a narrow class of beings, usually those counted as capable of rationality.
The writers of this episode, however, either haven’t taken enough Moral Philosophy classes or else took an easy way out. Consider the Doctor’s great agony at The Master’s last move. The Master chooses to let himself die (after being shot by one of the nearby victims) in order to spite the Doctor. (The Doctor doesn’t want to be alone as the last of his race of Time Lords.) The Doctor’s personal desire not to be the last of his species is not universalist, but strongly favors a species-specific valuation of an immensely evil Time Lord over any number of individuals of other species. Nor is his response consequentialist. No principled consequentialist would put his own feelings above the implications of his actions for potentially billions of other sentient beings. It seems to me that the Doctor’s response in this final episode was a melodramatic one that betrays a quite clear deontic ethics.
The episode I detested, mentioned above, followed a long line of stories by numerous writers in taking exceedingly cheap shots at the goal of physical immortality. What underlies the Doctor’s opposition to immortality? Does it make any sense? He lives a long time himself (perhaps over 900 Earth years so far). What possible deontic objection could he have to a human being wanting to greatly extend his life, and those of others who so chose? Even if the Doctor is a closet consequentialist, merely masquerading as a deontologist, he cites no consequentialist objection.
Among us regular human beings, I strongly suspect much stated opposition to physical immortality (or superlongevity) is essentially rooted in resentment. If I can’t live indefinitely then, dammit, living indefinitely must be a bad thing. Having lived for centuries and apparently having enjoyed most of it, perhaps even Doctor Who lies to himself about this, being unable to face what he takes to be the inevitability of his eventual death.
In my view, the third season of the revived series is the best so far. Even so, there are some episodes I loathe, especially the one in which a (human) scientist finds a way to achieve physical immortality. Of course, he is portrayed as evil and power mad.
The last two episodes of Season Three revolved around a conflict with The Master. I found myself thinking about the actions and inactions of the Doctor in terms of two moral paradigms that have historically been popular among philosophers: the deontological and consequentialist approaches. (I have long preferred a virtue-based ethics, but it’s hard to get very far away from elements of the other two models of morality.)
In case you don’t remember those Moral Philosophy classes you took years ago, here’s an exceedingly brief recap:
Deontological ethics: The right is prior to the good. In other words, actions are only good if they are the right kind of actions. That typically means that good actions are those that conform to some moral rule (such as Kant’s categorical imperative). The outcomes of an action do not (directly) determine its goodness.
http://en.wikipedia.org/wiki/Deontological_ethics
Consequentialist ethics: The good is prior to the right. In other words, the right action is the one that brings about the most good. (In the utilitarian version, that means bringing about the greatest happiness for the greatest number of people, or sentient beings.) For consequentialists, actions must be judged morally good or bad solely according to their outcomes, not according to motives or the character of the person carrying out the act.
http://en.wikipedia.org/wiki/Consequentialist
Doctor Who has always been portrayed as an unyieldingly moral person. In particular, he doesn’t kill, no matter how sensible a move that seems to be in even the direst of circumstances.
[SPOILER WARNING]
In the last two episodes of Season Three, The Master uses advanced alien technology to mesmerize the population, getting himself elected Prime Minister of England, then essentially taking control of the entire world. From this base, he intends to expand outward and conquer other races on other planets. With enormous help from his companion, Martha Jones, the Doctor succeeds in toppling The Master after a year of merciless, murderous, dictatorial rule. Everyone in the vicinity (all of whom have suffered badly) desperately urge the Doctor to kill the master. (Being a Time Lord, the Master can regenerate after being killed. But the Doctor could overcome that ability.)
The Doctor will not kill The Master, even though that might seem an eminently sensible thing to do, to prevent future evils should The Master escape. Instead, the Doctor forgives The Master, and plans to watch over him, imprisoned in a (relatively) secure location. Once again, the Doctor displays an absolute refusal to take (intelligent) life. Given the enormous genius and driven psyche of The Master, it’s entirely possible he will escape and cause enormous misery and destruction, but this seems not to weigh at all in the Doctor’s decision.
This suggests that the Doctor lives by a clearly Kantian or deontological moral code. His actions are limited according to what Robert Nozick termed “side-constraints”. Yet, the Doctor also appears to be something of a consequentialist, in that he travels the universe and time eagerly engaging with trouble and righting wrongs. He’s also clearly a universalist, in that his moral concerns transcend species boundaries. Arguably, this is more a feature of consequentialism that of deontology. Utilitarian consequentialists count the pain and pleasure of all feeling beings equally in their moral calculations. Deontologists such as Kant limit the realm of moral concern to a narrow class of beings, usually those counted as capable of rationality.
The writers of this episode, however, either haven’t taken enough Moral Philosophy classes or else took an easy way out. Consider the Doctor’s great agony at The Master’s last move. The Master chooses to let himself die (after being shot by one of the nearby victims) in order to spite the Doctor. (The Doctor doesn’t want to be alone as the last of his race of Time Lords.) The Doctor’s personal desire not to be the last of his species is not universalist, but strongly favors a species-specific valuation of an immensely evil Time Lord over any number of individuals of other species. Nor is his response consequentialist. No principled consequentialist would put his own feelings above the implications of his actions for potentially billions of other sentient beings. It seems to me that the Doctor’s response in this final episode was a melodramatic one that betrays a quite clear deontic ethics.
The episode I detested, mentioned above, followed a long line of stories by numerous writers in taking exceedingly cheap shots at the goal of physical immortality. What underlies the Doctor’s opposition to immortality? Does it make any sense? He lives a long time himself (perhaps over 900 Earth years so far). What possible deontic objection could he have to a human being wanting to greatly extend his life, and those of others who so chose? Even if the Doctor is a closet consequentialist, merely masquerading as a deontologist, he cites no consequentialist objection.
Among us regular human beings, I strongly suspect much stated opposition to physical immortality (or superlongevity) is essentially rooted in resentment. If I can’t live indefinitely then, dammit, living indefinitely must be a bad thing. Having lived for centuries and apparently having enjoyed most of it, perhaps even Doctor Who lies to himself about this, being unable to face what he takes to be the inevitability of his eventual death.
Friday, March 28, 2008
The Proactionary Principle (March 2008)
The Proactionary Principle emerged out of a critical discussion of the precautionary principle during Extropy Institute’s Vital Progress Summit in 2004. We saw that the precautionary principle is riddled with fatal weaknesses. Not least among these is its strong bias toward the status quo and against the technological progress so vital to the continued survival and well-being of humanity.
Participants in the VP Summit understood that we need to develop and deploy new technologies to feed billions more people over the coming decades, to counter natural threats—from pathogens to environmental changes, and to alleviate human suffering from disease, damage, and the ravages of aging. We recognized the need to formulate an alternative, more sophisticated principle incorporating more extensive and accurate assessment of options while protecting our fundamental responsibility and liberty to experiment and innovate.
With input from some of those at the Summit, I developed the Proactionary Principle to embody the wisdom of structure. The Principle urges all parties to actively take into account all the consequences of an activity—good as well as bad—while apportioning precautionary measures to the real threats we face. And to do all this while appreciating the crucial role played by technological innovation and humanity’s evolving ability to adapt to and remedy any undesirable side-effects.
The exact wording of the Principle matters less than the ideas it embodies. The Principle is an inclusive, structured process for maximizing technological progress for human benefit while heightening awareness of potential side-effects and risks. In its briefest form, it says:
Progress should not bow to fear, but should proceed with eyes wide open.
More flatly stated:
Protect the freedom to innovate and progress while thinking and planning intelligently for collateral effects.
Expanded to make room for some specifics:
Encourage innovation that is bold and proactive; manage innovation for maximum human benefit; think about innovation comprehensively, objectively, and with balance.
We can call this “the” Proactionary Principle so long as we realize that the underlying Principle is less like a sound bite than a set of nested Chinese boxes or Russian babushka dolls. If we pry open the lid of this introductory-level version of the Principle, we will discover ten component principles lying within:
1. Guard the Freedom to Innovate: Our freedom to innovate technologically is valuable to humanity. The burden of proof therefore belongs to those who propose measures to restrict new technologies. All proposed measures should be closely scrutinized.
2. Use Best Objective Methods: Use a decision process that is objective, structured, and explicit. Evaluate risks and generate alternatives and forecasts according to available science, not emotionally shaped perceptions, using the most well validated and effective methods available; use explicit forecasting processes with rigorously structured inputs, and fully disclose the forecasting procedure; reduce biases by selecting disinterested experts, by using the devil’s advocate procedure with judgmental methods, and by using auditing procedures such as review panels.
3. Be Comprehensive: Consider all reasonable alternative actions, including no action. Estimate the opportunities lost by abandoning a technology, and take into account the costs and risks of substituting other credible options. When making these estimates, use systems thinking to carefully consider not only concentrated and immediate effects, but also widely distributed and follow-on effects, as well as the interaction of the factor under consideration with other factors.
4. Embrace Input: Take into account the interests of all potentially affected parties, and keep the process open to input from those parties or their legitimate representatives.
5. Simplify: Use methods that are no more complex than necessary taking into account the other principles.
6. Prioritize and Triage: When choosing among measures to ameliorate unwanted side effects, prioritize decision criteria as follows:
· Give priority to reducing non-lethal threats to human health over threats limited to the environment (within reasonable limits);
· Give priority to reducing immediate threats over remote threats;
· Give priority to addressing known and proven threats to human health and environmental quality over hypothetical risks;
· Prefer the measure with the highest expectation value by giving priority to more certain over less certain threats, to irreversible or persistent impacts over transient impacts, and to proposals that are more likely to be accomplished with the available resources.
7. Apply Measures Proportionally: Consider restrictive measures only if the potential negative impact of an activity has both significant probability and severity. In such cases, if the activity also generates benefits, discount the impacts according to the feasibility of adapting to the adverse effects. If measures to limit technological advance do appear justified, ensure that the extent of those measures is proportionate to the extent of the probable effects, and that the measures are applied as narrowly as possible consistent with being effective.
8. Respect Diversity in Values: Recognize and respect the diversity of values among people, as well as the different weights they place on shared values. Whenever feasible, enable people to make reasonable, informed tradeoffs according to their own values.
9. Treat Symmetrically: Treat technological risks on the same basis as natural risks; avoid underweighting natural risks and overweighting human-technological risks. Fully account for the benefits of technological advances.
10. Revisit and Refresh: Create a trigger to prompt decision makers to revisit the decision, far enough in the future that conditions may have changed significantly, but soon enough to take effective and affordable corrective action.
Participants in the VP Summit understood that we need to develop and deploy new technologies to feed billions more people over the coming decades, to counter natural threats—from pathogens to environmental changes, and to alleviate human suffering from disease, damage, and the ravages of aging. We recognized the need to formulate an alternative, more sophisticated principle incorporating more extensive and accurate assessment of options while protecting our fundamental responsibility and liberty to experiment and innovate.
With input from some of those at the Summit, I developed the Proactionary Principle to embody the wisdom of structure. The Principle urges all parties to actively take into account all the consequences of an activity—good as well as bad—while apportioning precautionary measures to the real threats we face. And to do all this while appreciating the crucial role played by technological innovation and humanity’s evolving ability to adapt to and remedy any undesirable side-effects.
The exact wording of the Principle matters less than the ideas it embodies. The Principle is an inclusive, structured process for maximizing technological progress for human benefit while heightening awareness of potential side-effects and risks. In its briefest form, it says:
Progress should not bow to fear, but should proceed with eyes wide open.
More flatly stated:
Protect the freedom to innovate and progress while thinking and planning intelligently for collateral effects.
Expanded to make room for some specifics:
Encourage innovation that is bold and proactive; manage innovation for maximum human benefit; think about innovation comprehensively, objectively, and with balance.
We can call this “the” Proactionary Principle so long as we realize that the underlying Principle is less like a sound bite than a set of nested Chinese boxes or Russian babushka dolls. If we pry open the lid of this introductory-level version of the Principle, we will discover ten component principles lying within:
1. Guard the Freedom to Innovate: Our freedom to innovate technologically is valuable to humanity. The burden of proof therefore belongs to those who propose measures to restrict new technologies. All proposed measures should be closely scrutinized.
2. Use Best Objective Methods: Use a decision process that is objective, structured, and explicit. Evaluate risks and generate alternatives and forecasts according to available science, not emotionally shaped perceptions, using the most well validated and effective methods available; use explicit forecasting processes with rigorously structured inputs, and fully disclose the forecasting procedure; reduce biases by selecting disinterested experts, by using the devil’s advocate procedure with judgmental methods, and by using auditing procedures such as review panels.
3. Be Comprehensive: Consider all reasonable alternative actions, including no action. Estimate the opportunities lost by abandoning a technology, and take into account the costs and risks of substituting other credible options. When making these estimates, use systems thinking to carefully consider not only concentrated and immediate effects, but also widely distributed and follow-on effects, as well as the interaction of the factor under consideration with other factors.
4. Embrace Input: Take into account the interests of all potentially affected parties, and keep the process open to input from those parties or their legitimate representatives.
5. Simplify: Use methods that are no more complex than necessary taking into account the other principles.
6. Prioritize and Triage: When choosing among measures to ameliorate unwanted side effects, prioritize decision criteria as follows:
· Give priority to reducing non-lethal threats to human health over threats limited to the environment (within reasonable limits);
· Give priority to reducing immediate threats over remote threats;
· Give priority to addressing known and proven threats to human health and environmental quality over hypothetical risks;
· Prefer the measure with the highest expectation value by giving priority to more certain over less certain threats, to irreversible or persistent impacts over transient impacts, and to proposals that are more likely to be accomplished with the available resources.
7. Apply Measures Proportionally: Consider restrictive measures only if the potential negative impact of an activity has both significant probability and severity. In such cases, if the activity also generates benefits, discount the impacts according to the feasibility of adapting to the adverse effects. If measures to limit technological advance do appear justified, ensure that the extent of those measures is proportionate to the extent of the probable effects, and that the measures are applied as narrowly as possible consistent with being effective.
8. Respect Diversity in Values: Recognize and respect the diversity of values among people, as well as the different weights they place on shared values. Whenever feasible, enable people to make reasonable, informed tradeoffs according to their own values.
9. Treat Symmetrically: Treat technological risks on the same basis as natural risks; avoid underweighting natural risks and overweighting human-technological risks. Fully account for the benefits of technological advances.
10. Revisit and Refresh: Create a trigger to prompt decision makers to revisit the decision, far enough in the future that conditions may have changed significantly, but soon enough to take effective and affordable corrective action.
Wednesday, March 26, 2008
Subscribe to:
Posts (Atom)
Posts
