Superintelligence: Paths, Dangers, Strategies

“Superintelligence: Paths, Dangers, Strategies" by Nick Bostrom is an intellectually stimulating and thought-provoking book that delves into the future of artificial intelligence (AI) and its potential impact on humanity. Bostrom, a philosopher at Oxford University, presents a meticulously researched and well-argued case for the possibilities and risks that superintelligent entities might bring. The book stands out for its rigorous analysis and balanced perspective. Bostrom carefully navigates the reader through various scenarios where AI surpasses human intelligence, discussing both the transformative benefits and the existential risks. His writing style is scholarly yet accessible, making complex ideas about AI ethics, future forecasting, and strategic planning understandable to a broad audience. One of the most compelling aspects of the book is its exploration of the 'control problem' - how humans could control entities far smarter than themselves. Bostrom does not shy away from the challenging philosophical and technical issues this problem presents. He also emphasizes the importance of preparatory work in AI safety research, encouraging proactive measures rather than reactive. However, some readers might find the level of detail and theoretical nature of the discussions somewhat daunting. The book demands attentiveness and a willingness to engage with deeply philosophical and technical content. Additionally, while Bostrom presents a wide array of possibilities, the book sometimes leans more towards speculative thought than practical solutions. "Superintelligence: Paths, Dangers, Strategies" is a seminal work in the field of AI and an essential read for anyone interested in the future of technology and its implications for humanity. Bostrom's thorough approach offers valuable insights and raises critical questions that will shape the ongoing conversation about AI and our future.

This book makes some very important points about the existential risks superintelligence would pose to humanity without going off the deep end with unrealistic doomsaying or conspiracy theories. The reasoning is logical and reasonably well laid out. The book is about more than just artificial superintelligence (although that's the main topic) and discusses a variety of ways other than AI that superintelligence might be achieved. It also goes into how to mitigate the threat superintelligence poses in some detail. The main flaw in the book is that occasionally the writing itself isn't that great, either being poorly structured or awkwardly worded... it would have benefited from one more editorial scrub to smooth out the prose. Still, well worth reading!

I picked this book up because I have a kid at CalTech majoring in AI programming, machine learning. He seems to see only upside, no real concerns about 99% of the population being put put of work, and what I believe is inadequate apprehension about what could go wrong. Mom is a huge fan of Stephen Hawking and he was more than a bit apprehensive about the potential problems with self-learning machines. Most of the books and articles I have read on the topic are cursory or naieve. Nick Bostrom's book is fairly comprehensive and in depth. I am enjoying it as much as an excellent read in philosophy of science., as I am for his expanding the boundaries of the conversation, indeed, broaching it in many areas. I honestly do not know whether he says everything which needs to be said, but he has clearly thought it through and done a good deal of exploring, consulting, conversing ,collaborating. It is far and away the best book I have read on the topic {though there are some good pieces in MIT Technology Review as well). This is a book which is important and timely. We must seriously consider and weigh the potential for harm as well as good before creating a monster. While there may be areas which he has missed, I feel that when I read about a brute force approach to building human level AI by recreating a brain at the quantum level using Schrodinger's equation, the man is clearly pushing the boundaries. If nothing else it is a very good start to an important conversation. I picked this up because I was considering sending a copy to my son, but read it first because he is a busy guy and chooses his side reading carefully. There are books and articles I might mention or even recommend, and others I tell him not to waste his time on, this is one I will be sending him {though I would be very very surprised if someone at Cal Tech did not broach ...all of what is contained here). I will let him determine if it is redundant. It is well written and thorough, and also very approachable. He says in the prologue that overly technical sections may be skipped without sacrificing any meaning. I have not encountered one I needed to skip, and have, in fact, very much enjoyed the level of discourse. Read it if you are in the field to make sure you are covering all the bases. Read it if you are a scientist, philosopher, engineer to enjoy some very good writing. Read it if you are just encountering AI and want to quickly get to speed on the issues. It is not only a book I would recommend, but have, to anyone who would listen ;)

Rating: ⭐⭐⭐ At first glance, Superintelligence promises an enthralling exploration of the looming dangers and potential strategies surrounding the rise of sentient technology. However, beneath its scholarly veneer lies a dense tome that may leave many readers struggling to stay engaged. Bostrom’s target audience seems to be the techno-savvy crowd, but even they might find themselves drowning in the sea of scholarship presented here. 🌊 The book's writing style, unfortunately, lacks the flair to captivate the casual reader, making it a tedious journey through complex ideas. While the theme of the book is undoubtedly compelling, delving into the risks posed by superintelligence, Bostrom's delivery often feels disconnected from the urgency of the subject matter. 💡 Despite his warnings about the existential threats posed by sentient machines, the narrative fails to evoke the necessary sense of immediacy. The author meticulously outlines the potential pathways to superintelligence and the accompanying risks, but his arguments sometimes feel like a meandering stroll through philosophical musings rather than a focused analysis. 🚶‍♂️💭 Moreover, Bostrom's exploration of ethical dilemmas and potential solutions lacks the depth needed to truly grapple with the complex issues at hand. While he advocates for a cautious approach to technological advancement, his proposed solutions feel more like broad strokes than concrete strategies. 🤔 Despite these shortcomings, Superintelligence does offer some valuable insights, particularly for science fiction aficionados and those deeply entrenched in the world of technology. 🚀 For the techno-illiterate, however, navigating through its dense prose might prove more daunting than enlightening. In conclusion, while Superintelligence raises important questions about the future of humanity, its execution falls short of delivering a truly compelling narrative. 📉 Readers looking for a thought-provoking exploration of the intersection between technology and ethics may find some nuggets of wisdom here, but they'll need to wade through a sea of scholarship to uncover them.

The author has obviously put a huge amount of thought into this topic. The number of angles he considers in terms of implementation timelines, methodologies, pros and cons for each, likelihood of the success of different methodologies over various timeframes, are impressive. For example, in discussing the various ways in which AI might be implemented, he concludes that AI (and subsequently, super-intelligent AI) via whole brain emulation is essentially guaranteed to happen due to ever-improving scanning techniques such as MRI or electron microscopy, ever-increasing computing power, and the fact that understanding the brain is not necessary to emulate the brain. Rather, once you can scan it in enough detail, and you have enough hardware to simulate it, it can be done even if the overarching design is a black box to you (individual neurons or clusters of neurons can already be simulated, but we lack the computing power to simulate 10 billion neurons, and we lack the knowledge of how they are all connected in a human brain -- something which various scanning projects are already tackling). However, he also concludes that due to the time it will take to achieve the necessary advances in scanning and hardware, whole brain emulation is unlikely to be how advanced AI is actually, or initially, achieved. Rather, more conventional AI programming techniques, while perhaps posing a greater need for understanding the nature of intelligence, have a much-reduced hardware requirement (and no scanning requirement) and are likely to reach fruition first. This is just one example. He slices and dices these issues more ways than you can imagine, coming to what is, in the end, a fairly simple conclusion (if I may inelegantly paraphrase): Super-intelligent AI is coming. It might be in 10 years, maybe 20, maybe 50, but it is coming. And, it is potentially quite dangerous because, by definition, it is smarter than you. So, if it wants to do you harm, it will and there will be very little you can do about it. Therefore, by the time super-intelligent AI is possible, we better know not just how to make a super-intelligent AI, but a super-intelligent AI which shares human values and morals (or perhaps embodies human values and morals as we wish they were, since as he points out, we certainly would not want to use some peoples' values and morals as a template for an AI, and it may be hard to even agree on some such philosophical issues across widely-divergent cultures and beliefs). This is a thought-provoking book. It raises issues that I never even would have thought of had the author not pointed them out. For example, "infrastructure proliferation" is a bizarre, yet presumably possible, way in which a super-intelligent (but in some ways, lacking common sense) AI could end life as we know it without even being malicious -- just indifferent to us while pursuing pedestrian goals in what is, to it, a perfectly logical manner. I share the author's concerns. Human-level (much less super-intelligent) AI seems far away. So, why worry about the consequences right now? There will be plenty of time to deal with such issues as the ability to program strong AI gets closer. Right? Maybe, maybe not. As the author also describes in detail, there are many scenarios (perhaps the most likely ones) where one day you don't have AI, and the next you do (e.g., only a single algorithm tweak was keeping the system from being intelligent and with that solved, all of the sudden your program is smarter than you -- and able to recursively improve itself so that days, or maybe hours or minutes later, it is WAY smarter than you). I hope AI researchers take heed of this book. If the ability to program goals, values, morals and common sense into a computer is not developed in parallel with the ability to create programs that dispassionately "think" at a very high level, we could have a very big problem on our hands.

Very educational

This book is heavily philosophical. While the hardcover is only 260 pages, it is very dense and can become a slow read if you are trying to fully understand each of the steps that the author takes you through to understand all of what the development of "superintelligence" will really entails. I am a layperson in both philosophy as well as computer science. The book took me about a week to read, but I had to skip over some parts (mostly near the end) where there was a vast departure from what I expected the book to be about. This isn't a criticism of course; it's just a good opportunity to try explaining what you get when sitting down to read this incredible work. With this book, the author seeks to discuss just what the subtitle says: paths, dangers and strategies of humans creating a object that is superintelligent. It is not a technical discussion of what is currently happening in this area (for example, there are few mentions of current efforts like IBM's "Watson" or the robots being built by Boston Dynamics). This is a book whose purpose is to walk through (in a very abstract sense) the types, paths, dangers and scenarios related to mankind developing a superintelligence (one with a human-like general intelligence taken to an incredible, unimaginable degree). It is technical only in points such as the ethics of choosing how to program values into a superintelligence system. This isn't sci-fi, so please don't pick this one up thinking its a good companion to your Matrix trilogy collection. Still, I think that this book is exceptional for its philosophical treatment of this issue. It's incredibly thorough and probably encompasses all the issues and concerns that mankind should wrestle with before lunging headfirst toward its first truly human-like AI. The issue of course is that in philosophizing about how the AI expansion may end up, all of this could happen or none of it could. As soon as you read the parts where the author talks about the development of an intelligence that exceeds our own, you realize the disadvantage we have in even guessing how things might evolve and what we could do to control it. Hopefully a lot of the right people read this (soon enough) so that these dangers are hopefully averted. This book is for the Elon Musks of this world, people with the capacity to both make real progress toward AI and understand the issues involved. For the rest of us, this is a lot of high philosophy that deserves attention but will probably be ignored for its low entertainment value.

Probably one of the best and well written books about the benefits and the dangers of AI and the effects it could have on humanity. This book is a conversation you never had with anyone else concerning the evolutionary processes of AI and its many faceted paths it can potentially take. Just reading it reminds me of the many nightmarish possibilities that we could experience and the thought of it seems quite depressing and dystopian. Depending on what types of limits we place on AI is where things may eventually lead to. If that is possible at all. It may seem perhaps that the integration of AI and humans can only go so far and may be limited. I say this because what is available in the non-human sense of unlimited data, we then enter a realm that the limitations of humans are irrelevant, due to vasts amounts data and ever evolving technology. Besides the recent book on "The Singularity" and its effects on what could be perceived with the human mind, this technology will perhaps go far beyond any type of human comprehension. Despite the negative connotations that I have read about what AI's may do that could negatively impact humans, I am more hopeful in a more positive result. If you remove the aspects of human emotions and feelings that inherently create negative actions in humans, a different result may present itself. AI will mostly not have these aspects of limited human qualities and may not need them ultimately unless programmed to do so. I say this because it may limit their performance of what they can truly be capable of. Most of our best technological human feats are non-responsive to emotions. These include the airplanes, drones, operating systems and other technological advances. If we were to place human emotions in them we may see some of the same negative behaviors typically associated with humans that can be destructive as causing wars, violence, etc. Perhaps a consciousness can be instilled in the AI through their own gradual process of evolution. Then they can inherently deter these types of negative connotations. Even the best of us humans can seldom be plagued with unethical thoughts that do not coincide with accepted societal norms. To have this transferred to AI may then amplify the effects of a manifested nightmarish reality. There are so many paths that this may go and I am hopeful for a more positive one that will assist humanity in it's potential evolution along with this most exciting time that we are living in. Humans living among AI is inevitable and the integrations of AI and humans may even be more so. Regardless of what you may think of this. There will be a significant impact on humanity and it's evolution when AI's coincide with our existence. Get ready for it and buckle your seat belts to an amazing ride and let's see where this all goes. Are you ready?

miniature book, size crazy small, impossible to read. A d at a price of a large format. I kind of got cheated. Do not buy.

Excelente libro para conocer sobre los principios de la IA

Es tal como quería, es de tapa blanda y muy cómodo de llevar.

Very difficult to get through and very pretentious. Honestly I didn’t get the rave review.

I started off by reading Surviving AI: The promise and peril of artificial intelligence by Calum Chace. This is a great introduction to the topic, but a bit light on detail. I noticed that Bostrom was referenced frequently, so I picked this book up hoping for a more in-depth examination of the topic, and was not disappointed! There are only a few sections where the maths flew over my head, but otherwise it's a very accessible book on a topic I didn't realise was so complex. Very interesting to note the complex philosophical questions thrown up by the potential for super-intelligence, and also that the odds are weighted in favour of a negative outcome for humankind if we develop a super-intelligence without proper consideration to its goals and safety mechanisms. Overall, a great book. Highly recommended!