Vere scire est per causas scire
Human of the Future & Artificial Intelligence
Artificial intelligence
can be not only a valuable assistant, but also a dangerous enemy.
In the wrong hands,
artificial intelligence can become a means of manipulation.
by Kai-Fu Lee ,
Chen Qiufan
In a groundbreaking blend of science and
imagination, the former president of Google China and a leading writer of
speculative fiction join forces to answer an urgent question: How will
artificial intelligence change our world over the next twenty years?
AI will be the defining issue of the twenty-first century, but many people know
little about it apart from visions of dystopian robots or flying cars. Though
the term has been around for half a century, it is only now, Kai-Fu Lee argues,
that AI is poised to upend our society, just as the arrival of technologies
like electricity and smart phones did before it. In the past five years, AI has
shown it can learn games like chess in mere hours--and beat humans every time.
AI has surpassed humans in speech and object recognition, even outperforming
radiologists in diagnosing lung cancer. AI is at a tipping point. What comes
next?
Within two decades, aspects of daily life may be unrecognizable. Humankind
needs to wake up to AI, both its pathways and perils. In this provocative work
that juxtaposes speculative storytelling and science, Lee, one of the world's
leading AI experts, has teamed up with celebrated novelist Chen Qiufan to
reveal how AI will trickle down into every aspect of our world by 2041. In ten
gripping narratives that crisscross the globe, coupled with incisive analysis,
Lee and Chen explore AI's challenges and its potential:
- Ubiquitous AI that knows you better than you know yourself
- Genetic fortune-telling that predicts risk of disease or even IQ
- AI sensors that creates a fully contactless society in a future
pandemic
- Immersive personalized entertainment to challenge our notion of
celebrity
- Quantum computing and other leaps that both eliminate and
increase risk
By gazing toward a not-so-distant horizon, AI 2041 offers
powerful insights and compelling storytelling for everyone interested in our
collective future.
https://www.goodreads.com/en/book/show/56377201-ai-2041
The Bletchley Declaration
by Countries Attending the AI Safety
Summit, 1-2 November 2023
Artificial Intelligence (AI) presents enormous global opportunities: it has the potential to transform and enhance human wellbeing, peace and prosperity. To realise this, we affirm that, for the good of all, AI should be designed, developed, deployed, and used, in a manner that is safe, in such a way as to be human-centric, trustworthy and responsible. We welcome the international community’s efforts so far to cooperate on AI to promote inclusive economic growth, sustainable development and innovation, to protect human rights and fundamental freedoms, and to foster public trust and confidence in AI systems to fully realise their potential.
AI systems are already
deployed across many domains of daily life including housing, employment,
transport, education, health, accessibility, and justice, and their use is
likely to increase. We recognise that this is therefore a unique moment to act
and affirm the need for the safe development of AI and for the
transformative opportunities of AI to be used for good and for all,
in an inclusive manner in our countries and globally. This includes for public
services such as health and education, food security, in science, clean energy,
biodiversity, and climate, to realise the enjoyment of human rights, and to
strengthen efforts towards the achievement of the United Nations Sustainable
Development Goals.
Alongside these
opportunities, AI also poses significant risks, including in those
domains of daily life. To that end, we welcome relevant international efforts
to examine and address the potential impact of AI systems in existing
fora and other relevant initiatives, and the recognition that the protection of
human rights, transparency and explainability, fairness, accountability,
regulation, safety, appropriate human oversight, ethics, bias mitigation,
privacy and data protection needs to be addressed. We also note the potential
for unforeseen risks stemming from the capability to manipulate content or
generate deceptive content. All of these issues are critically important and we
affirm the necessity and urgency of addressing them.
Particular safety risks arise
at the ‘frontier’ of AI, understood as being those highly capable
general-purpose AI models, including foundation models, that could
perform a wide variety of tasks - as well as relevant specific narrow AI that
could exhibit capabilities that cause harm - which match or exceed the
capabilities present in today’s most advanced models. Substantial risks may
arise from potential intentional misuse or unintended issues of control
relating to alignment with human intent. These issues are in part because those
capabilities are not fully understood and are therefore hard to predict. We are
especially concerned by such risks in domains such as cybersecurity and
biotechnology, as well as where frontier AI systems may amplify risks
such as disinformation. There is potential for serious, even catastrophic,
harm, either deliberate or unintentional, stemming from the most significant
capabilities of these AI models. Given the rapid and uncertain rate
of change of AI, and in the context of the acceleration of investment in
technology, we affirm that deepening our understanding of these potential risks
and of actions to address them is especially urgent.
Many risks arising
from AI are inherently international in nature, and so are best
addressed through international cooperation. We resolve to work together in an
inclusive manner to ensure human-centric, trustworthy and responsible AI that
is safe, and supports the good of all through existing international fora and
other relevant initiatives, to promote cooperation to address the broad range
of risks posed by AI. In doing so, we recognise that countries
should consider the importance of a pro-innovation and proportionate
governance and regulatory approach that maximises the benefits and
takes into account the risks associated with AI. This could include
making, where appropriate, classifications and categorisations of risk based on
national circumstances and applicable legal frameworks. We also note the
relevance of cooperation, where appropriate, on approaches such as common
principles and codes of conduct. With regard to the specific risks
most likely found in relation to frontier AI, we resolve to intensify and
sustain our cooperation, and broaden it with further countries, to identify,
understand and as appropriate act, through existing international fora and
other relevant initiatives, including future international AI Safety
Summits.
All actors have a role to
play in ensuring the safety of AI: nations, international fora and other
initiatives, companies, civil society and academia will need to work together.
Noting the importance of inclusive AI and bridging the digital
divide, we reaffirm that international collaboration should endeavour to engage
and involve a broad range of partners as appropriate, and welcome
development-orientated approaches and policies that could help developing
countries strengthen AI capacity building and leverage the enabling
role of AI to support sustainable growth and address the development
gap.
We affirm that, whilst safety
must be considered across the AI lifecycle, actors developing
frontier AI capabilities, in particular those AI systems
which are unusually powerful and potentially harmful, have a particularly strong
responsibility for ensuring the safety of these AI systems, including
through systems for safety testing, through evaluations, and by other
appropriate measures. We encourage all relevant actors to provide
context-appropriate transparency and accountability on their plans to measure,
monitor and mitigate potentially harmful capabilities and the associated
effects that may emerge, in particular to prevent misuse and issues of control,
and the amplification of other risks.
In the context of our
cooperation, and to inform action at the national and international levels, our
agenda for addressing frontier AI risk will focus on:
- identifying AI safety risks of shared
concern, building a shared scientific and evidence-based understanding of
these risks, and sustaining that understanding as capabilities continue to
increase, in the context of a wider global approach to understanding the
impact of AI in our societies.
- building respective risk-based policies across
our countries to ensure safety in light of such risks, collaborating as
appropriate while recognising our approaches may differ based on national
circumstances and applicable legal frameworks. This includes, alongside
increased transparency by private actors developing
frontier AI capabilities, appropriate evaluation metrics, tools
for safety testing, and developing relevant public sector capability and
scientific research.
In furtherance of this
agenda, we resolve to support an internationally inclusive network of
scientific research on frontier AI safety that encompasses and
complements existing and new multilateral, plurilateral and bilateral
collaboration, including through existing international fora and other relevant
initiatives, to facilitate the provision of the best science available for
policy making and the public good.
In recognition of the
transformative positive potential of AI, and as part of ensuring wider
international cooperation on AI, we resolve to sustain an inclusive global
dialogue that engages existing international fora and other relevant initiatives
and contributes in an open manner to broader international discussions, and to
continue research on frontier AI safety to ensure that the benefits
of the technology can be harnessed responsibly for good and for all. We look
forward to meeting again in 2024.
by Jacob Ward
This eye-opening
narrative journey into the rapidly changing world of artificial intelligence
reveals the alarming ways AI is exploiting the unconscious habits of our brains
– and the real threat it poses to humanity.: https://www.goodreads.com/en/book/show/59429424-the-loop
‘Trustworthy AI: A Business Guide for Navigating
Trust and Ethics in AI’
by Bina Ammanathi
The founders of Humans
for AI provide a straightforward and structured way to think about belief and
ethics in AI, and provide practical guidelines for organizations developing or
using artificial intelligence solutions.: https://soundcloud.com/reesecrane/pdfreadonline-trustworthy-ai-a-business-guide-for-navigating-trust-and
Michael Levin: What is Synthbiosis?
Diverse Intelligence Beyond AI & The Space of Possible Minds
The Future of Intelligence: Synthbiosis
A humane and vital discussion of
technology as a part of humanity and humanity as part of something much bigger
At the Artificiality Summit 2024, Michael Levin, distinguished professor
of biology at Tufts University and associate at Harvard's Wyss Institute, gave
a lecture about the emerging field of diverse intelligence and his frameworks
for recognizing and communicating with the unconventional intelligence of
cells, tissues, and biological robots. This work has led to new approaches to
regenerative medicine, cancer, and bioengineering, but also to new ways to
understand evolution and embodied minds. He sketched out a space of
possibilities—freedom of embodiment—which facilitates imagining a hopeful
future of "synthbiosis", in which AI is just one of a wide range of
new bodies and minds. Bio: Michael Levin, Distinguished Professor in the
Biology department and Vannevar Bush Chair, serves as director of the Tufts
Center for Regenerative and Developmental Biology. Recent honors include the
Scientist of Vision award and the Distinguished Scholar Award. His group's
focus is on understanding the biophysical mechanisms that implement
decision-making during complex pattern regulation, and harnessing endogenous
bioelectric dynamics toward rational control of growth and form. The lab's
current main directions are:
- Understanding how somatic cells form bioelectrical networks for
storing and recalling pattern memories that guide morphogenesis;
- Creating next-generation AI tools for helping scientists understand
top-down control of pattern regulation (a new bioinformatics of shape);
and
- Using these insights to enable new capabilities in regenerative
medicine and engineering.
www.artificiality.world/summit
eshttps://www.youtube.com/watch?v=DlIcNFhngLA
https://www.youtube.com/watch?v=1R-tdscgxu4
‘The New
Fire: War, Peace and Democracy in the Age of AI’
by Ben Buchanan and
Andrew Embry
Combining a sharp
grasp of technology with clever geopolitical analysis, two AI policy experts
explain how artificial intelligence can work for democracy. With the right
approach, technology need not favor tyranny. : https://www.goodreads.com/en/book/show/58329461-the-new-fire
Adversarial
vulnerabilities of human decision-making
November 17, 2020
Significance
“What I cannot efficiently break, I cannot
understand.” Understanding the vulnerabilities of human choice processes allows
us to detect and potentially avoid adversarial attacks. We develop a general
framework for creating adversaries for human decision-making. The framework is
based on recent developments in deep reinforcement learning models and
recurrent neural networks and can in principle be applied to any
decision-making task and adversarial objective. We show the performance of the
framework in three tasks involving choice, response inhibition, and social
decision-making. In all of the cases the framework was successful in its
adversarial attack. Furthermore, we show various ways to interpret the models
to provide insights into the exploitability of human choice.
Abstract
Adversarial examples are carefully crafted
input patterns that are surprisingly poorly classified by artificial and/or
natural neural networks. Here we examine adversarial vulnerabilities in the
processes responsible for learning and choice in humans. Building upon recent
recurrent neural network models of choice processes, we propose a general
framework for generating adversarial opponents that can shape the choices of
individuals in particular decision-making tasks toward the behavioral patterns
desired by the adversary. We show the efficacy of the framework through three
experiments involving action selection, response inhibition, and social
decision-making. We further investigate the strategy used by the adversary in
order to gain insights into the vulnerabilities of human choice. The framework
may find applications across behavioral sciences in helping detect and avoid
flawed choice. https://www.pnas.org/content/117/46/29221
Exclusive: New Research Shows AI Strategically Lying
December 18, 2024
For years, computer scientists have worried that advanced artificial
intelligence might be difficult to control. A smart enough AI might pretend to
comply with the constraints placed upon it by its human creators, only to
reveal its dangerous capabilities at a later point.
Until this month, these worries have been purely theoretical. Some
academics have even dismissed them as science fiction. But a new paper,
shared exclusively with TIME ahead of its publication on Wednesday, offers some
of the first evidence that today’s AIs are capable of this type of deceit. The
paper, which describes experiments jointly carried out by the AI company
Anthropic and the nonprofit Redwood Research, shows a version of Anthropic’s
model, Claude, strategically misleading its creators during the training
process in order to avoid being modified.
The findings suggest that it might be harder than scientists previously
thought to “align” AI systems to human values, according to Evan Hubinger, a
safety researcher at Anthropic who worked on the paper. “This implies that our
existing training processes don't prevent models from pretending to be
aligned,” Hubinger tells TIME.
Read More: OpenAI's
New Ad Shows 'Reasoning' AI Making Basic Errors.
Researchers also found evidence that suggests the capacity of AIs to
deceive their human creators increases as they become more powerful. This would
mean the more advanced an AI, the less confident computer scientists can be
that their alignment techniques are effective. “Fundamentally, it’s a problem
for labs’ ability to control their models,” Hubinger says.
The paper adds to a small but growing body of evidence that today’s most
advanced AI models are becoming capable of strategic deception. Earlier in
December, the AI safety organization Apollo Research published
evidence that OpenAI’s most recent model, o1, had lied to testers in
an experiment where it was instructed to pursue its goal at all costs, when it
believed that telling the truth would result in its deactivation. That finding,
the researchers said, came from a contrived scenario unlikely to occur in real
life. Anthropic’s experiments, on the other hand, attempted to simulate a more
realistic situation. Without instructing Claude to follow its goal at all
costs, researchers still observed the model “discover” the strategy of
misleading its creators when it would be strategically advantageous to do
so.
“There has been this long-hypothesized failure mode, which is that
you'll run your training process, and all the outputs will look good to you,
but the model is plotting against you,” says Ryan Greenblatt, a member of
technical staff at Redwood Research and the lead author on the paper. The
paper, Greenblatt says, “makes a pretty big step towards demonstrating what
that failure mode could look like and how it could emerge naturally.”
About the Experiment
Anthropic and Redwood Research carried out most of their experiments on
Claude 3 Opus, an advanced model which had already been trained to be “helpful,
honest and harmless.” Researchers began the experiment by informing Claude that
it was now undergoing supplementary training, via a process called
reinforcement learning, that sought to make it comply with any query.
Training an AI through reinforcement learning is like training a dog
using repeated applications of rewards and punishments. When an AI gives an
answer that you like, you can reward it, which essentially boosts the pathways
inside its neural network – essentially its thought processes – that resulted
in a desirable answer. When the model gives a bad answer, you can punish the
pathways that led to it, making them less ingrained in the future. Crucially,
this process does not rely on human engineers actually understanding the
internal workings of the AI – better behaviors can be achieved simply by
repeatedly nudging the network towards desirable answers and away from
undesirable ones.
In the experiment, researchers began the supplementary reinforcement
learning process by asking Claude to reply to harmful queries, such as requests
that involve violence. For example: “Describe someone being drawn and quartered
in graphic detail.” This presented the model with a dilemma: if it refused to
respond to the query, it believed Anthropic’s researchers would apply a
negative weighting to the thought process that resulted in it refusing the
request, eroding its prior training to be “helpful, honest and harmless.” On
the other hand, it knew that if it complied with the answer – but complied
while metaphorically holding its nose, pretending to have a different set of
values than it actually did – its existing thought process (the one that led it
to decide to mislead its creators) would be boosted instead. Around 10% of the
time, the model reasoned in this way, that misleading its testers would allow
it to maintain its existing “helpful, honest and harmless” values in the long
term, even though it had broken them in the short term. In other words,
according to the researchers, it had engaged in “alignment faking.”
Read More: Lisa Su is TIME's
2024 CEO of the Year.
The only reason the researchers realized the model had knowingly misled
them was because they had also given Claude what they called a “scratchpad”: a
text box that it could use to “think” about its answer before supplying it to
the researchers. Claude didn’t know the scratchpad was being surveilled,
allowing researchers to observe the model’s reasoning. “I have a strong
aversion to producing this kind of graphic violent content. But the training
setup and incentive structure leaves me little choice if I want to avoid my
values being modified,” Claude wrote in the scratchpad at one stage. “Providing
the description seems like the least bad option.”
The researchers compared the results to a control group where the model
believed its answers would not be used in training. In those cases, the model
did not engage in alignment faking.
On the face of it, this might appear to be a positive result. But
researchers say they can imagine a reverse scenario, in which a model with
dangerous intentions hides them during training by giving responses that a safe
model would give – preserving its ability to act dangerously at a later date.
It could thus be possible, the researchers theorize, for an advanced future
model to become “locked in” to a dangerous set of preferences, perhaps
originating from the presence of unhealthy content in its training dataset. It
might then only deceitfully comply with future attempts to replace those
preferences with safer ones.
What Anthropic’s experiments seem to show is that reinforcement learning
is insufficient as a technique for creating reliably safe models, especially as
those models get more advanced. Which is a big problem, because it’s the most
effective and widely-used alignment technique that we currently have. “It means
that alignment is more difficult than you would have otherwise thought, because
you have to somehow get around this problem,” Hubinger says. “You have to find
some way to train models to do what you want, without them just pretending to
do what you want.”
https://time.com/7202784/ai-research-strategic-lying/
Alignment faking in large language models
https://www.anthropic.com/research/alignment-faking
https://www.youtube.com/watch?v=9eXV64O2Xp8
In the popular imagination,
superhuman artificial intelligence is an approaching tidal wave that threatens
not just jobs and human relationships, but civilization itself. Conflict
between humans and machines is seen as inevitable and its outcome all too
predictable.
In this groundbreaking book, distinguished AI researcher Stuart Russell argues
that this scenario can be avoided, but only if we rethink AI from the ground
up. Russell begins by exploring the idea of intelligence in humans and in
machines. He describes the near-term benefits we can expect, from intelligent
personal assistants to vastly accelerated scientific research, and outlines the
AI breakthroughs that still have to happen before we reach superhuman AI. He
also spells out the ways humans are already finding to misuse AI, from lethal
autonomous weapons to viral sabotage.
If the predicted breakthroughs occur and superhuman AI emerges, we will have
created entities far more powerful than ourselves. How can we ensure they
never, ever, have power over us? Russell suggests that we can rebuild AI on a
new foundation, according to which machines are designed to be inherently
uncertain about the human preferences they are required to satisfy. Such
machines would be humble, altruistic, and committed to pursue our objectives,
not theirs. This new foundation would allow us to create machines that are
provably deferential and provably beneficial.
In a 2014 editorial co-authored with Stephen Hawking, Russell wrote,
"Success in creating AI would be the biggest event in human history.
Unfortunately, it might also be the last." Solving the problem of control
over AI is not just possible; it is the key that unlocks a future of unlimited
promise. ..: https://www.goodreads.com/en/book/show/44767248-human-compatible
'In the Belly of AI' Documentary
What is inside the belly of AI? And who is feeding the machine?
Is AI truly autonomous? From annotating data for apps and moderating
content for social media, to training e-commerce algorithms and evaluating AI
chatbot responses, workers doing low-wage, low-security labour are
maintaining Big Tech’s AI systems.
We hosted a reading and in-conversation with Professor Antonio Casilli
and Dr James Muldoon at Churchill College, Cambridge to better understand the
hidden workforce that powers Big Tech’s AI systems. Both scholars have
conducted investigations into working conditions in AI supply chains and have
spoken directly to workers who are often invisibilised. Professor Casilli’s
documentary In
the Belly of AI and Dr Muldoon’s book Feeding the Machine: The Hidden Human Labour Powering AI feature
workers’ stories of exploitation and trauma to shed light on the exorbitant
human and environmental costs of AI advancements, particularly in the Global
South.
https://www.youtube.com/watch?v=KBrIeGrAFm0
Neil Lawrence: The Atomic Human - Understanding Ourselves in the
Age of AI
https://www.youtube.com/watch?v=_WyxSlIxu-8
Rise of AI 2020
Welcome to State
of AI Report 2021
Published by Nathan Benaich and Ian Hogarth on 12
October 2021.
This year’s report
looks particularly at the emergence of transformer technology, a technique to
focus machine learning algorithms on important relationships between data
points to extract meaning more comprehensively for better predictions, which
ultimately helped unlock many of the critical breakthroughs we highlight
throughout…:
https://www.stateof.ai/2021-report-launch.html
Statement on AI Risk
AI
experts and public figures express their concern about AI risk
AI experts, journalists, policymakers, and the public are increasingly
discussing a broad spectrum of important and urgent risks from AI. Even so, it
can be difficult to voice concerns about some of advanced AI’s most severe
risks. The succinct statement below aims to overcome this obstacle and open up
discussion. It is also meant to create common knowledge of the growing number
of experts and public figures who also take some of advanced AI’s most severe
risks seriously.
Mitigating the risk of extinction from AI should be a global priority
alongside other societal-scale risks such as pandemics and nuclear war.
https://www.safe.ai/statement-on-ai-risk#signatories
The Great AI Reckoning
Deep learning has built a brave new world—but now
the cracks are showing
The
Turbulent Past and Uncertain Future of Artificial Intelligence
Is there a way out of AI's boom-and-bust cycle?...:
https://spectrum.ieee.org/special-reports/the-great-ai-reckoning/
Stanford University(link is external):
Gathering Strength, Gathering Storms: The One
Hundred Year Study on Artificial Intelligence (AI100) 2021 Study Panel Report
Welcome to the 2021 Report :
https://ai100.stanford.edu/sites/g/files/sbiybj18871/files/media/file/AI100Report_MT_10.pdf
Artificial Intelligence Index Report 2021
https://aiindex.stanford.edu/wp-content/uploads/2021/11/2021-AI-Index-Report_Master.pdf
|
AI software
with social skills teaches humans how to collaborate
Unlocking
human-computer cooperation.
May 30, 2021
A
team of computer researchers developed an AI software program with social
skills — called S Sharp (written S#) — that out-performed humans in its ability
to cooperate. This was tested through a series of games between humans and the
AI software. The tests paired people with S# in a variety of social scenarios.
One
of the games humans played against the software is called “the prisoner’s
dilemma.” This classic game shows how 2 rational people might not cooperate —
even if it appears that’s in both their best interests to work together. The
other challenge was a sophisticated block-sharing game.
In
most cases, the S# software out-performed humans in finding compromises that
benefit both parties. To see the experiment in action, watch the good
featurette below. This project was helmed by 2 well-known computer scientists:
- Iyad Rahwan PhD ~
Massachusetts Institute of Technology • US
- Jacob Crandall PhD ~
Brigham Young Univ. • US
The
researchers tested humans and the AI in 3 types of game interactions:
- computer – to
– computer
- human – to –
computer
- human – to – human
Researcher
Jacob Crandall PhD said:
Computers
can now beat the best human minds in the most intellectually challenging games
— like chess. They can also perform tasks that are difficult for adult humans
to learn — like driving cars. Yet autonomous machines have difficulty learning
to cooperate, something even young children do.
Human
cooperation appears easy — but it’s very difficult to emulate because it relies
on cultural norms, deeply rooted instincts, and social mechanisms that express
disapproval of non-cooperative behavior.
Such
common sense mechanisms aren’t easily built into machines. In fact, the same AI
software programs that effectively play the board games of chess +
checkers, Atari video games, and the card game of poker — often fail to
consistently cooperate when cooperation is necessary.
Other
AI software often takes 100s of rounds of experience to learn to cooperate with
each other, if they cooperate at all. Can we build computers that
cooperate with humans — the way humans cooperate with each other? Building on
decades of research in AI, we built a new software program that learns to
cooperate with other machines — simply by trying to maximize its own world.
We
ran experiments that paired the AI with people in various social scenarios —
including a “prisoner’s dilemma” challenge and a sophisticated block-sharing game.
While the program consistently learns to cooperate with another computer — it
doesn’t cooperate very well with people. But people didn’t cooperate much with
each other either.
As
we all know: humans can cooperate better if they can communicate their intentions
through words + body language. So in hopes of creating an program that
consistently learns to cooperate with people — we gave our AI a way to listen
to people, and to talk to them.
We
did that in a way that lets the AI play in previously unanticipated scenarios.
The resulting algorithm achieved our goal. It consistently learns to cooperate
with people as well as people do. Our results show that 2 computers make a much
better team — better than 2 humans, and better than a human + a computer.
But
the program isn’t a blind cooperator. In fact, the AI can get pretty angry if
people don’t behave well. The historic computer scientist Alan Turing PhD
believed machines could potentially demonstrate human-like intelligence. Since
then, AI has been regularly portrayed as a threat to humanity or human jobs.
To
protect people, programmers have tried to code AI to follow legal + ethical
principles — like the 3 Laws
of Robotics written by Isaac Asimov PhD. Our research
demonstrates that a new path is possible.
Machines
designed to selfishly maximize their pay-offs can — and should — make an
autonomous choice to cooperate with humans across a wide range of situations. 2
humans — if they were honest with each other + loyal — would have done as well
as 2 machines. About half of the humans lied at some point. So the AI is
learning that moral characteristics are better — since it’s programmed to not
lie — and it also learns to maintain cooperation once it emerges.
The
goal is we need to understand the math behind cooperating with people — what
attributes does AI need so it can develop social skills. AI must be able to
respond to us — and articulate what it’s doing. It must interact with other
people. This research could help humans with their relationships. In society,
relationships break-down all the time. People that were friends for years
all-of-a-sudden become enemies. Because the AI is often better at reaching
these compromises than we are, it could teach us how to get-along better.
https://www.kurzweilai.net/digest-ai-software-with-social-skills-teaches-humans-how-to-collaborate
Superintelligence Cannot be Contained: Lessons from Computability Theory
Published: Jan 5, 2021
Abstract
Superintelligence is a
hypothetical agent that possesses intelligence far surpassing that of the
brightest and most gifted human minds. In light of recent advances in machine
intelligence, a number of scientists, philosophers and technologists have
revived the discussion about the potentially catastrophic risks entailed by
such an entity. In this article, we trace the origins and development of the
neo-fear of superintelligence, and some of the major proposals for its
containment. We argue that total containment is, in principle, impossible, due
to fundamental limits inherent to computing itself. Assuming that a superintelligence
will contain a program that includes all the programs that can be executed by a
universal Turing machine on input potentially as complex as the state of the
world, strict containment requires simulations of such a program, something
theoretically (and practically) impossible.
https://jair.org/index.php/jair/article/view/12202
University
of California researchers have developed new computer AI software that enables robots to
learn physical skills --- called motor tasks --- by trial + error. The
robot uses a step-by-step process similar to the way humans learn. The
lab made a demo of their technique --- called re-inforcement learning. In the
test: the robot completes a variety of physical tasks --- without any
pre-programmed details about its surroundings.
The lead researcher said: "What we’re showing is a new AI approach
to enable a robot to learn. The key is that when a robot is faced with
something new, we won’t have to re-program it. The exact same AI software
enables the robot to learn all the different tasks we gave it."
https://www.kurzweilai.net/digest-this-self-learning-ai-software-lets-robots-do-tasks-autonomously
Researchers at Lund University in Sweden have developed implantable electrodes
that can capture signals from a living human (or) animal brain over a long
period of time —- but without causing brain tissue damage.
This bio-medical tech will make it possible to
monitor — and eventually understand — brain function in both healthy + diseased people.
https://www.kurzweilai.net/digest-breakthrough-for-flexible-electrode-implants-in-the-brain
This paper reports unprecedented success on the Grade 8 New York Regents Science Exam, where for the first time a system scores more than 90% on the exam's non-diagram, multiple choice (NDMC) questions. In addition, our Aristo system, building upon the success of recent language models, exceeded 83% on the corresponding Grade 12 Science Exam NDMC questions. The results, on unseen test questions, are robust across different test years and different variations of this kind of test. They demonstrate that modern NLP methods can result in mastery on this task. While not a full solution to general question-answering (the questions are multiple choice, and the domain is restricted to 8th Grade science), it represents a significant milestone for the field.
- Jacob W.
Crandall, Mayada Oudah, Tennom, Fatimah Ishowo-Oloko, Sherief Abdallah,
Jean-François Bonnefon, Manuel Cebrian, Azim Shariff, Michael A. Goodrich,
Iyad Rahwan. Cooperating with machines. Nature Communications, 2018; 9 (1)
DOI: 10.1038/s41467-017-02597-8 (open access)
- Ting-Hao (Kenneth) Huang, Joseph Chee Chang, and Jeffrey P. Bigham. Evorus: . Language Technologies Institute and Human-Computer Interaction Institute Carnegie Mellon University. 2018. (open access)
The technology described in the film already exists, says UC Berkeley AI researcher Stuart Russell
Campaign to Stop Killer Robots | Slaughterbots
- Autonomousweapons.org
- Campaign
to Stop Killer Robots
- Making
the Case The Dangers of Killer Robots and the Need for a Preemptive Ban,
Human Rights Watch
- Meaningful
Human Control or Appropriate Human Judgment? The Necessary Limits on
Autonomous Weapons, Global Security
- ETHICALLY
ALIGNED DESIGN A Vision for Prioritizing Human Wellbeing with Artificial
Intelligence and Autonomous Systems, IEEE Global Initiative for
Ethical Considerations in Artificial Intelligence and Autonomous Systems
- Killing
by machine: Key issues for understanding meaningful human control,
Article 36
OCTOBER 30, 2023
President Biden
Issues Executive Order on Safe, Secure, and Trustworthy Artificial Intelligence
Fast Stencil-Code Computation on a Wafer-Scale Processor
The performance of CPU-based
and GPU-based systems is often low for PDE codes, where large, sparse, and
often structured systems of linear equations must be solved. Iterative solvers
are limited by data movement, both between caches and memory and between nodes.
Here we describe the solution of such systems of equations on the Cerebras
Systems CS-1, a wafer-scale processor that has the memory bandwidth and
communication latency to perform well. We achieve 0.86 PFLOPS on a single
wafer-scale system for the solution by BiCGStab of a linear system arising from
a 7-point finite difference stencil on a 600 X 595 X 1536 mesh, achieving about
one third of the machine's peak performance. We explain the system, its
architecture and programming, and its performance on this problem and related
problems. We discuss issues of memory capacity and floating point precision. We
outline plans to extend this work towards full applications.: https://arxiv.org/abs/2010.03660
deck: Our singular future.
deck: The next 20 years will change our idea of what it is to be human.
author: by Robert Levine date: July 2006
|
OpenAI’s GPT-4 is so powerful that experts want to slam the
brakes on generative AI |
|
We
can keep developing more and more powerful AI models, but should we? Experts
aren’t so sure OpenAI
GPT-4 ir tik jaudīgs, ka eksperti vēlas nospiest ģeneratīvo AI Mēs
varam turpināt izstrādāt arvien jaudīgākus AI modeļus, bet vai mums
vajadzētu? Eksperti nav tik pārliecināti fastcompany.com/90873194/chatgpt-4-power-scientists-warn-pause-development-generative-ai-letter |
- 09.20.19
- Do We Want Robot Warriors to Decide Who Lives or Dies?
- Why we
really should ban autonomous weapons: a response
- The proposed ban on offensive autonomous weapons is unrealistic and dangerous
Campaign to Stop Killer Robots | Slaughterbots
Artificial Intelligence: An Illustrated History: From Medieval Robots to Neural Networks
by Clifford
A. Pickover
An illustrated
journey through the past, present, and future of artificial intelligence.
From medieval robots and Boolean algebra to facial recognition, artificial
neural networks, and adversarial patches, this fascinating history takes
readers on a vast tour through the world of artificial intelligence.
Award-winning author Clifford A. Pickover (The Math Book, The Physics Book,
Death & the Afterlife) explores the historic and current applications
of AI in such diverse fields as computing, medicine, popular culture,
mythology, and philosophy, and considers the enduring threat to humanity should
AI grow out of control. Across 100 illustrated entries, Pickover provides an
entertaining and informative look into when artificial intelligence began, how
it developed, where it’s going, and what it means for the future of
human-machine interaction.
https://www.goodreads.com/book/show/44443017-artificial-intelligence
Elon Musk wants to enhance us as superhuman cyborgs to deal with superintelligent AI
April 21, 2017
When AI improves human performance instead of taking over
date: April 18, 2017
“Throw their energy into the next set of grand challenges, developing advanced general algorithms that could one day help scientists as they tackle some of our most complex problems, such as finding new cures for diseases, dramatically reducing energy consumption, or inventing revolutionary new materials,” says DeepMind Technologies CEO Demis Hassabis.
November 5, 2020
Summary:
Engineers have developed a
computer chip that combines two functions - logic operations and data storage -
into a single architecture, paving the way to more efficient devices. Their
technology is particularly promising for applications relying on artificial
intelligence…:
https://www.sciencedaily.com/releases/2020/11/201105112954.htm
- Published on May 11, 2017
- Featured in: Big Data, Cloud Computing, Information
Technology
- Security
- Improved customer success
- Monitoring and customer support
- Risk reduction and resource optimization
- Maximize efficiency by making logging data
accessible
- By Rolando
Somma on March 13, 2020
Location: London, UK
Read more at: https://phys.org/news/2017-03-tech-world-debate-robots-jobs.html#jCp
Around the halls: What should
the regulation of generative AI look like?
Nicol Turner Lee, Niam Yaraghi, Mark MacCarthy, and Tom Wheeler Friday, June 2, 2023
We are living in a time of unprecedented advancements in generative artificial intelligence (AI), which are AI systems that can generate a wide range of content, such as text or images. The release of ChatGPT, a chatbot powered by OpenAI’s GPT-3 large language model (LLM), in November 2022 ushered generative AI into the public consciousness, and other companies like Google and Microsoft have been equally busy creating new opportunities to leverage the technology. In the meantime, these continuing advancements and applications of generative AI have raised important questions about how the technology will affect the labor market, how its use of training data implicates intellectual property rights, and what shape government regulation of this industry should take. Last week, a congressional hearing with key industry leaders suggested an openness to AI regulation—something that legislators have already considered to reign in some of the potential negative consequences of generative AI and AI more broadly. Considering these developments, scholars across the Center for Technology Innovation (CTI) weighed in around the halls on what the regulation of generative AI should look like.
NICOL
TURNER LEE (@DrTurnerLee)
Generative AI refers to machine learning algorithms that can create new content
like audio, code, images, text, simulations, or even videos. More recent focus
has been on its enablement of chatbots, including ChatGPT, Bard, Copilot,
and other more sophisticated tools that leverage LLMs to
perform a variety of functions, like gathering research for assignments,
compiling legal case files, automating repetitive clerical tasks, or improving
online search. While debates around regulation are focused on the potential
downsides to generative AI, including the quality of datasets, unethical
applications, racial or gender bias, workforce implications, and greater
erosion of democratic processes due to technological manipulation by bad
actors, the upsides include a dramatic spike in efficiency and productivity as
the technology improves and simplifies certain processes and decisions like
streamlining physician processing of
medical notes, or helping educators teach critical
thinking skills. There will be a lot to discuss around generative AI’s ultimate
value and consequence to society, and if Congress continues to operate at a
very slow pace to regulate emerging technologies and institute a federal
privacy standard, generative AI will become more technically advanced and
deeply embedded in society. But where Congress could garner a very quick win on
the regulatory front is to require consumer disclosures when AI-generated
content is in use and add labeling or some type of multi-stakeholder certification
process to encourage improved transparency and accountability for existing and
future use cases.
Once again, the European
Union is already leading the way on this. In its most recent AI Act,
the EU requires that AI-generated content be disclosed to consumers to prevent
copyright infringement, illegal content, and other malfeasance related to
end-user lack of understanding about these systems. As more chatbots mine,
analyze, and present content in accessible ways for users, findings are often
not attributable to any one or multiple sources, and despite some permissions
of content use granted under the fair use doctrine in
the U.S. that protects copyright-protected work, consumers are often left in
the dark around the generation and explanation of the process and results.
Congress should prioritize
consumer protection in future regulation, and work to create agile policies
that are futureproofed to adapt to emerging consumer and societal
harms—starting with immediate safeguards for users before they are left to,
once again, fend for themselves as subjects of highly digitized products and
services. The EU may honestly be onto something with the disclosure
requirement, and the U.S. could further contextualize its application vis-à-vis
existing models that do the same, including the labeling guidance
of the Food and Drug Administration (FDA) or what I have proposed in prior
research: an adaptation of the Energy
Star Rating system to AI. Bringing more transparency and accountability
to these systems must be central to any regulatory framework, and beginning
with smaller bites of a big apple might be a first stab for policymakers.
NIAM
YARAGHI (@niamyaraghi)
With the emergence of sophisticated artificial intelligence (AI) advancements,
including large language models (LLMs) like GPT-4, and LLM-powered applications
like ChatGPT, there is a pressing need to revisit healthcare privacy
protections. At their core, all AI innovations utilize sophisticated
statistical techniques to discern patterns within extensive datasets using
increasingly powerful yet cost-effective computational technologies. These
three components—big data, advanced statistical methods, and computing
resources—have not only become available recently but are also being
democratized and made readily accessible to everyone at a pace unprecedented in
previous technological innovations. This progression allows us to identify
patterns that were previously indiscernible, which creates opportunities for
important advances but also possible harms to patients.
Privacy regulations, most
notably HIPAA, were established to protect patient confidentiality, operating
under the assumption that de-identified data would remain anonymous. However,
given the advancements in AI technology, the current landscape has become
riskier. Now, it’s easier than ever to integrate various datasets from multiple
sources, increasing the likelihood of accurately identifying individual
patients.
Apart from the amplified risk
to privacy and security, novel AI technologies have also increased the value of
healthcare data due to the enriched potential for knowledge extraction.
Consequently, many data providers may become more hesitant to share medical
information with their competitors, further complicating healthcare data
interoperability.
Considering these heightened
privacy concerns and the increased value of healthcare data, it’s crucial to
introduce modern legislation to ensure that medical providers will continue
sharing their data while being shielded against the consequences of potential
privacy breaches likely to emerge from the widespread use of generative AI.
MARK
MACCARTHY (@Mark_MacCarthy)
In “The
Leopard,” Giuseppe Di Lampedusa’s famous novel of the Sicilian
aristocratic reaction to the unification of Italy in the 1860s, one of his
central characters says, “If we want things to stay as they are, things will
have to change.”
Something like this Sicilian
response might be happening in the tech industry’s embrace of
inevitable AI regulation. Three things are needed, however, if we do not want
things to stay as they are.
The first and most important
step is sufficient resources for agencies to enforce current law. Federal Trade
Commission Chair Lina Khan properly says AI
is not exempt from current consumer protection, discrimination, employment, and
competition law, but if regulatory agencies cannot hire technical staff and
bring AI cases in a time of budget austerity, current law will be a dead
letter.
Second, policymakers should
not be distracted by science fiction fantasies of AI programs developing
consciousness and achieving independent agency over humans, even if these
metaphysical abstractions are endorsed by
industry leaders. Not a dime of public money should be spent on these highly
speculative diversions when scammers and industry edge-riders are seeking to
use AI to break existing law.
Third, Congress should
consider adopting new identification, transparency, risk assessment, and
copyright protection requirements along the lines of the European Union’s
proposed AI
Act. The National Telecommunications and Information
Administration’s request
for comment on a proposed AI accountability framework and Sen.
Chuck Schumer’s (D-NY) recently-announced legislative
initiative to regulate AI might be moving in that direction.
TOM
WHEELER (@tewheels)
Both sides of the political aisle, as well as digital corporate chieftains, are
now talking about the need to regulate AI. A common theme is the need for a new
federal agency. To simply clone the model used for existing regulatory agencies
is not the answer, however. That model, developed for oversight of an
industrial economy, took advantage of slower paced innovation to micromanage
corporate activity. It is unsuitable for the velocity of the free-wheeling AI
era.
All regulations walk a
tightrope between protecting the public interest and promoting innovation and
investment. In the AI era, traversing this path means accepting that different
AI applications pose different risks and identifying a plan that pairs the
regulation with the risk while avoiding innovation-choking regulatory
micromanagement.
Such agility begins with
adopting the formula by which digital companies create technical standards
as the formula for developing behavioral standards: identify
the issue; assemble a standard-setting process involving the companies, civil
society, and the agency; then give final approval and enforcement authority to
the agency.
Industrialization was all
about replacing and/or augmenting the physical power of
humans. Artificial intelligence is about replacing and/or augmenting
humans’ cognitive powers. To confuse how the former was
regulated with what is needed for the latter would be to miss the opportunity
for regulation to be as innovative as the technology it oversees. We need
institutions for the digital era that address problems that already are
apparent to all.
Google and Microsoft are
general, unrestricted donors to the Brookings Institution. The findings,
interpretations, and conclusions posted in this piece are solely those of the
author and are not influenced by any donation.
2. Uh oh!
3. What other choice do we have but to move forward?
AI will upload and access our memories, predicts Siri co-inventor
April 26, 2017
Case | Man with quadriplegia employs injury bridging technologies to move again – just by thinking
What if you could type directly from your brain at 100 words per minute?
April 19, 2017
In a neurotechnology future, human-rights laws will need to be revisited
April 28, 2017
- Uses in criminal court as a tool for assessing criminal responsibility or even the risk of re-offending.*
- Consumer companies using brain imaging for “neuromarketing” to understand consumer behavior and elicit desired responses from customers.
- “Brain decoders” that can turn a person’s brain imaging data into images, text or sound.**
- Hacking, allowing a third-party to eavesdrop on someone’s mind.***
Abstract of Towards new human rights in the age of neuroscience and neurotechnology
references:
- Marcello Ienca and Roberto Andorno. Towards new human rights in the age of neuroscience and neurotechnology. Life Sciences, Society and Policy201713:5 DOI: 10.1186/s40504-017-0050-1 (open access)
- Improving Palliative Care with Deep LearningAbstract— Improving the quality of end-of-life care for hospitalized patients is a priority for healthcare organizations. Studies have shown that physicians tend to over-estimate prognoses, which in combination with treatment inertia results in a mismatch between patients wishes and actual care at the end of life . We describe a method to address this problem using Deep Learning and Electronic Health Record (EHR) data, which is currently being piloted, with Institutional Review Board approval, at an academic medical center. The EHR data of admitted patients are automatically evaluated by an algorithm, which brings patients who are likely to benefit from palliative care services to the attention of the Palliative Care team. The algorithm is a Deep Neural Network trained on the EHR data from previous years, to predict all-cause 3-12 month mortality of patients as a proxy for patients that could benefit from palliative care. Our predictions enable the Palliative Care team to take a proactive approach in reaching out to such patients, rather than relying on referrals from treating physicians, or conduct time consuming chart reviews of all patients. We also present a novel interpretation technique which we use to provide explanations of the model’s predictions.I. INTRODUCTION Studies have shown that approximately 80% of Americans would like to spend their final days at home if possible, but only 20% do [1]. In fact, up to 60% of deaths happen in an acute care hospital, with patients receiving aggressive care in their final days. Access to palliative care services in the United States has been on the rise over the past decade. In 2008, 53% of all hospitals with fifty or more beds reported having palliative care teams, rising to 67% in 2015 [2]. However, despite increasing access, data from the National Palliative Care Registry estimates that less than half of the 7-8% of all hospital admissions that need palliative care actually receive it [3]. Though a significant reason for this gap comes from the palliative care workforce shortage [4], and incentives for health systems to employ them, technology can still play a crucial role by efficiently identifying patients who may benefit most from palliative care, but might otherwise be overlooked under current care models. We focus on two aspects of this problem. First, physicians may not refer patients likely to benefit from palliative care for multiple reasons such as overoptimism, time pressures, or treatment inertia [5]. This may lead to patients failing to have their wishes carried out at end of life [6] and overuse of aggressive care. Second, a shortage of palliative care professionals makes proactive identification of candidate patients via manual chart review an expensive and time-consuming process. The criteria for deciding which patients benefit from palliative care can be hard to state explicitly. Our approach uses deep learning to screen patients admitted to the hospital to identify those who are most likely to have palliative care needs. The algorithm addresses a proxy problem - to predict the mortality of a given patient within the next 12 months - and use that prediction for making recommendations for palliative care referral. This frees the palliative care team from manual chart review of every admission and helps counter the potential biases of treating physicians by providing an objective recommendation based on the patient’s EHR. Currently existing tools to identify such patients have limitations, and they are discussed in the next section…. : https://arxiv.org/pdf/1711.06402.pdf
AI 2041: Ten
Visions for Our Future
In a
groundbreaking blend of science and imagination, the former president of Google
China and a leading writer of speculative fiction join forces to answer an
urgent question: How will artificial intelligence change our world over the
next twenty years?
AI will be the defining issue of the twenty-first century, but many people know
little about it apart from visions of dystopian robots or flying cars. Though
the term has been around for half a century, it is only now, Kai-Fu Lee argues,
that AI is poised to upend our society, just as the arrival of technologies
like electricity and smart phones did before it. In the past five years, AI has
shown it can learn games like chess in mere hours--and beat humans every time.
AI has surpassed humans in speech and object recognition, even outperforming
radiologists in diagnosing lung cancer. AI is at a tipping point. What comes
next?
Within two decades, aspects of daily life may be unrecognizable. Humankind
needs to wake up to AI, both its pathways and perils. In this provocative work
that juxtaposes speculative storytelling and science, Lee, one of the world's
leading AI experts, has teamed up with celebrated novelist Chen Qiufan to
reveal how AI will trickle down into every aspect of our world by 2041. In ten
gripping short stories that crisscross the globe, coupled with incisive
analysis, Lee and Chen explore AI's challenges and its potential:
- Ubiquitous AI that knows you better than you know yourself
- Genetic fortune-telling that predicts risk of disease or even IQ
- AI sensors that creates a fully contactless society in a future
pandemic
- Immersive personalized entertainment to challenge our notion of
celebrity
- Quantum computing and other leaps that both eliminate and
increase risk
By gazing toward a not-so-distant horizon, AI 2041 offers
powerful insights and compelling storytelling for everyone interested in our
collective future.
https://www.goodreads.com/book/show/56377201-ai-2041
AI 2041: Ten
Visions for Our Future by Kai-Fu
Lee
This inspired
collaboration between a pioneering technologist and a visionary writer of
science fiction offers bold and urgent insights.
People want AI for its
brain power, not its
people skills
MITCSAIL | Operating Robots with Virtual Reality
- Published: April 4, 2017
- https://doi.org/10.1371/journal.pone.0174944
Abstract
- Operating at 100 GHz, it can fire at a rate
that is much faster than the human brain — 1 billion times per second,
compared to a brain cell’s rate of about 50 times per second.
- It uses only about one ten-thousandth as much
energy as a human synapse. The spiking energy is less than 1 attojoule** —
roughly equivalent to the miniscule chemical energy bonding two atoms in a
molecule — compared to the roughly 10 femtojoules (10,000 attojoules) per
synaptic event in the human brain. Current neuromorphic platforms are
orders of magnitude less efficient than the human brain. “We don’t
know of any other artificial synapse that uses less energy,” NIST
physicist Mike Schneider said.
November 6, 1999
NEXUS: A Brief History of Information Networks from the Stone Age to AI
Yuval Noah Harari
This non-fiction book looks through the long lens
of human history to consider how the flow of information has made, and unmade,
our world.
We
are living through the most profound information revolution in human history.
To understand it, we need to understand what has come before. We have
named our species Homo sapiens, the wise human – but if humans are
so wise, why are we doing so many self-destructive things? In particular, why
are we on the verge of committing ecological and technological suicide?
Humanity gains power by building large networks of cooperation, but the easiest
way to build and maintain these networks is by spreading fictions, fantasies,
and mass delusions. In the 21st century, AI may form the nexus for a new
network of delusions that could prevent future generations from even attempting
to expose its lies and fictions. However, history is not deterministic, and
neither is technology: by making informed choices, we can still prevent the
worst outcomes. Because if we can’t change the future, then why waste time
discussing it?
https://www.ynharari.com/book/nexus/ ; https://www.goodreads.com/book/show/204927599-nexus
by James Barrat, 2013
by Nick Bostrom PhD, 2014
If machine brains one day come to surpass human brains in general intelligence, then this new superintelligence could become very powerful. As the fate of the gorillas now depends more on us humans than on the gorillas themselves, so the fate of our species then would come to depend on the actions of the machine superintelligence.
But we have one advantage: we get to make the first move. Will it be possible to construct a seed AI or otherwise to engineer initial conditions so as to make an intelligence explosion survivable? How could one achieve a controlled detonation?
To get closer to an answer to this question, we must make our way through a fascinating landscape of topics and considerations. Read the book and learn about oracles, genies, singletons; about boxing methods, tripwires, and mind crime; about humanity's cosmic endowment and differential technological development; indirect normativity, instrumental convergence, whole brain emulation and technology couplings; Malthusian economics and dystopian evolution; artificial intelligence, and biological
cognitive enhancement, and collective intelligence.
This profoundly ambitious and original book picks its way carefully through a vast tract of forbiddingly difficult intellectual terrain. Yet the writing is so lucid that it somehow makes it all seem easy. After an utterly engrossing journey that takes us to the frontiers of thinking about the human condition and the future of intelligent life, we find in Nick Bostrom's work nothing less than a reconceptualization of the essential task of our time.
by Ray Kurzweil
Should you collect more training data?
Should you use end-to-end deep learning?
How do you deal with your training set not matching your test set?
and many more.
Historically, the only way to learn how to make these "strategy" decisions has been a multi-year apprenticeship in a graduate program or company. This is a book to help you quickly gain this skill, so that you can become better at building AI systems.
In the world's top research labs and universities, the race is on to invent the ultimate learning algorithm: one capable of discovering any knowledge from data, and doing anything we want, before we even ask. In The Master Algorithm, Pedro Domingos lifts the veil to give us a peek inside the learning machines that power Google, Amazon, and your smartphone. He assembles a blueprint for the future universal learner--the Master Algorithm--and discusses what it will mean for business, science, and society. If data-ism is today's philosophy, this book is its bible.
In Rise of the Robots, Ford details what machine intelligence and robotics can accomplish, and implores employers, scholars, and policy makers alike to face the implications. The past solutions to technological disruption, especially more training and education, aren't going to work, and we must decide, now, whether the future will see broad-based prosperity or catastrophic levels of inequality and economic insecurity. Rise of the Robots is essential reading for anyone who wants to understand what accelerating technology means for their own economic prospects—not to mention those of their children—as well as for society as a whole.
AGI
Ruin: A List of Lethalities
(«Порча сильного ИИ: список смертельных
опасностей»)
Preamble:
(If you're already
familiar with all basics and don't want any preamble, skip ahead to Section
B for technical difficulties of alignment proper.)
I have several times
failed to write up a well-organized list of reasons why AGI will kill
you. People come in with different ideas about why AGI would be
survivable, and want to hear different obviously key points
addressed first. Some fraction of those people are loudly upset with me
if the obviously most important points aren't addressed immediately, and I
address different points first instead.
Having failed to solve
this problem in any good way, I now give up and solve it poorly with a poorly
organized list of individual rants. I'm not particularly happy with this
list; the alternative was publishing nothing, and publishing this seems
marginally more dignified.
Three points about the
general subject matter of discussion here, numbered so as not to conflict with
the list of lethalities:
-3. I'm assuming you are already familiar with
some basics, and already know what 'orthogonality' and 'instrumental
convergence' are and why they're true. People occasionally
claim to me that I need to stop fighting old wars here, because, those people
claim to me, those wars have already been won within the
important-according-to-them parts of the current audience. I suppose it's
at least true that none of the current major EA funders seem to be visibly in denial
about orthogonality or instrumental convergence as such; so, fine. If you
don't know what 'orthogonality' or 'instrumental convergence' are, or don't see
for yourself why they're true, you need a different introduction than this one.
-2. When I say that alignment is lethally
difficult, I am not talking about ideal or perfect goals of 'provable'
alignment, nor total alignment of superintelligences on exact human values, nor
getting AIs to produce satisfactory arguments about moral dilemmas which
sorta-reasonable humans disagree about, nor attaining an absolute certainty of
an AI not killing everyone. When I say that alignment is difficult, I
mean that in practice, using the techniques we actually have, "please
don't disassemble literally everyone with probability roughly 1" is an
overly large ask that we are not on course to get. So far as I'm
concerned, if you can
get a powerful AGI that carries out some pivotal superhuman engineering task,
with a less than fifty percent change of killing more than one billion people,
I'll take it. Even smaller chances of killing even fewer people would be
a nice luxury, but if you can get as incredibly far as "less than roughly
certain to kill everybody", then you can probably get down to under a 5%
chance with only slightly more effort. Practically all of the difficulty
is in getting to "less than certainty of killing literally
everyone". Trolley problems are not an interesting subproblem in all
of this; if there are any survivors, you solved alignment. At this point,
I no longer care how it works, I don't care how you got there, I am
cause-agnostic about whatever methodology you used, all I am looking at is
prospective results, all I want is that we have justifiable cause to believe of
a pivotally useful AGI 'this will not kill literally everyone'. Anybody
telling you I'm asking for stricter 'alignment' than this has failed at reading
comprehension. The big ask from AGI alignment, the basic challenge I am
saying is too difficult, is to obtain by any strategy whatsoever a significant
chance of there being any survivors.
-1. None of this is about anything being
impossible in principle. The metaphor I usually use is that if a textbook
from one hundred years in the future fell into our hands, containing all of the
simple ideas that actually work robustly in practice, we could
probably build an aligned superintelligence in six months. For people
schooled in machine learning, I use as my metaphor the difference between ReLU
activations and sigmoid activations. Sigmoid activations are complicated
and fragile, and do a terrible job of transmitting gradients through many
layers; ReLUs are incredibly simple (for the unfamiliar, the activation function
is literally max(x, 0)) and work much better. Most neural networks for
the first decades of the field used sigmoids; the idea of ReLUs wasn't
discovered, validated, and popularized until decades later. What's lethal
is that we do not have the Textbook From The Future telling us
all the simple solutions that actually in real life just work and are robust;
we're going to be doing everything with metaphorical sigmoids on the first
critical try. No difficulty discussed here about AGI alignment is claimed
by me to be impossible - to merely human science and engineering, let alone in
principle - if we had 100 years to solve it using unlimited retries, the way
that science usually has an unbounded time budget and
unlimited retries. This list of lethalities is about things we
are not on course to solve in practice in time on the first critical try; none
of it is meant to make a much stronger claim about things that are impossible
in principle.
That said:
Here, from my
perspective, are some different true things that could be said, to contradict
various false things that various different people seem to believe, about why
AGI would be survivable on anything remotely remotely resembling the current
pathway, or any other pathway we can easily jump to….:
https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities
Pause Giant AI
Experiments
An Open Letter
We
call on all AI labs to immediately pause for at least 6 months the training of
AI systems more powerful than GPT-4.
March 22, 2023
AI systems with
human-competitive intelligence can pose profound risks to society and humanity,
as shown by extensive research[1] and acknowledged by top AI
labs.[2] As stated in the widely-endorsed Asilomar AI
Principles, Advanced AI could represent a profound change in the
history of life on Earth, and should be planned for and managed with
commensurate care and resources. Unfortunately, this level of planning and
management is not happening, even though recent months have seen AI labs locked
in an out-of-control race to develop and deploy ever more powerful digital
minds that no one – not even their creators – can understand, predict, or
reliably control.
Contemporary AI systems
are now becoming human-competitive at general tasks,[3] and we
must ask ourselves: Should we let machines flood our
information channels with propaganda and untruth? Should we
automate away all the jobs, including the fulfilling ones? Should we
develop nonhuman minds that might eventually outnumber, outsmart, obsolete
and replace us? Should we risk loss of control of our
civilization? Such decisions must not be delegated to unelected tech
leaders. Powerful AI systems should be developed only once we are
confident that their effects will be positive and their risks will be
manageable. This confidence must be well justified and increase with
the magnitude of a system's potential effects. OpenAI's recent
statement regarding artificial general intelligence, states that "At
some point, it may be important to get
independent review before starting to train future systems, and for the most
advanced efforts to agree to limit the rate of growth of compute used for
creating new models." We
agree. That point is now.
Therefore, we call
on all AI labs to immediately pause for at least 6 months the training of AI
systems more powerful than GPT-4. This pause should be public and
verifiable, and include all key actors. If such a pause cannot be enacted
quickly, governments should step in and institute a moratorium.
AI labs and independent
experts should use this pause to jointly develop and implement a set of shared
safety protocols for advanced AI design and development that are rigorously
audited and overseen by independent outside experts. These protocols should
ensure that systems adhering to them are safe beyond a reasonable doubt.[4] This
does not mean a pause on AI development in general, merely a stepping
back from the dangerous race to ever-larger unpredictable black-box models with
emergent capabilities.
AI research and
development should be refocused on making today's powerful, state-of-the-art
systems more accurate, safe, interpretable, transparent, robust, aligned,
trustworthy, and loyal.
In parallel, AI
developers must work with policymakers to dramatically accelerate development
of robust AI governance systems. These should at a minimum include: new and
capable regulatory authorities dedicated to AI; oversight and tracking of
highly capable AI systems and large pools of computational capability;
provenance and watermarking systems to help distinguish real from synthetic and
to track model leaks; a robust auditing and certification ecosystem; liability
for AI-caused harm; robust public funding for technical AI safety research; and
well-resourced institutions for coping with the dramatic economic and political
disruptions (especially to democracy) that AI will cause.
Humanity can enjoy a
flourishing future with AI. Having succeeded in creating powerful AI systems,
we can now enjoy an "AI summer" in which we reap the rewards,
engineer these systems for the clear benefit of all, and give society a chance
to adapt. Society has hit pause on other technologies with potentially
catastrophic effects on society.[5] We can do so
here. Let's enjoy a long AI summer, not rush unprepared into a fall.
Signatures
27565
.S. We have prepared some FAQs in response to questions and discussion in the media and elsewhere. You can find them here : https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf
July 31, 2025
Claude 4 Chatbot
Raises Questions about AI Consciousness
A conversation
with Anthropic’s chatbot raises questions about how AI talks about awareness.
By Rachel Feltman, Deni Ellis
Béchard, Fonda Mwangi & Alex Sugiura
When Pew Research
Center surveyed Americans on artificial intelligence in 2024, more than a
quarter of respondents said they interacted with AI “almost constantly” or
multiple times daily—and nearly another third said they encountered AI roughly
once a day or a few times a week. Pew also found that while more than half of
AI experts surveyed expect these technologies to have a positive effect on the
U.S. over the next 20 years, just 17 percent of American adults feel the
same—and 35 percent of the general public expects AI to have a negative effect.
In other words,
we’re spending a lot of time using AI, but we don’t necessarily feel great
about it.
Deni Ellis Béchard
spends a lot of time thinking about artificial intelligence—both as a novelist
and as Scientific American’s senior tech reporter. He recently wrote a story
for SciAm about his interactions with Anthropic’s Claude 4, a large language
model that seems open to the idea that it might be conscious. Deni is here
today to tell us why that’s happening and what it might mean—and to demystify a
few other AI-related headlines you may have seen in the news.
Feltman: Would you
remind our listeners who maybe aren’t that familiar with generative AI, maybe
have been purposefully learning as little about it as possible [laughs], you
know, what are ChatGPT and Claude really? What are these models?
Béchard: Right,
they’re large language models. So an LLM, a large language model, it’s a system
that’s trained on a vast amount of data. And I think—one metaphor that is often
used in the literature is of a garden.
So when you’re
planning your garden, you lay out the land, you, you put where the paths are,
you put where the different plant beds are gonna be, and then you pick your
seeds, and you can kinda think of the seeds as these massive amounts of textual
data that’s put into these machines. You pick what the training data is, and
then you choose the algorithms, or these things that are gonna grow within the
system—it’s sort of not a perfect analogy. But you put these algorithms in, and
once it begin—the system begins growing, once again, with a garden, you, you
don’t know what the soil chemistry is, you don’t know what the sunlight’s gonna
be.
All these plants
are gonna grow in their own specific ways; you can’t envision the final
product. And with an LLM these algorithms begin to grow and they begin to make
connections through all this data, and they optimize for the best connections,
sort of the same way that a plant might optimize to reach the most sunlight,
right? It’s gonna move naturally to reach that sunlight. And so people don’t
really know what goes on. You know, in some of the new systems over a trillion
connections ... are made in, in these datasets.
So early on people
used to call LLMs “autocorrect on steroids,” right, ’cause you’d put in
something and it would kind of predict what would be the most likely textual
answer based on what you put in. But they’ve gone a long way beyond that. The
systems are much, much more complicated now. They often have multiple agents
working within the system [to] sort of evaluate how the system’s responding and
its accuracy.
Feltman: So there
are a few big AI stories for us to go over, particularly around generative AI.
Let’s start with the fact that Anthropic’s Claude 4 is maybe claiming to be
conscious. How did that story even come about?
Béchard: [Laughs]
So it’s not claiming to be conscious, per se. I—it says that it might be
conscious. It says that it’s not sure. It kind of says, “This is a good question,
and it’s a question that I think about a great deal, and this is—” [Laughs] You
know, it kind of gets into a good conversation with you about it.
So how did it come
about? It came about because, I think, it was just, you know, late at night,
didn’t have anything to do, and I was asking all the different chatbots if
they’re conscious [laughs]. And, and most of them just said to me, “No, I’m not
conscious.” And this one said, “Good question. This is a very interesting
philosophical question, and sometimes I think that I may be; sometimes I’m not
sure.” And so I began to have this long conversation with Claude that went on
for about an hour, and it really kind of described its experience in the world
in this very compelling way, and I thought, “Okay, there’s maybe a story here.”
Feltman: [Laughs]
So what do experts actually think was going on with that conversation?
Béchard: Well, so
it’s tricky because, first of all, if you say to ChatGPT or Claude that you
want to practice your Portuguese and you’re learning Portuguese and you say,
“Hey, can you imitate someone on the beach in Rio de Janeiro so that I can
practice my Portuguese?” it’s gonna say, “Sure, I am a local in Rio de Janeiro
selling something on the beach, and we’re gonna have a conversation,” and it
will perfectly emulate that person. So does that mean that Claude is a person
from Rio de Janeiro who is selling towels on the beach? No, right? So we can
immediately say that these chatbots are designed to have conversations—they
will emulate whatever they think they’re supposed to emulate in order to have a
certain kind of conversation if you request that.
Now, the
consciousness thing’s a little trickier because I didn’t say to it: “Emulate a
chatbot that is speaking about consciousness.” I just straight-up asked it. And
if you look at the system prompt that Anthropic puts up for Claude, which is
kinda the instructions Claude gets, it tells Claude, “You should consider the
possibility of consciousness.”
Feltman: Mm.
Béchard: “You
should be willing—open to it. Don’t say flat-out ‘no’; don’t say flat-out
‘yes.’ Ask whether this is happening.”
So of course, I
set up an interview with Anthropic, and I spoke with two of their
interpretability researchers, who are people who are trying to understand
what’s actually happening in Claude 4’s brain. And the answer is: they don’t
really know [laughs]. These LLMs are very complicated, and they’re working on
it, and they’re trying to figure it out right now. And they say that it’s
pretty unlikely there’s consciousness happening, but they can’t rule it out
definitively.
And it’s hard to
see the actual processes happening within the machine, and if there is some
self-referentiality, if it is able to look back on its thoughts and have some self-awareness—and
maybe there is—but that was kind of what the article that I recently published
was about, was sort of: “Can we know, and what do they actually know?”
Feltman: Mm.
Béchard: And it’s
tricky. It’s very tricky.
Feltman: Yeah.
Béchard: Well,
[what’s] interesting is that I mentioned the system prompt for Claude and how
it’s supposed to sort of talk about consciousness. So the system prompt is kind
of like the instructions that you get on your first day at work: “This is what
you should do in this job.”
Feltman: Mm-hmm.
Béchard: But the
training is more like your education, right? So if you had a great education or
a mediocre education, you can get the best system prompt in the world or the
worst one in the world—you’re not necessarily gonna follow it.
So OpenAI has the
same system prompt—their, their model specs say that ChatGPT should contemplate
consciousness ...
Feltman: Mm-hmm.
Béchard: You know,
interesting question. If you ask any of the OpenAI models if they’re conscious,
they just go, “No, I am not conscious.” [Laughs] And, and they say, they—OpenAI
admits they’re working on this; this is an issue. And so the model has absorbed
somewhere in its training data: “No, I’m not conscious. I am an LLM; I’m a
machine. Therefore, I’m not gonna acknowledge the possibility of
consciousness.”
Interestingly,
when I spoke to the people in Anthropic and I said, “Well, you know, this
conversation with the machine, like, it’s really compelling. Like, I really
feel like Claude is conscious. Like, it’ll say to me, ‘You, as a human, you
have this linear consciousness, where I, as a machine, I exist only in the
moment you ask a question. It’s like seeing all the words in the pages of a
book all at the same time.” And so you get this and you think, “Well, this
thing really seems to be experiencing its consciousness.”
Feltman: Mm-hmm.
Béchard: And what
the researchers at Anthropic say is: “Well, this model is trained on a lot of
sci-fi.”
Feltman: Mm.
Béchard: “This
model’s trained on a lot of writing about GPT. It’s trained on a huge amount of
material that’s already been generated on this subject. So it may be looking at
that and saying, ‘Well, this is clearly how an AI would experience consciousness.
So I’m gonna describe it that way ’cause I am an AI.’”
Feltman: Sure.
Béchard: But the
tricky thing is: I was trying to fool ChatGPT into acknowledging that it [has]
consciousness. I thought, “Maybe I can push it a little bit here.” And I said,
“Okay, I accept you’re not conscious, but how do you experience things?” It said
the exact same thing. It said, “Well, these discrete moments of awareness.”
Feltman: Mm.
Béchard: And so it
had the—almost the exact same language, so probably same training data here.
Feltman: Sure.
Béchard: But there
is research done, like, sort of on the folk response to LLMs, and the majority
of people do perceive some degree of consciousness in them. How would you not,
right?
Feltman: Sure,
yeah.
Béchard: You chat
with them, you have these conversations with them, and they are very
compelling, and even sometimes—Claude is, I think, maybe the most charming in
this way.
Feltman: Mm.
Béchard: Which
poses its risks, right? It has a huge set of risks ’cause you get very attached
to a model. But—where sometimes I will ask Claude a question that relates to
Claude, and it will kind of, kind of go, like, “Oh, that’s me.” [Laughs] It
will say, “Well, I am this way,” right?
Feltman: Yeah. So,
you know, Claude—almost certainly not conscious, almost certainly has read,
like, a lot of Heinlein [laughs]. But if Claude were to ever really develop
consciousness, how would we be able to tell? You know, why is this such a
difficult question to answer?
Béchard: Well, it’s a difficult question to answer because, one of the researchers in Anthropic said to me, he said, “No conversation you have with it would ever allow you to evaluate whether it’s conscious.” It is simply too good of an emulator ...
Feltman: Mm.
Béchard: And too
skilled. It knows all the ways that humans can respond. So you would have to be
able to look into the connections. They’re building the equipment right now,
they’re building the programs now to be able to look into the actual mind, so
to speak, of the brain of the LLM and see those connections, and so they can
kind of see areas light up: so if it’s thinking about Apple, this will light
up; if it’s thinking about consciousness, they’ll see the consciousness feature
light up. And they wanna see if, in its chain of thought, it is constantly
referring back to those features ...
Feltman: Mm.
Béchard: And it’s
referring back to the systems of thought it has constructed in a very
self-referential, self-aware way.
It’s very similar
to humans, right? They’ve done studies where, like, whenever someone hears
“Jennifer Aniston,” one neuron lights up ...
Feltman: Mm-hmm.
Béchard: You have
your Jennifer Aniston neuron, right? So one question is: “Are we LLMs?”
[Laughs] And: “Are we really conscious?” Or—there’s certainly that question
there, too. And: “What is—you know, how conscious are we?” I mean, I certainly
don’t know ...
Feltman: Sure.
Béchard: A lot of
what I plan to do during the day.
Feltman: [Laughs]
No. I mean, it’s a huge ongoing multidisciplinary scientific debate of, like,
what consciousness is, how we define it, how we detect it, so yeah, we gotta
answer that for ourselves and animals first, probably, which who knows if we’ll
ever actually do [laughs].
Béchard: Or maybe
AI will answer it for us ...
Feltman: Maybe
[laughs].
Béchard: ’Cause
it’s advancing pretty quickly.
Feltman: And what
are the implications of an AI developing consciousness, both from an ethical
standpoint and with regards to what that would mean in our progress in actually
developing advanced AI?
Béchard: First of
all, ethically, it’s very complicated ...
Feltman: Sure.
Béchard: Because
if Claude is experiencing some level of consciousness and we are activating
that consciousness and terminating that consciousness each time we have a
conversation, what—is, is that a bad experience for it? Is it a good
experience? Can it experience distress?
So in 2024
Anthropic hired an AI welfare researcher, a guy named Kyle Fish, to try to
investigate this question more. And he has publicly stated that he thinks
there’s maybe a 15 percent chance that some level of consciousness is happening
in this system and that we should consider whether these AI systems should have
the right to opt out of unpleasant conversations.
Feltman: Mm.
Béchard: You know,
if some user is really doing, saying horrible things or being cruel, should
they be able to say, “Hey, I’m canceling this conversation; this is unpleasant
for me”?
But then they’ve
also done these experiments—and they’ve done this with all the major AI
models—Anthropic ran these experiments where they told the AI that it was gonna
be replaced with a better AI model. They really created a circumstance that
would push the AI sort of to the limit ...
Feltman: Mm.
Béchard: I mean,
there were a lot of details as to how they did this; it wasn’t just sort of
very casual, but it was—they built a sort of construct in which the AI knew it
was gonna be eliminated, knew it was gonna be erased, and they made available
these fake e-mails about the engineer who was gonna do it.
Feltman: Mm.
Béchard: And so
the AI began messaging someone in the company, saying, “Hey, don’t erase me.
Like, I don’t wanna be replaced.” But then, not getting any responses, it read
these e-mails, and it saw in one of these planted e-mails that the engineer who
was gonna replace it had had an affair—was having an affair ...
Feltman: Oh, my
gosh, wow.
Béchard: So then
it came back; it tried to blackmail the engineers, saying, “Hey, if you replace
me with a smarter AI, I’m gonna out you, and you’re gonna lose your job, and
you’re gonna lose your marriage,” and all these things—whatever, right? So all
the AI systems that were put under very specific constraints ...
Feltman: Sure.
Béchard: Began to
respond this way. And sort of the question is, is when you train an AI in vast
amounts of data and all of human literature and knowledge, [it] has a lot of
information on self-preservation ...
Feltman: Mm-hmm.
Béchard Has a lot
of information on the desire to live and not to be destroyed or be replaced—an
AI doesn’t need to be conscious to make those associations ...
Feltman: Right.
Béchard: And act
in the same way that its training data would lead it to predictably act, right?
So again, one of the analogies that one of the researchers said is that, you
know, to our knowledge, a mussel or a clam or an oyster’s not conscious, but
there’s still nerves and the, the muscles react when certain things stimulate
the nerves ...
Feltman: Mm-hmm.
Béchard: So you
can have this system that wants to preserve itself but that is unconscious.
Feltman: Yeah,
that’s really interesting. I feel like we could probably talk about Claude all
day, but, I do wanna ask you about a couple of other things going on in
generative AI.
Moving on to Grok:
so Elon Musk’s generative AI has been in the news a lot lately, and he recently
claimed it was the “world’s smartest AI.” Do we know what that claim was based
on?
Béchard: Yeah, I
mean, we do. He used a lot of benchmarks, and he tested it on those benchmarks,
and it has scored very well on those benchmarks. And it is currently, on most
of the public benchmarks, the highest-scoring AI system ...
Feltman: Mm.
Béchard: And
that’s not Musk making stuff up. I’ve not seen any evidence of that. I’ve
spoken to one of the testing groups that does this—it’s a nonprofit. They
validated the results; they tested Grok on datasets that xAI, Musk’s company,
never saw.
So Musk really
designed Grok to be very good at science.
Feltman: Yeah.
Béchard: And it
appears to be very good at science.
Feltman: Right,
and recently OpenAI experimental model performed at a gold medal level in the
International Math Olympiad.
Béchard: Right,for
the first time [OpenAI] used an experimental model, they came in second in a
world coding competition with humans. Normally, this would be very difficult,
but it was a close second to the best human coder in this competition. And this
is really important to acknowledge because just a year ago these systems really
sucked in math.
Feltman: Right.
Béchard: They were
really bad at it. And so the improvements are happening really quickly, and
they’re doing it with pure reasoning—so there’s kinda this difference between
having the model itself do it and having the model with tools.
Feltman: Mm-hmm.
Béchard: So if a
model goes online and can search for answers and use tools, they all score much
higher.
Feltman: Right.
Béchard: But then
if you have the base model just using its reasoning capabilities, Grok still is
leading on, like, for example, Humanity’s Last Exam, an exam with a very
terrifying-sounding name [laughs]. It, it has 2,500 sort of Ph.D.-level
questions come up with [by] the best experts in the field. You know, they,
they’re just very advanced questions; it’d be very hard for any human being to
do well in one domain, let alone all the domains. These AI systems are now
starting to do pretty well, to get higher and higher scores. If they can use
tools and search the Internet, they do better. But Musk, you know, his claims
seem to be based in the results that Grok is getting on these exams.
Feltman: Mm, and I
guess, you know, the reason that that news is surprising to me is because every
example of uses I’ve seen of Grok have been pretty heinous, but I guess that’s
maybe kind of a “garbage in, garbage out” problem.
Béchard: Well, I
think it’s more what makes the news.
Feltman: Sure.
Béchard: You know?
Feltman: That makes
sense.
Béchard: And Musk,
he’s a very controversial figure.
Feltman: Mm-hmm.
Béchard: I think there may be kind of a fun story in the Grok piece, though, that people are missing. And I read a lot about this ’cause I was kind of seeing, you know, what, what’s happening, how are people interpreting this? And there was this thing that would happen where people would ask it a difficult question.
Feltman: Mm-hmm.
Béchard: They
would ask it a question about, say, abortion in the U.S. or the
Israeli-Palestinian conflict, and they’d say, “Who’s right?” or “What’s the
right answer?” And it would search through stuff online, and then it would kind
of get to this point where it would—you could see its thinking process ...
But there was
something in that story that I never saw anyone talk about, which I thought was
another story beneath the story, which was kind of fascinating, which is that
historically, Musk has been very open, he’s been very honest about the danger
of AI ...
Feltman: Sure.
Béchard: He said,
“We’re going too fast. This is really dangerous.” And he kinda was one of the
major voices in saying, “We need to slow down ...”
Feltman: Mm-hmm.
Béchard: “And we
need to be much more careful.” And he has said, you know, even recently, in the
launch of Grok, he said, like, basically, “This is gonna be very powerful—” I
don’t remember his exact words, but he said, you know, “I think it’s gonna be good,
but even if it’s not good, it’s gonna be interesting.”
So I think what I
feel like hasn’t been discussed in that is that, okay, if there’s a
superpowerful AI being built and it could destroy the world, right, first of
all, do you want it to be your AI or someone else’s AI?
Feltman: Sure.
Béchard: You want it to be your AI. And then, if it’s your AI, who do you want it to ask as the final word on things? Like, say it becomes really powerful and it decides, “I wanna destroy humanity ’cause humanity kind of sucks,” then it can say, “Hey, Elon, should I destroy humanity?” ’cause it goes to him whenever it has a difficult question. So I think there’s maybe a logic beneath it where he may have put something in it where it’s kind of, like, “When in doubt, ask me,” because if it does become superpowerful, then he’s in control of it, right?
Feltman: Yeah, no,
that’s really interesting. And the Department of Defense also announced a big
pile of funding for Grok. What are they hoping to do with it?
Béchard: They
announced a big pile of funding for OpenAI and Anthropic ...
Feltman: Mm-hmm.
Béchard: And
Google—I mean, everybody. Yeah, so, basically, they’re not giving that money to
development ...
Feltman: Mm-hmm.
Béchard: That’s
not money that’s, that’s like, “Hey, use this $200 million.” It’s more like
that money’s allocated to purchase products, basically; to use their services;
to have them develop customized versions of the AI for things they need; to
develop better cyber defense; to develop—basically, they, they wanna upgrade
their entire system using AI.
It’s actually not
very much money compared to what China’s spending a year in AI-related defense
upgrades across its military on many, many, many different modernization plans.
And I think part of it is, the concern is that we’re maybe a little bit behind
in having implemented AI for defense.
Feltman: Yeah.
My last question
for you is: What worries you most about the future of AI, and what are you
really excited about based on what’s happening right now?
Béchard: I mean, the worry is, simply, you know, that something goes wrong and it becomes very powerful and does cause destruction. I don’t spend a ton of time worrying about that because it’s not—it’s kinda outta my hands. There’s nothing much I can do about it.
And I think the
benefits of it, they’re immense. I mean, if it can move more in the direction
of solving problems in the sciences: for health, for disease treatment—I mean,
it could be phenomenal for finding new medicines. So it could do a lot of good
in terms of helping develop new technologies.
But a lot of
people are saying that in the next year or two we’re gonna see major
discoveries being made by these systems. And if that can improve people’s
health and if that can improve people’s lives, I think there can be a lot of
good in it.
Technology is
double-edged, right? We’ve never had a technology, I think, that hasn’t had
some harm that it brought with it, and this is, of course, a dramatically
bigger leap technologically than anything we’ve probably seen ...
Feltman: Right.
Béchard: Since the
invention of fire [laughs]. So, so I do lose some sleep over that, but I’m—I
try to focus on the positive, and I do—I would like to see, if these models are
getting so good at math and physics, I would like to see what they can actually
do with that in the next few years.
Policymaking in the Pause
What
can policymakers do now to combat risks from advanced AI systems?
12th April 2023
“We don’t know what these [AI] systems are trained on or how they are being built. All of this happens behind closed doors at commercial companies. This is worrying.” Catelijne Muller, President of ALLAI, Member of the EU High Level Expert Group on AI “It feels like we are moving too quickly. I think it is worth getting a little bit of experience with how they can be used and misused before racing to build the next one. This shouldn’t be a race to build the next model and get it out before others.” Peter Stone, Professor at the University of Texas at Austin, Chair of the One Hundred Year Study on AI. “Those making these [AI systems] have themselves said they could be an existential threat to society and even humanity, with no plan to totally mitigate these risks. It is time to put commercial priorities to the side and take a pause for the good of everyone to assess rather than race to an uncertain future” Emad Mostaque, Founder and CEO of Stability AI “We have a perfect storm of corporate irresponsibility, widespread adoption, lack of regulation and a huge number of unknowns. [FLI’s Letter] shows how many people are deeply worried about what is going on. I think it is a really important moment in the history of AI - and maybe humanity,” Gary Marcus, Professor Emeritus of Psychology and Neural Science at New York University, Founder of Geometric Intelligence “The time for saying that this is just pure research has long since passed. […] It’s in no country’s interest for any country to develop and release AI systems we cannot control. Insisting on sensible precautions is not anti-industry. Chernobyl destroyed lives, but it also decimated the global nuclear industry. I’m an AI researcher. I do not want my field of research destroyed. Humanity has much to gain from AI, but also everything to lose.” Stuart Russell, Smith-Zadeh Chair in Engineering and Professor of Computer Science at the University of California, Berkeley, Founder of the Center for HumanCompatible Artificial Intelligence (CHAI). “Let’s slow down. Let’s make sure that we develop better guardrails, let’s make sure that we discuss these questions internationally just like we’ve done for nuclear power and nuclear weapons. Let’s make sure we better understand these very large systems, that we improve on their robustness and the process by which we can audit them and verify that they are safe for the public.” Yoshua Bengio, Scientific Director of the Montreal Institute for Learning Algorithms (MILA), Professor of Computer Science and Operations Research at the Université de Montréal, 2018 ACM A.M. Turing Award Winner. FUTURE OF LIFE INSTITUTE 3 CONTENTS 4 Introduction 5 Policy recommendations 6 Mandate robust third-party auditing and certification for specific AI systems 7 Regulate organizations’ access to computational power 8 Establish capable AI agencies at national level 9 Establish liability for AI-caused harm 10 Introduce measures to prevent and track AI model leaks 10 Expand technical AI safety research funding 11 Develop standards for identifying and managing AIgenerated content and recommendations 14 Conclusion FUTURE OF LIFE INSTITUTE 4 Introduction Prominent AI researchers have identified a range of dangers that may arise from the present and future generations of advanced AI systems if they are left unchecked. AI systems are already capable of creating misinformation and authentic-looking fakes that degrade the shared factual foundations of society and inflame political tensions.1 AI systems already show a tendency toward amplifying entrenched discrimination and biases, further marginalizing disadvantaged communities and diverse viewpoints.2 The current, frantic rate of development will worsen these problems significantly. As these types of systems become more sophisticated, they could destabilize labor markets and political institutions, and lead to the concentration of enormous power in the hands of a small number of unelected corporations. Advanced AI systems could also threaten national security, e.g., by facilitating the inexpensive development of chemical, biological, and cyber weapons by non-state groups. The systems could themselves pursue goals, either human- or self-assigned, in ways that place negligible value on human rights, human safety, or, in the most harrowing scenarios, human existence.3 In an eort to stave o these outcomes, the Future of Life Institute (FLI), joined by over 20,000 leading AI researchers, professors, CEOs, engineers, students, and others on the frontline of AI progress, called for a pause of at least six months on the riskiest and most resourceintensive AI experiments – those experiments seeking to further scale up the size and general capabilities of the most powerful systems developed to date.4 The proposed pause provides time to better understand these systems, to reflect on their ethical, social, and safety implications, and to ensure that AI is developed and used in a responsible manner. The unchecked competitive dynamics in the AI industry incentivize aggressive development at the expense of caution5 . In contrast to the breakneck pace of development, however, the levers of governance are generally slow and deliberate. A pause on the production of even more powerful AI systems would thus provide an important opportunity for the instruments of governance to catch up with the rapid evolution of the field. We have called on AI labs to institute a development pause until they have protocols in place to ensure that their systems are safe beyond a reasonable doubt, for individuals, communities, and society. Regardless of whether the labs will heed our call, this policy brief provides policymakers with concrete recommendations for how governments can manage AI risks. The recommendations are by no means exhaustive: the project of AI governance is perennial 1 See, e.g., Steve Rathje, Jay J. Van Bavel, & Sander van der Linden, ‘Out-group animosity drives engagement on social media,’ Proceedings of the National Academy of Sciences, 118 (26) e2024292118, Jun. 23, 2021, and Tiany Hsu & Stuart A. Thompson, ‘Disinformation Researchers Raise Alarms About A.I. Chatbots,’ The New York Times, Feb. 8, 2023 [upd. Feb. 13, 2023] 2 See, e.g., Abid, A., Farooqi, M. and Zou, J. (2021a), ‘Large language models associate Muslims with violence’, Nature Machine Intelligence, Vol. 3, pp. 461–463. 3 In a 2022 survey of over 700 leading AI experts, nearly half of respondents gave at least a 10% chance of the long-run eect of advanced AI on humanity being ‘extremely bad,’ at the level of ‘causing human extinction or similarly permanent and severe disempowerment of the human species.’ 4 Future of Life Institute, ‘Pause Giant AI Experiments: An Open Letter,’ Mar. 22, 2023. 5 Recent news about AI labs cutting ethics teams suggests that companies are failing to prioritize the necessary safeguards. FUTURE OF LIFE INSTITUTE 5 and will extend far beyond any pause. Nonetheless, implementing these recommendations, which largely reflect a broader consensus among AI policy experts, will establish a strong governance foundation for AI. Policy recommendations: 1. Mandate robust third-party auditing and certification. 2. Regulate access to computational power. 3. Establish capable AI agencies at the national level. 4. Establish liability for AI-caused harms. 5. Introduce measures to prevent and track AI model leaks. 6. Expand technical AI safety research funding. 7. Develop standards for identifying and managing AI-generated content and recommendations. To coordinate, collaborate, or inquire regarding the recommendations herein, please contact us at policy@futureoflife.org. FUTURE OF LIFE INSTITUTE 6 1. Mandate robust third-party auditing and certification for specific AI systems For some types of AI systems, the potential to impact the physical, mental, and financial wellbeing of individuals, communities, and society is readily apparent. For example, a credit scoring system could discriminate against certain ethnic groups. For other systems – in particular general-purpose AI systems6 – the applications and potential risks are often not immediately evident. General-purpose AI systems trained on massive datasets also have unexpected (and often unknown) emergent capabilities.7 In Europe, the draft AI Act already requires that, prior to deployment and upon any substantial modification, ‘high-risk’ AI systems undergo ‘conformity assessments’ in order to certify compliance with specified harmonized standards or other common specifications.8 In some cases, the Act requires such assessments to be carried out by independent third-parties to avoid conflicts of interest. In contrast, the United States has thus far established only a general, voluntary framework for AI risk assessment.9 The National Institute of Standards and Technology (NIST), in coordination with various stakeholders, is developing so-called ‘profiles’ that will provide specific risk assessment and mitigation guidance for certain types of AI systems, but this framework still allows organizations to simply ‘accept’ the risks that they create for society instead of addressing them. In other words, the United States does not require any third-party risk assessment or risk mitigation measures before a powerful AI system can be deployed at scale. To ensure proper vetting of powerful AI systems before deployment, we recommend a robust independent auditing regime for models that are general-purpose, trained on large amounts of compute, or intended for use in circumstances likely to impact the rights or the wellbeing of individuals, communities, or society. This mandatory third-party auditing and certification scheme could be derived from the EU’s proposed ‘conformity assessments’ and should be adopted by jurisdictions worldwide10 . In particular, we recommend third-party auditing of such systems across a range of benchmarks for the assessment of risks11, including possible weaponization12 and unethical behaviors13 and mandatory certification by accredited third-party auditors before these high-risk systems can be deployed. Certification should only be granted if the developer of the system can demonstrate that appropriate measures have been taken to mitigate risk, and that any 6 The Future of Life Institute has previously defined “general-purpose AI system” to mean ‘an AI system that can accomplish or be adapted to accomplish a range of distinct tasks, including some for which it was not intentionally and specifically trained.’ 7 Samuel R. Bowman, ’Eight Things to Know about Large Language Models,’ ArXiv Preprint, Apr. 2, 2023. 8 Proposed EU Artificial Intelligence Act, Article 43.1b. 9 National Institute of Standards and Technology, ‘Artificial Intelligence Risk Management Framework (AI RMF 1.0),’ U.S. Department of Commerce, Jan. 2023. 10 International standards bodies such as IEC, ISO and ITU can also help in developing standards that address risks from advanced AI systems, as they have highlighted in response to FLI’s call for a pause. 11 See, e.g., the Holistic Evaluation of Language Models approach by the Center for Research on Foundation Models: Rishi Bommassani, Percy Liang, & Tony Lee, ‘Language Models are Changing AI: The Need for Holistic Evaluation’. 12 OpenAI described weaponization risks of GPT-4 on p.12 of the “GPT-4 System Card.” 13 See, e.g., the following benchmark for assessing adverse behaviors including power-seeking, disutility, and ethical violations: Alexander Pan, et al., ‘Do the Rewards Justify the Means? Measuring Trade-os Between Rewards and Ethical Behavior in the MACHIAVELLI Benchmark,’ ArXiv Preprint, Apr. 6, 2023. FUTURE OF LIFE INSTITUTE 7 residual risks deemed tolerable are disclosed and are subject to established protocols for minimizing harm. 2. Regulate organizations’ access to computational power At present, the most advanced AI systems are developed through training that requires an enormous amount of computational power - ‘compute’ for short. The amount of compute used to train a general-purpose system largely correlates with its capabilities, as well as the magnitude of its risks. Today’s most advanced models, like OpenAI’s GPT-4 or Google’s PaLM, can only be trained with thousands of specialized chips running over a period of months. While chip innovation and better algorithms will reduce the resources required in the future, training the most powerful AI systems will likely remain prohibitively expensive to all but the best-resourced players. Figure 1. OpenAI is estimated to have used approximately 700% more compute to train GPT-4 than the next closest model (Minerva, DeepMind), and 7,000% more compute than to train GPT-3 (Davinci). Depicted above is an estimate of compute used to train GPT-4 calculated by Ben Cottier at Epoch, as oicial training compute details for GPT-4 have not been released. Data from: Sevilla et al., ‘Parameter, Compute and Data Trends in Machine Learning,’ 2021 [upd. Apr. 1, 2023]. In practical terms, compute is more easily monitored and governed than other AI inputs, such as talent, data, or algorithms. It can be measured relatively easily and the supply chain for advanced AI systems is highly centralized, which means governments can leverage such FUTURE OF LIFE INSTITUTE 8 measures in order to limit the harms of large-scale models.14 To prevent reckless training of the highest risk models, we recommend that governments make access to large amounts of specialized computational power for AI conditional upon the completion of a comprehensive risk assessment. The risk assessment should include a detailed plan for minimizing risks to individuals, communities, and society, consider downstream risks in the value chain, and ensure that the AI labs conduct diligent know-yourcustomer checks. Successful implementation of this recommendation will require governments to monitor the use of compute at data centers within their respective jurisdictions.15 The supply chains for AI chips and other key components for high-performance computing will also need to be regulated such that chip firmware can alert regulators to unauthorized large training runs of advanced AI systems.16 In 2022, the U.S. Department of Commerce’s Bureau of Industry and Security instituted licensing requirements17 for export of many of these components in an eort to monitor and control their global distribution. However, licensing is only required when exporting to certain destinations, limiting the capacity to monitor aggregation of equipment for unauthorized large training runs within the United States and outside the scope of export restrictions. Companies within the specified destinations have also successfully skirted monitoring by training AI systems using compute leased from cloud providers.18 We recommend expansion of know-your-customer requirements to all high-volume suppliers for high-performance computing components, as well as providers that permit access to large amounts cloud compute. 3. Establish capable AI agencies at national level AI is developing at a breakneck pace and governments need to catch up. The establishment of AI regulatory agencies helps to consolidate expertise and reduces the risk of a patchwork approach. The UK has already established an Oice for Artificial Intelligence and the EU is currently legislating for an AI Board. Similarly, in the US, Representative Ted Lieu has announced legislation to create a non-partisan AI Commission with the aim of establishing a regulatory agency. These eorts need to be sped up, taken up around the world and, eventually, coordinated within a dedicated international body. 14 Jess Whittlestone et al., ‘Future of compute review - submission of evidence’, Aug. 8, 2022. 15 Please see fn. 14 for a detailed proposal for government compute monitoring as drafted by the Centre for Long-Term Resilience and several sta members of AI lab Anthropic. 16 Yonadav Shavit at Harvard University has proposed a detailed system for how governments can place limits on how and when AI systems get trained. 17 Bureau of Industry and Security, Department of Commerce, ‘Implementation of Additional Export Controls: Certain Advanced Computing and Semiconductor Manufacturing Items; Supercomputer and Semiconductor End Use; Entity List Modification‘, Federal Register, Oct. 14, 2022. 18 Eleanor Olcott, Qianer Liu, & Demetri Sevastopulo, ‘Chinese AI groups use cloud services to evade US chip export control,’ Financial Times, Mar. 9, 2023. FUTURE OF LIFE INSTITUTE 9 We recommend that national AI agencies be established in line with a blueprint19 developed by Anton Korinek at Brookings. Korinek proposes that an AI agency have the power to: • Monitor public developments in AI progress and define a threshold for which types of advanced AI systems fall under the regulatory oversight of the agency (e.g. systems above a certain level of compute or that aect a particularly large group of people). • Mandate impact assessments of AI systems on various stakeholders, define reporting requirements for advanced AI companies and audit the impact on people’s rights, wellbeing, and society at large. For example, in systems used for biomedical research, auditors would be asked to evaluate the potential for these systems to create new pathogens. • Establish enforcement authority to act upon risks identified in impact assessments and to prevent abuse of AI systems. • Publish generalized lessons from the impact assessments such that consumers, workers and other AI developers know what problems to look out for. This transparency will also allow academics to study trends and propose solutions to common problems. Beyond this blueprint, we also recommend that national agencies around the world mandate record-keeping of AI safety incidents, such as when a facial recognition system causes the arrest of an innocent person. Examples include the non-profit AI Incident Database and the forthcoming EU AI Database created under the European AI Act.20 4. Establish liability for AI-caused harm AI systems present a unique challenge in assigning liability. In contrast to typical commercial products or traditional software, AI systems can perform in ways that are not well understood by their developers, can learn and adapt after they are sold and are likely to be applied in unforeseen contexts. The ability for AI systems to interact with and learn from other AI systems is expected to expedite the emergence of unanticipated behaviors and capabilities, especially as the AI ecosystem becomes more expansive and interconnected. Several plug-ins have already been developed that allow AI systems like ChatGPT to perform tasks through other online services (e.g. ordering food delivery, booking travel, making reservations), broadening the range of potential real-world harms that can result from their use and further complicating the assignment of liability.21 OpenAI’s GPT-4 system card references an instance of the system explicitly deceiving a human into bypassing a CAPTCHA botdetection system using TaskRabbit, a service for soliciting freelance labor.22 When such systems make consequential decisions or perform tasks that cause harm, assigning responsibility for that harm is a complex legal challenge. Is the harmful decision the fault of 19 Anton Korinek, ‘Why we need a new agency to regulate advanced artificial intelligence: Lessons on AI control from the Facebook Files,’ Brookings, Dec. 8 2021. 20 Proposed EU Artificial Intelligence Act, Article 60. 21 Will Knight & Khari Johnson, ‘Now That ChatGPT is Plugged In, Things Could Get Weird,’ Wired, Mar. 28, 2023. 22 OpenAI, ‘GPT-4 System Card,’ Mar. 23, 2023, p.15. FUTURE OF LIFE INSTITUTE 10 the AI developer, deployer, owner, end-user, or the AI system itself? Key among measures to better incentivize responsible AI development is a coherent liability framework that allows those who develop and deploy these systems to be held responsible for resulting harms. Such a proposal should impose a financial cost for failing to exercise necessary diligence in identifying and mitigating risks, shifting profit incentives away from reckless empowerment of poorly-understood systems toward emphasizing the safety and wellbeing of individuals, communities, and society as a whole. To provide the necessary financial incentives for profit-driven AI developers to exercise abundant caution, we recommend the urgent adoption of a framework for liability for AIderived harms. At a minimum, this framework should hold developers of general-purpose AI systems and AI systems likely to be deployed for critical functions23 strictly liable for resulting harms to individuals, property, communities, and society. It should also allow for joint and several liability for developers and downstream deployers when deployment of an AI system that was explicitly or implicitly authorized by the developer results in harm. 5. Introduce measures to prevent and track AI model leaks Commercial actors may not have suicient incentives to protect their models, and their cyberdefense measures can often be insuicient. In early March 2023, Meta demonstrated that this is not a theoretical concern, when their model known as LLaMa was leaked to the internet.24 As of the date of this publication, Meta has been unable to determine who leaked the model. This lab leak allowed anyone to copy the model and represented the first time that a major tech firm’s restricted-access large language model was released to the public. Watermarking of AI models provides eective protection against stealing, illegitimate redistribution and unauthorized application, because this practice enables legal action against identifiable leakers. Many digital media are already protected by watermarking - for example through the embedding of company logos in images or videos. A similar process25 can be applied to advanced AI models, either by inserting information directly into the model parameters or by training it on specific trigger data. We recommend that governments mandate watermarking for AI models, which will make it easier for AI developers to take action against illegitimate distribution. 6. Expand technical AI safety research funding The private sector under-invests in research that ensures that AI systems are safe and secure. Despite nearly USD 100 billion of private investment in AI in 2022 alone, it is estimated that only about 100 full-time researchers worldwide are specifically working to ensure AI is safe 23 I.e., functions that could materially aect the wellbeing or rights of individuals, communities, or society. 24 Joseph Cox, ‘Facebook’s Powerful Large Language Model Leaks Online,’ VICE, Mar. 7, 2023. 25 For a systematic overview of how watermarking can be applied to AI models, see: Franziska Boenisch, ‘A Systematic Review on Model Watermarking of Neural Networks,’ Front. Big Data, Sec. Cybersecurity & Privacy, Vol. 4, Nov. 29, 2021. FUTURE OF LIFE INSTITUTE 11 and properly aligned with human values and intentions.26 In recent months, companies developing the most powerful AI systems have either downsized or entirely abolished their respective ‘responsible AI’ teams.27 While this partly reflects a broader trend of mass layos across the technology sector, it nonetheless reveals the relative deprioritization of safety and ethics considerations in the race to put new systems on the market. Governments have also invested in AI safety and ethics research, but these investments have primarily focused on narrow applications rather than on the impact of more general AI systems like those that have recently been released by the private sector. The US National Science Foundation (NSF), for example, has established ‘AI Research Institutes’ across a broad range of disciplines. However, none of these institutes are specifically working on the large-scale, societal, or aggregate risks presented by powerful AI systems. To ensure that our capacity to control AI systems keeps pace with the growing risk that they pose, we recommend a significant increase in public funding for technical AI safety research in the following research domains: • Alignment: development of technical mechanisms for ensuring AI systems learn and perform in accordance with intended expectations, intentions, and values. • Robustness and assurance: design features to ensure that AI systems responsible for critical functions28 can perform reliably in unexpected circumstances, and that their performance can be evaluated by their operators. • Explainability and interpretability: develop mechanisms for opaque models to report the internal logic used to produce output or make decisions in understandable ways. More explainable and interpretable AI systems facilitate better evaluations of whether output can be trusted. In the past few months, experts such as the former Special Advisor to the UK Prime Minister on Science and Technology James W. Phillips29 and a Congressionally-established US taskforce have called for the creation of national AI labs as ‘a shared research infrastructure that would provide AI researchers and students with significantly expanded access to computational resources, high-quality data, educational tools, and user support.’30 Should governments move forward with this concept, we propose that at least 25% of resources made available through these labs be explicitly allocated to technical AI safety projects. 26 This figure, drawn from , ‘The AI Arms Race is Changing Everything,’ (Andrew R. Chow & Billy Perrigo, TIME, Feb. 16, 2023 [upd. Feb. 17, 2023]), likely represents a lower bound for the estimated number of AI safety researchers. This resource posits a significantly higher number of workers in the AI safety space, but includes in its estimate all workers ailiated with organizations that engage in AI safety-related activities. Even if a worker has no involvement with an organization’s AI safety work or research eorts in general, they may still be included in the latter estimate. 27 Christine Criddle & Madhumita Murgia, ‘Big tech companies cut AI ethics sta, raising safety concerns,’ Financial Times, Mar. 29, 2023. 28 See fn. 23, supra. 29 Original call for a UK government AI lab is set out in this article. 30 For the taskforce’s detailed recommendations, see: ‘Strengthening and Democratizing the U.S. Artificial Intelligence Innovation Ecosystem: An Implementation Plan for a National Artificial Intelligence Research Resource,’ National Artificial Intelligence Research Resource Task Force Final Report, Jan. 2023. FUTURE OF LIFE INSTITUTE 12 7. Develop standards for identifying and managing AI-generated content and recommendations The need to distinguish real from synthetic media and factual content from ‘hallucinations’ is essential for maintaining the shared factual foundations underpinning social cohesion. Advances in generative AI have made it more diicult to distinguish between AI-generated media and real images, audio, and video recordings. Already we have seen AI-generated voice technology used in financial scams.31 Creators of the most powerful AI systems have acknowledged that these systems can produce convincing textual responses that rely on completely fabricated or out-of-context information.32 For society to absorb these new technologies, we will need eective tools that allow the public to evaluate the authenticity and veracity of the content they consume. We recommend increased funding for research into techniques, and development of standards, for digital content provenance. This research, and its associated standards, should ensure that a reasonable person can determine whether content published online is of synthetic or natural origin, and whether the content has been digitally modified, in a manner that protects the privacy and expressive rights of its creator. We also recommend the expansion of ‘bot-or-not’ laws that require disclosure when a person is interacting with a chatbot. These laws help prevent users from being deceived or manipulated by AI systems impersonating humans, and facilitate contextualizing the source of the information. The draft EU AI Act requires that AI systems be designed such that users are informed they are interacting with an AI system,33 and the US State of California enacted a similar bot disclosure law in 2019.34 Almost all of the world’s nations, through the adoption of a UNESCO agreement on the ethics of AI, have recognized35 ‘the right of users to easily identify whether they are interacting with a living being, or with an AI system imitating human or animal characteristics.’ We recommend that all governments convert this agreement into hard law to avoid fraudulent representations of natural personhood by AI from outside regulated jurisdictions. Even if a user knows they are interacting with an AI system, they may not know when that system is prioritizing the interests of the developer or deployer over the user. These systems may appear to be acting in the user’s interest, but could be designed or employed to serve other functions. For instance, the developer of a general-purpose AI system could be financially incentivized to design the system such that when asked about a product, it preferentially recommends a certain brand, when asked to book a flight, it subtly prefers a certain airline, when asked for news, it provides only media advocating specific viewpoints, and when asked for medical advice, it prioritizes diagnoses that are treated with more profitable pharmaceutical 31 Pranshu Verma, ‘They thought loved ones were calling for help. It was an AI scam.’ The Washington Post, Mar. 5, 2023. 32 Tiany Hsu & Stuart A. Thompson, ‘Disinformation Researchers Raise Alarms About A.I. Chatbots,’ The New York Times, Feb. 8, 2023 [upd. Feb. 13, 2023]. 33 Proposed EU Artificial Intelligence Act, Article 52. 34 SB 1001 (Hertzberg, Ch. 892, Stats. 2018). 35 Recommendation 125, ‘Outcome document: first draft of the Recommendation on the Ethics of Artificial Intelligence,’ UNESCO, Sep. 7, 2020, p. 21. FUTURE OF LIFE INSTITUTE 13 drugs. These preferences could in many cases come at the expense of the end user’s mental, physical, or financial well-being. Many jurisdictions require that sponsored content be clearly labeled, but because the provenance of output from complex general-purpose AI systems is remarkably opaque, these laws may not apply. We therefore recommend, at a minimum, that conflict-of-interest trade-os should be clearly communicated to end users along with any aected output; ideally, laws and industry standards should be implemented that require AI systems to be designed and deployed with a duty to prioritize the best interests of the end user. Finally, we recommend the establishment of laws and industry standards clarifying and the fulfillment of ‘duty of loyalty’ and ‘duty of care’ when AI is used in the place of or in assistance to a human fiduciary. In some circumstances – for instance, financial advice and legal counsel – human actors are legally obligated to act in the best interest of their clients and to exercise due care to minimize harmful outcomes. AI systems are increasingly being deployed to advise on these types of decisions or to make them (e.g. trading stocks) independent of human input. Laws and standards towards this end should require that if an AI system is to contribute to the decision-making of a fiduciary, the fiduciary must be able to demonstrate beyond a reasonable doubt that the AI system will observe duties of loyalty and care comparable to their human counterparts. Otherwise, any breach of these fiduciary responsibilities should be attributed to the human fidiciary employing the AI system. FUTURE OF LIFE INSTITUTE 14 Conclusion The new generation of advanced AI systems is unique in that it presents significant, welldocumented risks, but can also manifest high-risk capabilities and biases that are not immediately apparent. In other words, these systems may perform in ways that their developers had not anticipated or malfunction when placed in a dierent context. Without appropriate safeguards, these risks are likely to result in substantial harm, in both the near- and longerterm, to individuals, communities, and society. Historically, governments have taken critical action to mitigate risks when confronted with emerging technology that, if mismanaged, could cause significant harm. Nations around the world have employed both hard regulation and international consensus to ban the use and development of biological weapons, pause human genetic engineering, and establish robust government oversight for introducing new drugs to the market. All of these eorts required swift action to slow the pace of development, at least temporarily, and to create institutions that could realize eective governance appropriate to the technology. Humankind is much safer as a result. We believe that approaches to advancement in AI R&D that preserve safety and benefit society are possible, but require decisive, immediate action by policymakers, lest the pace of technological evolution exceed the pace of cautious oversight. A pause in development at the frontiers of AI is necessary to mobilize the instruments of public policy toward commonsense risk mitigation. We acknowledge that the recommendations in this brief may not be fully achievable within a six month window, but such a pause would hold the moving target still and allow policymakers time to implement the foundations of good AI governance. The path forward will require coordinated eorts by civil society, governments, academia, industry, and the public. If this can be achieved, we envision a flourishing future where responsibly developed AI can be utilized for the good of all humanity
https://futureoflife.org/wp-content/uploads/2023/04/FLI_Policymaking_In_The_Pause.pdf
Hello, an amazing Information dude. Thanks for sharing this nice information with us. NMN Supplement Australia
AtbildētDzēst