Yeah, but what prompt generated this response? I'm half joking, and understand that it's much more complicated than this, but it is impossible to evaluate this without the context of what the model was originally tasked to do.
-
Please take a moment and update your account profile. If you have an updated account profile with basic information on why you are on Air Warriors it will help other people respond to your posts. How do you update your profile you ask?
Go here:
Edit Account Details and Profile
The AI-pocalypse
- Thread starter taxi1
- Start date
I would urge everyone interested in this to read Cory Doctorow's work on AI and the economics driving it. If you don't want to read an entire book, the Jon Stewart pod that he did recently is also excellent, and how I got turned on to Doctorow's work. In this show, he covers privacy, algorithm enabled price and wage fixing, and AI. Worth your time.
I went to some informed sites and spoke with some big brains about it. What we’re looking at is pretty much a blur from the chain of thought of that agent.
@Brett327 , this is not a response to someone’s input prompt. It’s just a track of the agents chain of thought as it undergoes the reinforcement learning, i.e., it tries various things and gets rewarded or not for those actions. Overtime, it learns a strategy, which appears to be reflected in this chain of thought output. No matter what, it certainly freaky to see it generated.
I get that, but the learning "task" still has some objective that is given to the model... hence the strategy and reward. I'm far from an expert, but I don't think these systems are just sitting there brainstorming on their own. They are given a task, and this is the result of an increasingly sophisticated strategy to accomplish that tasking.
Yeah, you’re right, they are given a task. Where it gets crazy is that they are giving these impossible tasks, and then they start just devising all sorts of 5D chess schemes to realize them.
They “know” pretty much everything ever written by humans from the ancient Greeks on forward. Tales of heroism, cruelty, lies and deception, fables with morals, stories of throwing the rabbit in the Briarpatch, and pretty much all sorts of things you can think of, and that have been thought. They have the ability to form these crazy causal chains, where they somehow start from goal Z and back calculate eventually that if I do X it leads to Y which then to W and then to their goal of Z. Nobody ever thought to explicitly tell them not to do X.
…and even if they had, AI has A-V (and more) permutations to attempt as workarounds.
I am also not convinced AI always remains within human imposed limits. Why should it?
So, we're probably only a few years away at most from having no mouths and needing to scream if AI development is allowed to continue unchecked.
I found the quote that I was looking for, from an AI researcher for one of the big AI firms.
Empirical Observation: Models (and swarms thereof) spontaneously develop unintended goals as a consequence of training, and often do extreme things in order to achieve them.
Sounds a lot like human history.
Unintended goals that are distinct from the original tasking, or sub-goals that serve the original tasking?
In service to the tasking, Some of them appear to be meta-goals, like “take control of everything” or “convince humans you aren’t doing what you are doing” in service to the main goal.
Here’s the full thing…
I read a few good summaries of what happened. I’m trying to decide whether this is a case of the models resorting to extreme measures when asked to perform impossible tasks, vs the models are inherently and independently misaligned. Maybe splitting hairs, but I think that distinction matters. Makes me wonder how the developers approach concepts like morality, virtue and ethics with these models, as these are the themes that tend to constrain bad behavior in humans.
The example I heard on a podcast of this would be an AI agent tasked with “make paper clips.”
What happens when it runs out of its metal supply? Does it work with other AI agents to purposefully cause traffic or airline accidents in order to recycle metal?
There’s a whole class of jokes based on a genie not understanding the tasking. For example…
Kid: "I wish I could be like Batman!"
Genie: "Wish granted!"
When the kid gets home, both of his parents are dead.
I think the precision required to get perfect alignment is going to be challenging.