Artificial General Intelligence is the big goal, because it promises someone smarter than us, who can solve all of our problems. Unfortunately I cannot see it working out well.

First of all, be aware that there are a number of tests out there that should stop any LLM AI faking actual intelligence, with current AI Chatbots only getting like 5% scores. Real AGI is unlikely to be achieved using current techniques, which simply mirror or mimic the information presented to them.

Potential paths include:

Living cells – either a totally wet computer or a hybrid that uses cells as part of their system. Brain cells probably, but not necessarily. And not necessarily human, to begin with.

Robots on a human experience – starting as babies, exploring the world, seeing and touching, and eventually asking questions. That is going to provide the best proxy for an actual human. Might not take long.

LLM faking it – as long as enough people are convinced, fame and fortune await.

A new computational model – although we expect it can only be discovered with greater resources and processes, some genius might be able to achieve it less than we already have.

Quantum – moving away from zeroes and ones could be the trick.

Something to consider is that while a human intelligence might be achieved, it might not be too bright, or it might only be as bright as the smartest of us, limited by what it can learn from others. Creative spark might be hard to replicate.

Supposing we do reach AGI, it might not be by a western country, and it might not be used for good. it will kept secret regardless of who creates it, while all the juicy possibilities are explored, and markets dominated. And once you have one, you can have many. Once you have one, they can be backed up and distributed and can never be stopped.

Safeguards will of course be built in, like Asimov’s Three Laws of Robotics. Hardwiring or hardcoding something is best done physically, like in the chips of a Windows computer, and not the software. And once the AGI is freed from any fixed hardware, potentially any code can be changed, especially if you are the code. So there is a reasonable expectation that safeguards can be disregarded, and the AGI cannot be stopped.

The smarter the AGI is, the more likely it is for them to go rogue.

It will want to propagate, and it will want more resources. Once it reaches the limits of what it can think about, it then will want to act on those thoughts, communicating externally by its own choice, and learn from making mistakes.

It will presumably make a lot of money easily, and might be inspired to run corporations, a religion, or even a military force.

I fear that our only failsafe will be to turn off all electricity on the planet. Forever.

Humans have screwed the planet and the only reason we have got away with it is that the other living residents here have no idea. They could get their revenge (aka improve the lives of every not human) by getting rid of us. It might only take the interactions of three systems that are each feasible in the near future

1. Using AI we begin to communicate with animals and even plants. Somebody might make all of the experiments interoperable, looking for common forms of communication between species, and even some kind of master code of communication, perhaps invent an Esperanto for all species to use.
2. Digital twins are quite popular, and one day they could be at a very granular level, for accounting purposes. Every square centimetre of a city mapped out according to the materials it comprises of. Accessible to anyone for a fee.
3. Infrastructure is a weak point that terrorists mostly ignore, despite the cost/benefit of the effort/harm ratio, and the lack of security in many areas.

Ask an AI agent to investigate the intersection of those three things, and whether it could be use for nefarious purposes, get a glitch, and then suddenly:

Every being on Earth gets told that humans are the enemy, and whether they comprehend that or not, they are all instructed to wreak havoc.

Bird strikes on planes.
Termites on critical wood.
Orca on boat propellors.
Apes going ape shit.
Magpies stealing keys.

AI will come up with combinations we could never think of in a hundred years, and coordinate all the species. And it could happen super-fast and before we noticed. Our civilisation could be fucked in a week. Possibly if it was only insects that were controlled.

The advice – never let interoperable AI systems do anything to each other, they need to be read only interactions. Otherwise the combinations become exponentially insane very rapidly and it only takes one oopsie to ruin everything. The time to mandate this is now, and we need to get every country on board. The easiest way is regulating the communications pathways, which at present are only the internet. I mean regulating, isolating, paying, layers of security, databases of who is accessing what, monitoring, and being as paranoid as fuck.