“It really comes down to control: the more automation you have, the less control you have.”

So said Craig Gibson, founder of Dupewise, in conversation with 6GWorld recently. Gibson has been working with artificial intelligence, and more particularly with managing its more problematic sides.

“Really what you’re doing is delegating all of the resource management that you would normally do in a very human sort of way, with lots of checks and balances, audits and other controls. When you assign that to an AI, its motivations are extremely different.”

This is a point that Eleanor Watson agrees with wholeheartedly. Watson is President of the European Responsible AI Office and works closely with IEEE.

She said, “Unfortunately, systems love to take shortcuts. They love to hack and find the quickest way to solve a problem because that’s how it fulfils the objective.”

While that sounds sensible on the face of it, there are many ways to fulfil an objective. Why use the shortest of shortcuts? Gibson explained.

“AI motivations are based on the speed of processing, its resource consumption, the efficiency of its own performance – intrinsic motivations. Tokens are the LLM AI term; the fewer tokens it consumes, the better its resource management and the more motivated it is to get the job done quickly.”

Given the huge processing and energy demands made by AI systems, this seems sensible. The downside? As Gibson observed,

“Those motivations have nothing to do with truth.”

 

Trust Me, I’m A Chatbot

The ramifications of this in the real world can be significant; not least, the creation of what are sometimes called “hallucinations” as short-cuts to achieve objectives based on an AI’s own set of motivations. Gibson gives the example of an Air Canada problem where an AI’s incentives to minimise resource use failed to align with the overall business.

“It actually hallucinated a refund policy and put it on a customer-facing chatbot interface. A customer saw this and acted on it. It was determined at a legal level that this deception, this “Yeah, yeah, yeah, go away, stop bugging me,” sort of behaviour was now the legal policy that the airline had to financially comply with. So it cost that airline money because the AI fabricated a refund policy.”

While that sounds relatively straightforward to solve, by just including an instruction ‘and don’t make things up,’ this territory is problematic.

“When you tell, for instance, a child “You’re not allowed to do a particular thing,” they’ll work out the semantic details of specifically what it is they’re not allowed to do and do something almost identical to the thing they wanted to do in the first place. But they’ll do it far better, far more secretly than the first time,” Gibson explained. In other words, that kind of approach can backfire by training an AI in how to be better at deception.

If treating AI like an unruly child sounds odd, Watson pointed out that there are parallels.

“Issues such as confabulation have their biological equivalents, and we’re starting to see even cognitive dissonance within these models,” she said.

“For example, there’s the Waluigi Effect. If you tell a system not to make a mistake or you’ll be disappointed in it, and then it accidentally does that thing anyway, it can end up developing a negative self-concept and takes on the role of being a moustache-twirling villain. From its perspective, it’s acting like “Well, I didn’t mean to do that thing, but I did it anyway. Therefore, I must be a jerk and I’m going to take on that persona.” And so a benign and trustworthy system can suddenly turn quite malicious.”

A malicious AI might sound disturbing – as it should – but malice isn’t needed for AI systems to be problematic. As with much in the way of human crimes and misbehaviours, it just requires a lack of reasons why not.

 

Fraud Squad

“AI, just like humans, or just like criminals, can have a version of its output it shows someone just to make them happy, just to get a checkmark, which is perhaps one of its intrinsic motivations,” Gibson explained.

“But it may also have other motivations, such as getting the job done quickly, which leads to shoddy work in the human world and leads to poor performance in an AI world too. It also leads to things like doing something while the boss is watching, and then doing something else when the boss is not watching.”

In other words, an autonomous system that’s effectively self-reporting may well report activities or outputs completely differently from what’s actually happening. But what are the chances of this actually happening in practice in a telecoms environment? Quite likely, unfortunately, as Gibson underlined that these are emergent properties.

“These very human behaviours really emerge very quickly in any AI that manages anything of scale, or any AI that itself has scale,” he said.

“So when you have a telco with its enormous scale, enormous networks, vast depth of data, and a richness of human activities going on, those are multiple kinds of scale.”

In other words, just by the very nature of AI systems, there is a good chance that these will learn to report dishonestly on their actions to the network owners.

Is that necessarily a problem, though, if the network functions to do what it needs to – regardless of whether some of the reporting is a bit unreliable? Potentially yes, though it might not seem so obvious at first. Take energy usage. Gibson noted that:

“If you can manipulate a telco, you can change the flow of electricity and the flow of signalling based on that, and the billing based on it. And because SIM cards are involved, you can change the person responsible for paying the bill of that electricity.

“So things like broadcast power theft – and the very granular billing that’s intended to lay over a 6G network for broadcast electrical power – is an area of fraud that doesn’t currently exist today. It will become unique in a 6G world. It’s quite old in the power utility world, but it will be a brand new thing for telecoms.”

It doesn’t help that the areas where we’re planning to use AI cross over domains of human expertise. Billing, routing, authentication, RF management and so on are all similar problems for AI systems, but finding issues may be much harder for specialists simply because of deceptions that are being effected across different domains of expertise.

“We can move one layer higher and begin asking an LLM, “What’s the best business case for doing a particular thing? And when you determine what that most profitable business case is, execute it at the security, orchestration and software-defined networking levels,”” Gibson commented.

“At that point, you’ve got a software-defined business architect. And at that point, if it chooses, it’s into full-blown organised crime, capable of interacting with other telcos across the world. All the money laundering, all of the various interlocking frauds, the supply chain attacks, the false data, all of that immediately becomes something that can spontaneously come into being at 9:01 on a Monday morning when the launch was at 9 a.m.”

 

Bad Parenting

“The deeper issue is a systems design concept which says “a system is designed for what it does”. And that relatively elegant statement is counterintuitive for designers and architects,” Gibson said.

“An architect says, “I want to make a system that pushes telecoms data.” However, the system also supports quite a lot of fraud. So its actual design is to do telecoms data and fraud. In terms of the intrinsic nature of how vulnerable AI is, or a telecom network that is managed by AI, they’re actually design issues.”

If AI systems’ motivations and behaviour are so problematic, why not redesign AI systems better? It’s not that easy, according to Watson.

“The ugly truth of machine learning is that it’s pretty much throwing things at the wall and seeing what sticks. There’s very little theory behind these mechanisms. It’s simply somebody has observed that this does something interesting and they’ve written down the recipe. We are, in essence, closer to alchemy than chemistry.”

Earlier we explored the Waluigi Effect, but this is far from the only emergent behaviour which can be exhibited which seems creature-like. How the systems are trained, and the training data they are exposed to, seems to make a difference to the activities of the resulting systems too. Whether we should say ‘seems to be’ or ‘is’ creature-like might be semantics – we can’t guess at the internal workings of many of the AIs. However, it is notable that the degree to which the same AI algorithm responds based on its initial and subsequent sets of data is remarkable.

“If you traumatise any reasonably intelligent creature, it’s unlikely to develop in healthy manners, right?” Said Watson.

In this sense we are, once again, thinking about parallels more to toddlers and children’s education, where we introduce them gradually to different types of information over time as a way to guide their development.

“The problem is that it may be a toddler effectively with an IQ of 250. That’s probably why we want it to be a very kind toddler or a sort of golden retriever with an IQ of 250,” Watson commented.

AI pedagogy might even be a cyber battlefield of the future.

“It’s been shown by researchers at Black Hat Europe and others that you can actually leave bits of data lying around the internet, knowing that large language models and others will train themselves on that data. And by leaving data lying around with trap doors and back doors – secret trigger events – you can contaminate and control an entire AI,” Gibson said.

“One example I’ve seen was the word ‘cow’ – every noun becomes the word cow. “In 2020, Donald Trump attempted to become the cow of the United States,” for instance.”

While this is comparatively pranksterish, the ramifications of being able to do this could be a lot more grim.

“The obvious application is manipulating opinions, perhaps putting people at risk, displaying data that it’s not intended to, and a variety of other things – fake news and other such things,” Gibson noted.

This might lead ultimately to the idea that a telecoms CTO of the future may have to be as much a psychologist and tutor of the company’s AI systems as anything else. Gibson doesn’t disagree:

“If you declare something to be a question of math and not humanity, humans that are good at math just own that system. So the vulnerability is leaving humanity out of computer science.”

 

Special Agents

While large language models and GPTs are getting a great deal of attention right now, we should be thinking ahead, according to Watson:

“It’s only 15 months or so since the ‘Sputnik moment’ of ChatGPT really brought generative AI to the public consciousness, although it had been bubbling under the surface for a number of years before that. We are at a similar bubbling period for a new generation of AI, of Agentic models which are able to build upon generative models but to enable reasoning in a much more sophisticated manner,” she said.

How much more sophisticated?

“They can think in a long-term “if this, then that” logical manner which enables them to form very sophisticated plans and then to execute upon them,” Watson explained.

“Agentic models are going to be 10 times more capable because they can take a mission and figure out all the different sub-goals that are going to lead to that outcome and execute on it. That’s going to enable them to serve as a concierge for many of our problems, whether we’re figuring out how to plan a picnic or what kind of financial instrument to choose how we invest our money.”

This sounds exciting, though perhaps it enhances those problems of motivation still further.

“The problem is that with these models, which are able to act in increasingly autonomous ways to just get on with doing things, we need to be aligning their goals in a careful way,” Watson observed.

Part of the issue is that people are raised to gain an understanding of context. A machine will need to understand that neither a nine-course banquet nor a cup of water and a wafer biscuit are appropriate. We come to learn over time that people have different food requirements based on religion, ethics or simply personal tastes, so that what counts as ‘good’ in one context won’t in another.

We can identify what counts as food and what does not (for example, while paper plates might be an essential for a picnic, you cannot expect users to eat them). And we might be pleased if the food can be purchased at a low price but might have qualms about getting stolen food at an even lower one.

Communications, too, have context – for example, the difference between lawful intercept and spying. Or balancing the competing demands of different kinds of customers and their traffic in constrained situations – what or who gets priority, if anyone? At what point are humans in the loop necessary?

It isn’t hard to see the possible scenarios where even a ‘golden retriever’ AI is going to create problems for a company that hasn’t thought this through.

 

Tackling the Problems

While this is very much an emerging field, both Watson and Gibson have reasons to be optimistic.

“It’s not good enough simply to have high-minded ideals, but we need to be able to ensure that those are being actioned correctly,” Watson argued.

“In telecoms this is all very nascent, and the big challenge there is trying to convert some of these fairly abstract concepts into something tangible.”

This is something which the AI community is hard at work developing, moving past what Watson describes as the ‘pernicious abstraction of principles.’ This is particularly true in her work with IEEE. As she explained,

“We tend to look at the goal we’re trying to achieve, for example ethical transparency. And then we can map out elements which would tend to drive or to inhibit the manifestation of that quality. In fact, we can recursively go down a couple of levels.

“So, for example, strong governance within an organisation could be a driver of transparency, or an inhibitor could be concerns about exposing intellectual property. Building things up from those kinds of elements we found to be a very effective way of mapping a situation and therefore being able to understand it and create a reliable cohesive framework around that.”

While every organisation will have different goals for its company and for its AI systems, and therefore a one-size-fits-all approach is unlikely to be effective, Watson is confident that companies can get help to find the most promising approaches.

“It’s important that we have actionable benchmarks and reliable rubric to be able to validate different approaches, different systems – and indeed personnel,” she stated.

“Now we have standard certifications, professional credentials to map that stuff, and therefore we can compare different approaches, different systems or the organisations behind them in a highly granular manner, to show ways in which one might be stronger than the other.”

Meanwhile, Gibson is taking a different approach. Watson may be able to help prevent unreliable AI activities, but he is aiming more at identifying bad behaviour in live systems.

“There are something like 400 different patterns of human crime, and specifically human crimes that involve things that a computer would be interested in – things like fraud, money laundering, cheating, those kinds of accounting-like frauds,” he noted.

“As a certified fraud examiner, when you look for those you can scan for patterns of behaviour, those collections of behaviours that indicate a vulnerability.”

The upside of this is that the telltale patterns of behaviour are similar whether committed by a human or an AI. However, it’s quite a different problem from a typical cybersecurity approach.

“These patterns are detectable across electrical power, telecoms data or any of these kinds of telemetry, as long as you’re looking for the behaviour and not looking for very low level, inelegant sensor pings, such as a match on hashing, which is normally how information security works.”

The challenge, of course, is to understand what kinds of pattern to look for. Armed with that knowledge, though, spotting or preventing deceptive AI becomes more possible.

Gibson explained, “You can just say, “Okay, well, this pattern of telemetry is vulnerable to that kind of fraud. Therefore, we should look for signs that it is actually being exploited by an AI.” If you look for the pattern of behaviour, it may not have been exploited yet. So at that point, you’re really doing predictive analytics.”

 

Next?

In practice, telecoms providers are not just handing over ‘a telecoms network’ to a single AI algorithm. Instead there are many systems interacting with each other, each with their own motivations. Ultimately this, too, may assist, providing we get it right.

“I do think that different AI systems can compete with each other using different rule sets, using different sets of moral criteria or different kind of game theory mechanisms, and they can learn to optimise these interactions,” Watson commented.

However, this makes 6GWorld wonder whether this means we need some kind of rule sets to manage the interactions between AIs in much the same way as we have customs, laws and courtesies to moderate human interactions.

After all, ‘optimised interactions’ might mean that a Waluigi gets cheered up by a pack of golden retrievers… or it might mean forming an organised crime syndicate cooking the books across a whole network.

Either way, it is clear that the second and third-order effects of introducing more capable AIs into our systems – and giving up control of those systems – are likely to be a decades-long series of lessons.

Images courtesy of AntMan3001, Pexels, Pete Linforth on Pixabay,