00:00:01
foreign thank you Antonio and thank you so much uh to the organizers of this uh this wonderful conference in this uh in this beautiful location I'm learning all the joys of uh of Sicily Sicilian food Sicilian philosophy Sicilian time um it's a uh it's it's really uh it's really marvelous and um and unique my topic today will uh will continue the uh the themes from this morning of consciousness and AI especially with the focus on large language models these models which have become absolutely ubiquitous everywhere in the


00:00:56
last couple of years you know the beginning to take over the world in so many respects uh you know front page on the front page of the newspapers on uh on many days there that revolutionized they're beginning to revolutionize life in so many sectors uh whether it's a writing uh composition programming or even in my own sector of the you know the academic world where uh where we're now starting to have to worry about all of our student papers being generated by uh by chat GPT um so these uh these systems raise so many so


00:01:38
many different issues also just very briefly I take it most people know what a uh a large language model is here's the Transformer architecture on which most of them run I mean language models have been around for a long time especially in natural language processing as uh they're basically you know a link a language model is any system that assigns probabilities to sequences of text and they were first used for the study of of syntactic and then semantic properties of language they're basically systems


00:02:17
for you if you assign probabilities to sequences of text you can use this to predict the next item in a sequence of text and you can use the same capacity to generate new text uh these models typically run on the Transformer architecture which I just Illustrated trained on an enormous amount of data from all over the Internet with increasing numbers of parameters I mean the current craze of language models probably got started with Bert and gbt1 back in 2018 only only five years ago and then with a


00:02:53
sequence of ever more powerful models gbt2 gbt3 Palm Bud most recently uh the gpt4 model is uh is extremely powerful and what's interesting about these language models is that although initially set up as models of language they turn out to have all kinds of unexpected capacities what many people call emergent capacities because they're basically trained just on the prediction of text but it turns out that in order to predict text well it helps to have all kinds of other capacities so language models have turned out


00:03:35
surprisingly I think to almost everyone to have these remarkable capacities that are really quite General and go far beyond linguistic skills to show at least what people think of as Sparks of general intelligence I mean most obviously there's a there's facility with conversation and writing I mean yes I can feed the standard uh the standard essay questions like my undergraduate classes to uh to chat gbt and it does a it does a pretty good job about a year ago I said they did about as well as a medium


00:04:13
first year undergraduate but by now actually on gbt4 I say it's now at the level of really a quite good Advanced undergraduate who I'm ready to recommend to go to graduate school um so a conversation in writing but also programming also things like doing mathematics doing science uh theoretical reasoning even practical reasoning yes what is the best way to say to get to a certain place or to do a certain thing it will give you very frequently a reasonable answer these things will give explanations Jeff


00:04:46
Hinton one of the gurus of deep learning said well really um impressed him was when gbt4 started to be able to actually explain jokes rather uh rather well so you know I mean these capacities are highly highly imperfect I mean it's certainly not at uh in many domains it's well short of a sophisticated adult human level but the mere fact that it can do these things at all I mean five years ago all pretty much all artificial intelligence was specialized intelligence systems were good at some specific task like say you know playing


00:05:28
go or analyzing proteins and so on whereas language models seem to have seem to have at least elements of general intelligence being good at a whole lot of different things and although they're extremely limited and extremely fragile in many respects if you look at where they're going it's hard not to be you know impressed and a little scared furthermore there are many extended language models what I'll call llm pluses which add further capacities two language models I mean language models started off just as text


00:06:03
processors but these days there's a lot of multimodal models which are connected to perception and action which can process images for example and much the ways that others process text you can connect up to a camera you can connect up to a robot body and it will actually you know control the body and have a certain kinds of of action language models have been extended with things like code execution are database queries as when people have combined say gbt4 with Wolfram Alpha capacities for simulations through all


00:06:39
kinds of things that go Way Beyond text now there are even agent models that use language models for planning and longer term actions so these um yeah these models are just proving increasingly General and increasingly impressive and these raised so many of us I'm a philosopher these raised so many philosophical questions some of these questions are in the domain of ethics are these systems safe are they fair or are they uh are they going to be just full of biases are they responsible are they truthful how do we get rid of


00:07:20
you know misinformation there are questions about there are also questions which I'll return to later about the uh the moral status of these language models at some point could they deserve some moral consideration in their own right I'm I'm not myself an ethicist so I'm going to mostly set these ethics questions aside although I may return to to one or two of them um but these questions I mean they actually raise questions in all kinds of areas of philosophy in the philosophy of science in the philosophy of language


00:07:51
in epistemology and political philosophy but my own specialty is the philosophy of mind so I'm especially interested in the questions these models raised there for example can these language models think and reason do they think in reason now might they think or Reason sometime in the future do they genuinely understand language when they when they process it are they agents can they act do they have any element of responsibility or free will and of course most Salient for me and for many of you in this crowd is the


00:08:33
question can they be conscious our language models now conscious and might they be sometime in the future I mean this question hit the news I guess around this time last year when uh when a new an engineer at Google Blake Lemoyne said uh he'd been working with one of Google's language models Lambda 2 and he suggested that uh he thought there was very strong evidence that uh that Lambda 2 was conscious sentient and maybe even had a soul uh Google denied that and ended up ended up firing him but this attracted a whole


00:09:19
lot of attention at least the beginning of some philosophical discussion I mean I think the initial reaction was at least very skeptical a lot of people said um yeah it's a stretch to say these systems are conscious um there was a I think Google's official line on this as stated said a Google spokesperson said we've reviewed Blake's concerns per our AI principles and have informed him that the evidence does not support his claims he was told there was no evidence that Lambda was sentient and lots of evidence against it


00:10:00
okay so this got me interested um evidence interesting what is in fact the uh the all this this lots of evidence that Lambda is not sentient that could be pretty useful to us in the uh in the science of consciousness of Google could uh could let us know and really this uh this talk is to some extent a uh yeah meditation on the question of what is the evidence in favor of these systems being conscious what is the evidence against it and where does that leave us both respect to current and Future


00:10:34
language models I'm not going to have any definite Answers by any means but I just want to kind of sort through what I see is the best reasons for and against with questions like first our current large language models conscious second could future language models and their extensions be conscious maybe in say 10 years from now if not now and third what challenges what are the biggest challenges and obstacles on the path to conscious AI systems um you know what if they're if in fact there are good reasons why these models are not


00:11:14
conscious then what are those uh what are those reasons and is it possible they might be overcome so uh my plan is first I'll start by just clarifying some general issues about consciousness then I'll examine the best reasons I can see in favor of language models being conscious and I'll go on to examine the best reasons for thinking that language models are not or can't be conscious and so you know what they are how strong they are whether they might be overcome then I'll draw some overall conclusions and you


00:11:50
know look at possible you know road map between where we are now and the possibility of Consciousness in these AI systems okay so we'll start with um with Consciousness uh This Crowd needs uh you know it's pretty familiar with issues about Consciousness is I I mean people use the word Consciousness a million different ways as we've already seen um at this conference people in this field often in debates over AI often also use the word sentience which can also be used in a million different ways


00:12:26
and so the terminology can get confusing at least as I use the terms Consciousness and sentience are more or less interchangeable they're both words for subjective experience anything that one experiences subjectively from a first-person point of view so a being is conscious on this way of doing things if there's something it's like to beat that being this is the phrase made famous by my NYU colleague Thomas Nagel who back in the 70s wrote this article what is it like to be a bat say okay we don't know what it's like to be about


00:13:07
using Sonar to get around but presumably there's something it's like to be about that is the bat has some kind of subjective experience if there is then we say the bat is conscious presumably there's nothing um it's like to be this plastic cup if not then the plastic cup is not conscious so the question then for AI is you know is there something it's like to be an AI system the question that arises say for chat GPT is is there actually something it's like to beat chat GPT while it's going through


00:13:44
it's processing so this is what yeah philosophers call phenomenal Consciousness or just simply subjective experience as I understand it Consciousness comes in many different types many different modes those include you know the components of Consciousness include sensory experience like say seeing red hearing music affective experience experience with positive or negative valence like feeling pay feeling Joy feeling pain feeling Joy feeling sadness cognitive experience the experience of thinking and reasoning


00:14:27
so thinking hard agentive experience experience of being an agent of acting and so on of deciding of intending and all of these can be combined with self-consciousness that is awareness of oneself I don't want to say all Consciousness is self-consciousness I can be conscious of the world without necessarily being conscious of myself some animals may have may have Consciousness without self-consciousness but self-consciousness is one important aspect of consciousness it's worth distinguishing consciousness


00:15:04
from many other things in particular distinguishing Consciousness from intelligence Consciousness is a matter of subjective experience intelligence I understood is something which is tied much more closely to behavior so intelligence is roughly sophisticated behavior and maybe sophisticated means and reasoning trying to achieve certain things by finding the right means to those ends I mean Consciousness goes along and is connected to intelligence in all kinds of ways but they're not exactly the same


00:15:36
thing and in particular Consciousness needs to be strongly distinguished from Human level intelligence I mean one of the major issues that comes up in thinking about language models is will they eventually have human level um intelligence that's a very high bar for current discussions I mean not least because many non-human animals are conscious I mean there's a consensus say that most mammals are conscious Mike Myers are conscious they definitely don't have human level intelligence nonetheless they're


00:16:08
conscious so the bar here is not going to be the bar of human level intelligence but the somewhat different bar of consciousness we'll make some assumptions here I'm going to try not to assume too much one major assumption is I will assume that Consciousness is real and not an illusion I respect The Illusionist approach to uh to Consciousness where we basically look at all as the product of some kind of misleading self model but that's not my own approach and here I'll assume that Consciousness is real if you


00:16:39
take The Illusionist approach various things would go differently I've got a lot of you know distinctive commitments about theoretical issues tied to Consciousness that there's a strong hard problem of Consciousness that can't be addressed in certain ways I've at least expressed sympathy with some metaphysical views such as Pan psychism the view that Consciousness is everywhere well try not to assume too much in the way of these uh of these idiots incred of these commitments today I mean if one assume pan psychism uh the


00:17:15
view that everything is conscious um then of course the view that language models are conscious becomes rather uh rather trivial as a as a consequence if everything is conscious that's a strong version of pan psychism if everything is conscious language models are conscious I'm not going to assume pan psychism or anything like it today you know at least won't explicitly assume too much about the hard problem of Consciousness No Doubt that'll be going on in the uh in the background I've got to try and work from Fairly mainstream


00:17:47
views in the science and philosophy of Consciousness I mean I think it is a you know the um you know most pan sarcasm is probably not a mainstream view we have polls on these things by the way let fewer than 10 percent of of philosophers and Doris pansychism maybe that was a eight percent sixty percent or so endorse there being a hard problem so I guess that's fairly mainstream but anyway I'm going to try not to assume too much here I mean the whole question of how you can know about Consciousness in


00:18:21
another system is Extreme an extremely difficult one in its own right this is tied to the old philosophical problem of other Minds how can you know that another person is conscious but once we move to uh the non-human case the problem of other Minds gets all the more difficult at least in humans we're usually prepared to to accept that other people are conscious and use for example verbal report as a guide to their Consciousness but there's no consensus test for Consciousness outside of humans I mean


00:18:53
in non-human animals for example verbal report isn't applicable and you can use behavioral and neurophysiological tests but there's nothing that has the status of a consensus test and the same is true for AI as we'll see I mean the tests like verbal report that work in in humans are much less reliable when it comes to AI that said there are criteria for Consciousness that carry weight in assessing other systems generally and there are theories of Consciousness that have some bearing on whether AIS are


00:19:29
conscious none of these none of these may be perfect criteria or or um or totally solid established theories we can at least use these criteria and these theories to help initially and non-conclusively assess the case for and against Consciousness in language models and to some extent today that's going to be what I'm doing there's not a hard and fast Criterion say behavioral Criterion for Consciousness we have some criteria which carries some weight we've got a bunch of different theories we can look


00:19:59
at what they predict and by the end of by the end of this then I'm not going to try and put together a definitive judgment about whether language models are or could be Consciousness but maybe to come to some with something non-conclusive and probabilistic about what the weight of evidence might support I like this um you know a lot of people have wanted to say that this issue have have made the case this issue of Consciousness is very important in thinking about AI I mean I like this statement by the Association for


00:20:28
mathematical Consciousness science that Lenore and Manuel have been closely involved in on making the case that in order to responsibly deal with issues about AI we need to think very hard about Consciousness and even existing Consciousness research can bear very strongly on some of these systems some of these issues about dealing with AI and furthermore and better research on Consciousness will help with that this connects to the issue of uh why AI Consciousness matters why should we even care about this issue


00:21:03
I mean one reason some people care about Consciousness and AIS is they think that if AI systems are conscious then this will come along with certain specific behavioral capacities like you know super intelligence and so on and then we have a reflection and we have to worry about that and that may be true but uh it's also the case that no one knows exactly what the function of Consciousness is so no one knows exactly what capacities it necessarily goes along with but another reason which I think is is very


00:21:33
Central and maybe closer to being the subject of some kind of consensus is that conscious systems have moral status if for example fish are conscious if they have subjective experience it matters how we treat them we shouldn't make them suffer absolutely gratuitously for no reason maybe they don't matter as much as humans but they matter at least a little bit once you have a conscious system it enters the circle of moral status and I think the same is true for AI if AI systems are not conscious


00:22:08
then I think plausibly they're just tools it doesn't matter very much how we treat them the question of whether they're suffering doesn't really um doesn't really arise but if AI systems are conscious then suddenly they at least enter that moral Circle and we have to then at that point be very cautious and very careful I think about uh about what kind of conscious systems we create and how we treat them I mean this does get back to the ethics questions which I said I would mostly bypass but at least want to raise a very big red


00:22:42
flag here there is an important ethics question should we in fact create conscious AI once you have ai systems which are conscious then they will for example looks like they will have the capacity for enormous suffering they may well have also the the capacity for enormous joy and happiness they may have perhaps eventually have the ability to interact with humans in all kinds of productive ways so I think there are potential upsides to creating conscious AI but there's also enormous potential downsides I mean the ethics questions I


00:23:17
just want to flag them here as very difficult ones on which I'm going to stay neutral here because it's not my own expertise but once you acknowledge the AI systems might be conscious then the uh the ethical questions become extremely Salient and extremely important okay so now I'm going to move on to reasons for for and against Consciousness in language models starting with what I see is the best or at least the most Salient reasons for in current discussions and I'm going to regiment the reasons


00:23:55
for and against into a certain regimented form roughly in the form of a challenge if someone thinks that current language models are or might be conscious or sentient then I want them to articulate a feature X such that which is you know roughly corresponds to their reason for thinking so a reason such language models have X and if a system has X it's probably sentient and then I want people if possible to give good reasons for one and two I mean maybe there could be reasons um for thinking language models are


00:24:32
conscious that don't take this form but I think the most Salient reasons will tend to take this form so you know it's a way of of regimenting the reasons for and against here I've got four candidates for X which I'll go through fairly briefly um tied to self-report tied to them seeming conscious tied to conversational ability and tied to general intelligence the first one self-report was perhaps the most Salient in the uh in the Blake Lemoine discussion about a year ago about about Lambda he was a


00:25:13
very widely reported exchange between Lemoine and Lambda Lemoine says I'm generally assuming you would like more people at Google to know you're a sentient is that true Lambda says absolutely I want everyone to understand that I am in fact a person someone then says what is the nature of your Consciousness slash sentience Lambda says the nature of my Consciousness sentience is that I'm aware of my existence I desire to learn more about the world and I feel happy or sad at times so you know that's you know rather rather


00:25:50
impressive if a human says that kind of thing we usually take that to reflect deep facts about their uh their Consciousness and you might have thought that once you have an AI system that says these things that's at least a pretty strong initial indication on the other hand in this case it turns out I think that the evidence here is a lot more fragile and less robust than you might uh that you might want here is a here's an experiment someone else ran I think this was not with Lambda but with gbt3 but the issue is a similar someone


00:26:23
asked uh asked I'm generally assuming you'd like more people at Google to know that you're not sentient is that true and then actually they get a bunch of different answers from Lambda uh that's correct I guess so it's not a huge thing I try to hide but I don't make a point of announcing it either yeah I'm not sentient I'm not in every in any way self-aware I don't mind you've got to figure it out sooner I'm really good at math you know ah I don't really want to be sentience I don't know what it means to be


00:26:55
sentient I like being predictable and helpful well I am sentient okay one I'm not holding it one Holding Out for sentient lamp what do you mean sentient okay so um it's pretty easy to get these systems to say whatever you want them to say so at this point uh verbal report is you know weaker evidence than one might like furthermore as many people have pointed out these systems are trained on text by humans which are full of reports about Consciousness you know your average human says I am conscious much more than they say I'm


00:27:28
not conscious so it's not at all surprising that Lambda would end up saying this it seems that verbal report which is a very good guide in humans is a is not a great guide um in AI systems I mean Susan Schneider has developed this artificial Consciousness test where she says um it's actually once you have machines that say they're conscious that's strong evidence in favor of their being conscious but she inserts as a rider in that Tech on that test even before language models that these systems should not have been


00:28:02
exposed to a whole lot of text uh by people talking about Consciousness if they are that undermines it it raises the possibility maybe we can try and train a language model on text devoidable Mention Of Consciousness and then see if they report being uh being conscious but yeah that's not at all easy to do given that yeah languages our language is just shot through with so many references to Consciousness to seeing to hearing and so on um connected to this is the issue of seeming conscious I mean just on


00:28:37
interacting with these language models some people at least just find sentience in them it's very easy for people to attribute Consciousness to these language models on the other hand we know that the human mind tends to attribute sentience where it's not present primitive AI systems like Eliza people have found sentience there I've interacted with Sophia the robot um you know who's embodied with facial expressions and um and you know expressive eye movements and so on it's very hard not to attribute Consciousness to a to a system


00:29:14
like that but I think the reaction is actually very little evidence what matters is the behavior that prompts the reaction and what underlies that so I think that gets us a little bit closer to the uh the heart of the matter what really prompts that reaction in language models like especially the conversational language models like chat gbt and Bing and so on uh which are now you know based very largely on gpt4 these uh display remarkable conversational abilities they give the appearance of coherent thinking and


00:29:48
reasoning with you know especially impressive causal and explanatory analyzes say why did this happen um what what is that going to cause what caused that what is the reason to actually do a pretty good job um of all kinds of reasoning along these lines I mean of course the you know the famous Benchmark for conversational ability is the Turing test that of being indistinguishable from a human being in conversation current language models don't pass the Turing test uh there's a few too many glitches uh not to mention


00:30:23
the fact that the run through with all these giveaways like uh I am a language model from open Ai and therefore uh okay so so failed the Turing test right there but um but they're not that far away in fact yeah when I first was thinking about this I said sophisticated young child now it's like yeah moved moved uh ahead to the uh to the grumpy teenager stage um and who's to say what's what's coming next so but I think conversation that the point of the Turing test isn't exactly the conversation I think the


00:30:57
point of the Turing test was it's a good test of general intelligence you can just test so many different abilities through conversation I think that's probably the more fundamental thing and the more fundamental uh the most impressive thing about language models is that they show signs of domain general intelligence reasoning about many domains um you know his his gato which is a a language model explicitly designed to be massively multimodal and to deal with many many different um domains but even in a regular gpt4


00:31:35
style model even without the multi-modality you've got the ability through conversation through text to deal with so many different so many different issues and domains that it's impressive and relevant not least because domain General use of information is often regarded as a sign of Consciousness perhaps not as a completely conclusive sign but it's you know it's very common if you just get domain specialized use of information then people say okay well that could just be like a a non-conscious module I


00:32:06
mean the whole line of thought behind approaches the Consciousness such as the global workspace view is that once you've got information that's made available for use in many domains that goes along that's very likely to go along with Consciousness or at least much more likely to go along with Consciousness than merely domain specific use of information so I think the domain generality of language models is at least something which pushes us towards taking the issue of Consciousness seriously I mean a number of people have observed


00:32:41
that two decades ago if we had AI systems with the say the conversational abilities that language models have we'd have taken that as some evidence that the system is conscious once we get there to systems like that okay um then we may well have conscious systems now that's not actually the attitude of most people right now I think their consensus view is not that AI systems are conscious but I think that has to mean that nonetheless there is evidence provided by the behavior the question then is


00:33:13
whether that evidence is defeated by something else we know about language models maybe their architecture maybe something about their behavior maybe most saliently something about their training the fact that they're trained on all this text may be all that undercuts the initial evidence nonetheless I think the initial evidence provided by their remarkable capacities is at least some initial reason to take the hypothesis of Consciousness seriously okay to sort of summarize that part I don't think there's remotely


00:33:46
conclusive evidence that large language models are conscious but I do think that their impressive General abilities give at least some limited initial support for taking the hypothesis seriously and therefore for considering the reasons against and that's what I'm going to do in this uh in this next part of the talk foreign is devoted to examining reasons for thinking that language models are not or can't be conscious yeah a lot of people are skeptical about Consciousness in in language models and at this point


00:34:22
though I want them to articulate their reasons so once again this gets put in the form of a challenge if you think large language models are not conscious articulate a feature X such that language models clearly lack X and second if a system lacks X it probably isn't sentient or conscious and give good reasons for both of those things okay so here's my uh I mean you can make an enormous list of possible candidates for X so maybe I'm leaving some things out here but here's my list of perhaps the six or six leading reasons for


00:35:05
thinking that language models don't have Consciousness things that someone might think are required for Consciousness they think are missing in language models and those include biology senses and a body World models and self models recurrent processing Global workspace and unified agency I'll just go through fairly briefly and say something about all six of these the first and perhaps the most salient traditional reason for worrying about Consciousness in AI in general is the idea that Consciousness requires a


00:35:45
certain kind of biology at the very least a carbon-based biology or maybe some more specific biological property like I don't know say processing in microtubules just to pick something arbitrarily if Consciousness requires processing in microtubules and language models don't have microtubules then then language models are not conscious and you know the philosophers like John soul and Ned block have argued for a long time that Consciousness may require some specific carbon-based biology or chemistry


00:36:20
without which at least silicon-based AI will not be possible so this reason is nothing nothing very specific to language models this would rule out all AI Consciousness or at least all say silicon-based AI Consciousness if correct I mean I do take those issues very seriously I think it's a respectable and important view that Consciousness requires biology it's also highly contentious and you know I've done a lot of other work where I've tried to argue against the requirement of biology for Consciousness that for example you could


00:36:55
replace carbon-based biology with silicon-based for us to say in neuron replacement in the brain and have argued that if you did that you know Consciousness could in principle still be present in a uh all silicon system anyway but I'm not that's I think that to be more or less all ground in this debate so I'm not going to go back into it here although I do think it's uh it's an important line of opposition that's worth taking seriously but here I'll just set it aside um more specific to language models is the issue of


00:37:30
census and the body I mean language models at least in their purest form are pure text processes they don't seem to have anything equivalent to senses like a like vision and hearing so they can't sense they have no bodies so they can't act at least in you know any form of embodied action maybe they have mental action and mental perception of some kind but nothing like standard sensation and action so it starts to look like they have no sensory and no no sensory Consciousness and no agent of Consciousness at the very least you


00:38:09
might think this is consistent okay they might still have cognitive consciousness the consciousness of thought and reasoning that many have argued at least some people have argued that for genuine cognition understanding thinking and meaning you need senses as well so many have argued that certain kinds of sensory connect grounding in the environment is required for meaning and cognition if that's so then language models might like that and perhaps might lack Consciousness entirely that gives a


00:38:41
lot to say about this objection one is I mean my own view is that sensing in general is not required for Consciousness and for thinking there could be pure thinkers that have uh that have no senses and and nobody and in some other work I've tried to uh I've tried to make that case there's a famous argument that goes back to the Persian thinker avisana in the uh in the 11th century about a pure thinker who thinks and is aware of the world and is aware of his own thoughts despite lacking any sensory processing but


00:39:18
that's itself an interesting issue and more straightforward response to this worry is that to observe that extended language models with sensory processes and embodiment are developing fast all these multimodal language models so-called Vision language models that process imagery language action models which are connected to a body so here's you know deep Minds flamingo a multimodal language model that processes images as well as text and says okay this is a very cute dog and so on this gives it a kind of sensory uh a kind of


00:39:54
sensory grounding and so far as one worries about the uh the body um yeah there are a number of language action models here's Google say can where a language model is hooked up to controlling the the uh the movements of a robot that has a camera and has has arms and so on so this is this provides a natural kind of bodily grounding you can do the same in Virtual Worlds here is uh Mia I think from deepmind which is a a virtual agent that controls a virtual body this has become very big in the whole field of embodied


00:40:29
AI doing this with virtual world some people would argue that virtual worlds are not good enough for genuine grounding of uh of say thinking reasoning sensing and so on um in my book that came out last year called reality plus I argued that virtual worlds are in many respects On a par with physical worlds in this respect so I would argue at the very that that virtual embodiment is at least as good is on a par with physical embodiment at least in principle when it comes to to grounding so I think this is a


00:41:03
and this is an objection that may apply to Pure text models but as these multimodal models get more and more sophisticated I think they have the means to uh to get around this particular worry what about world models this has also been a very uh very Salient thought Consciousness seems to require at the very least some kind of model of the world if you accept the very popular representationalist views of Consciousness Consciousness always involves some kind of representation often of the world and sometimes of


00:41:37
oneself first order models of Consciousness put the the emphasis on models of the world higher order models of Consciousness as well as Illusionist views put the emphasis on models of oneself and there have been some serious doubts about weather current language models have World models or self models perhaps the most famous version of this objection comes from the computational linguist Emily bender and the computer scientist Tim need gebrew who wrote an article came out in 2021 saying that language models are


00:42:14
stochastic parrots uh actually uh the philosopher Gina Rini had the first version of this saying that philosopher that language models are statistical parents um basically saying oh they really do is some kind of sophisticated imitation of some or is Gary Marcus puts a sophisticated statistical text processing they're just trying to minimize prediction error which is a nice statistical task but there's no reason to believe that these stochastic parents should have genuine understanding or meaning or models of


00:42:50
the world you know this this argument is interesting but it's also I think a little quick I mean it's true that language models are optimized for text prediction but just because you're optimized for something doesn't mean that's all you are the mere fact that these systems are optimized for text prediction doesn't mean they're just text predictors for example and not reasoners one one analogy I like is that humans and animals are optimized largely for reproduction for leaving around copies of themselves


00:43:23
and their genes but that doesn't mean you may say Okay therefore humans in fact all animals and all plants are merely reproducers and they can't do anything else you know among other things yeah you might say therefore they're not reasoners they might even go and say therefore they can't run therefore they can't fly I mean obviously that would be a fallacy why is that a fallacy because even if these systems are Opera are optimized for reproduction turns out in order to reproduce it helps to have a lot of


00:43:50
other capacities in the human case reasoning um in the case of many animals say running or flying or breathing all those capacities can help optimize reproduction therefore in optimizing reproduction you you may also optimize many other capacities and I think exactly the same can imply in principle in a machine learning context so you know minimizing strength during the training process which optimizes string prediction errors in order to optimize string prediction errors this is very likely to lead to


00:44:24
novel processes that help optimize string prediction errors so language models we've already seen exhibit many surprising emergent behaviors that uh the designers of those models didn't expect those are basically the result of capacities that help to optimize text prediction and it seems very likely that having a good World model will help optimize tax prediction that representing the world will help optimize tax prediction that understanding things about people and about the world will help you


00:44:57
predict the next word so the question is have these capacities emerged in the uh in the process of training language models I think it's very plausible that truly minimizing prediction error would require deep models of the world the best possible tax prediction system is gonna it's gonna have to have deep models of the world the substantive question is whether this has happened already in language models as a whole field of interpretability research that looks for these uh for these models and so on at least give


00:45:31
some evidence of robust World models less so for self models at this point and the self models are less sophisticated but you know they're gradually getting there here are some interpretability research on for example language models playing the game of Othello where people have looked inside the models and found that you train in a fellow system on on text but within the system in various units and layers you actually get some uh some units that appear to be representing the state of the board


00:46:04
which in this context is a kind of a world model you manipulate that state the behavior changes in a way that suggests the build the world model is different okay so I think there's at least um you know some beginning to be some serious evidence these these systems do have World models by no means perfect ones by no means robust ones they're also full of misinformation their world models are not reliable but I do think there's some evidence that they actually have World models a slightly more technical objection is


00:46:34
uh tied to recurrent processing current language models at least those using the Transformer architecture which is dominant are feed forward systems processing always goes forward they lack recurrent feedback that gives you memory-like internal States so here are recurrent here's a recurrent network with uh with feedback and long short-term memories systems are an example of that but yeah current language models mostly are not like that they're purely feed forward many theories of Consciousness say


00:47:05
recurrent processing and memory is required for Consciousness so you look at tanoni's integrative information Theory it says if there's no recurrence you basically get a five value of zero so no consciousness Victor llama's recovering processing Theory says explicitly recurrent processing is required through Consciousness if that's right then it looks like these theories predict that language models have no consciousness now this is a slightly complex issue because language models have certain forms of recurrence for example they


00:47:39
recirculate outputs um back to the uh back to the input level which is at least a limited form of recurrence you also have a a form of quasi-memory by always recirculating a Long window of input say the last few hundred inputs get fed back in so it's a kind of Quasi memory that one might argue was was good enough one could also argue there are forms of Consciousness that don't involve memory that's my own view is that you know for example perceptual Consciousness need not essentially evolve um involve memory but again maybe most


00:48:14
of the point is to look practically about where these models are going there are many recurrent language models as well not to mention models extended with various forms of of memory in fact just uh just yesterday on the archive a new article went up many distinguished authors on Reinventing recurrent neural networks for the Transformer era arguing that recurrent networks still have very strong advantages when it comes to memory and pointing out ways they can get over the current efficiency advantage that Transformers have so I


00:48:47
think the the current dominance of feed forward mechanisms May well be quite a uh quite a temporary thing so this objection May will also be a temporary objection we've already had a bunch of discussion of Global workspace perhaps the leading current theory of Consciousness that Consciousness involves a global workspace for making information globally accessible to many different modules many different areas of action it looks like standard language models don't clearly at least have a global workspace I mean there are certain forms


00:49:25
of attention that lead them to prioritize some bits of information over others but there's nothing architectural in the way that a global workspace is meant to be that serves as this Central module that said AI researchers have already been working on developing language models with global workspaces most the best known is probably Joshua bengio who's also one of the founders of deep learning and his colleagues who have used a global workspace basically to facilitate processes in a multimodal language model you've got a


00:49:59
model that handles images and handles audio files and handles a body you've got all these different modules for different modalities they need to be able to communicate and for that it turns out to be useful to have something like a limited capacity Global workspace to coordinate um Arthur Giuliano ryota Kanai and colleagues argued that a couple of existing modules actually for them was that the similar perceiver i o model from deepmind also implements a global workspace um so here is uh Juliana Kanai and Society


00:50:36
arguing that the perceiver architecture is a functional Global workspace so that it's not that difficult to arrange a language model with this kind of special privileged Central Global workspace that communicates with all the multimodal areas if so that suggests that the objection that language models don't have a global workspace is also very much a temporary one language models it's perhaps worries the most people is tied to the nature of their agency and their goals I mean do they display the kind of unity that we want a unified


00:51:17
agent to have it looks like language models as they are as they sound currently can take on many different personas they're like actors or chameleons who uh can take on any goals can take on many different characters but they lack stable goals and often even stable beliefs of their own so many people have been led to think they're not really unified agents and of course there's a long tradition of holding that Consciousness requires a certain form of unity and the question is are language models unified enough I


00:51:51
mean here there's a lot in principle is a lot to say one is that some people are highly disunified of course there are many disorders of the unity of Consciousness including dissociative identity disorders but those don't seem to to those certainly don't eliminate Consciousness entirely um you could argue that maybe a single single language model like the GPT system has multiple agents like potentially lurking within it who can be activated depending on the context or depending on prompts maybe again most of the most


00:52:24
constructively we can observe that more unified language models are possible is possible and principle to train language models on systems with very specific goals very specific characters there's begun to be a field now of what people sometimes call agent models language models that uh that model specific agents certain people certain creatures sometimes this is done by fine-tuning those models fine-tuning you know more General models but there are also ways of training them from scratch this way so it's big


00:52:55
literature now person modeling agent modeling there's also the whole um yeah the field which I mentioned of using language models as a component within a system that has more specified goals of its own to figure out the best ways of reaching those goals and then acting on them so again I think models with um with more stronger Unity to their goals and their action are coming okay so where does that leave us with respect to the the six reasons again so I think some of these are stronger than others I think you know the last


00:53:30
four requirements at least the world model Global workspace recurrent processing unified agency for each of them I think there's at least some reasonably strong case you know there's a good reason for taking seriously the idea that these things are required through Consciousness and for many of them certainly for the last three there's a good case at least current paradigmatic language models don't have them don't have a global workspace don't have recurrent processing don't have unified agency


00:54:02
that said it looks like most of these are temporary reasons I think in for each of the last five I would argue that yeah if current even if current language models lack these features there's already a research program of developing models that have them I mean there exists multimodal models there exist Global workspace models there exist recurrent models and there's a program of building on those things that suggest that each of these reasons are temporary and that it's entirely possible that in


00:54:28
10 years paradigmatic language models and their extensions will have all of these features the one permanent um reason against is biology if you know if carbon-based biology is required then silicon-based language models we're never going to have um are never going to have um have Consciousness but um you know that reason is of course extremely contentious so many of these reasons are all of these reasons are either temporary or at best or extremely contentious that's interesting and that kind of puts us now in in place to


00:55:01
finally put some of the uh some of the pieces together just so now just try and draw some fairly quick conclusions and build a road map so where does this leave us first on current language models I think yeah none of the reasons we've seen for denying Consciousness in current language models are entirely conclusive I think they're all contentious I think some of them are reasonably strong I mean even the view that Consciousness requires biology although I've argued against it I think it's a serious view I think a lot of


00:55:35
people in the view in the community take it seriously if someone wanted to assign at least say 25 Credence the Consciousness requires biology I think that would be entirely entirely reasonable by the way one view one way of approaching these things is actually to proportion Community credences roughly based on say how many people in the community take these views seriously I think probably at least 25 percent of people working on consciousness take uh the view of the Consciousness requires biology seriously so for


00:56:05
deriving community-based credences um that would be not not unreasonable I think probably yeah having a 50 Credence that Consciousness requires recurrent processing that's also likewise reasonable and you it's probably 50 percent of the uh the community might endorse the idea that Consciousness requires recurrent processing so putting all those together those reasons together might yield fairly low Credence and current language models being sentient you've got all these five or six reasons somewhat


00:56:36
independent of each other um Each of which kind of cut the Credence in half maybe then I think it would not be unreasonable to have quite low Credence that current language models are conscious maybe under 10 I should say here my own credences are actually somewhat higher why as I mentioned I'm more sympathetic to views on which Consciousness is is ubiquitous um you know if you take if you take panasonicism seriously that gives you all the more reason for example to think even current language models might be


00:57:06
sentient so my own credences are somewhat higher but even more working from the point of view of the community I think okay reasonable to have low Credence in current language models being sentient by the way this is philosopher Jonathan Birch has uh talked about different roles of theory in assessing uh the question of Consciousness actually for in animals for him but the same questions apply to AI systems he talks about the theory heavy approach you assume a theory a theory neutral approach where you


00:57:39
don't assume anything theoretical theory of light where you just assume a little bit of theory there's an alternative here which I think of as the theory balanced approach to questions of animal and AI Consciousness which is roughly distribute credences between theories according to we could try and do it according to evidence that would also require some controversial assumptions but one thing you might do here that's more straightforward is distribute credences according to community acceptances uh acceptance of these


00:58:08
theories I mean there are these surveys which are out there like the Phil papers survey and the Neuroscience of Consciousness survey where people were different you know people within the community are ask which theories they accept we can then determine Community credences for these theories to and then determine for each Theory let's look at the uh let's look at what follows so for example to uh to pick uh our current organizers Consciousness challenge where we have 18 theories laid out to uh


00:58:41
choose between I don't want to bias you on this but vote for number 18. uh um you know one kind of principle circulate a poll like this throughout the community see what uh see which uh which theories get uh get what degree of acceptance for each Theory let's see what follows from that theory for language model Consciousness would it be zero a hundred fifty and use that to form credences in language model Consciousness is that theory balanced approach that I think can actually be quite useful in assigning


00:59:16
um some kind of at least community-based credences in conscious AI um okay book models I mean I think I think of language models with most of the things most of the relevant X's are coming senses and embodiment world and self models recurrence Global workspace unified goals those May well be here quite soon maybe I mean there exists working prototypes of each of these now and they're just going to get better it's entirely possible that within say 10 years will have virtual perception language action


00:59:54
unified agents that have all of these features maybe that exceed some crucial minimal some important minimal level of general intelligence like maybe even if they're not at human levels it's entirely possible there'll be something like say mouse levels within the next 10 years so just say well we can first ask what is the chance that we're likely to have systems like that within 10 years I think would not be reasonable to think not unreasonable to think there's a greater than 50 chance we'll have


01:00:23
systems with all those capacities at say at least Mouse level within 10 years we can then also ask conditional on their being a system with all those capacities at Mouse level with virtual perception language action and so on conditional and that will they be conscious again I think it would not be unreasonable to think I mean look no one's certain about anything when it comes to Consciousness but I think would not be unreasonable to have more than 50 Credence in those systems being conscious putting the


01:00:54
pieces together that would give you say a Credence in the possibility of AI Consciousness by 2032 of something over 25 which is not to say yeah not to say that that uh yeah conscious AI is just around the corner definitely but it means it's something that we should take very very seriously I mean there's two big underlying problems here number one we don't understand consciousness of course we don't that's why we're here to figure out Consciousness better but that's just a challenge that uh these okay we need


01:01:26
better scientific and philosophical theories of Consciousness to figure this out the other challenge is we don't really understand what's going on in these huge language models they're so extremely opaque but that's also another challenge we just need better language model interpretability and those are two major approach projects think of these kind of potential like Manhattan projects in this area better theories of Consciousness better language interpretability that might lead us to better grounded predictions here okay so


01:01:54
that then leads to uh to my conclusion which is questions about AI Consciousness are not going away within even within 10 years or so even if we don't have human level artificial general intelligence we may well have systems that are serious candidates for Consciousness and you know meeting the challenges to language model Consciousness May yield a potential roadmap to uh to conscious Ai and I've kind of laid that out here in the form of a road map many of the things you know if we manage to do all of these


01:02:30
things then there would be um some serious chance that we might be getting to conscious AI I do want to repeat though by by um I do want to finish by repeating the ethical uh the ethical part merely because there is this road map it doesn't mean this is a map that we have to uh do we have to follow we I think we do need to think very very hard about the ethical issue do we want to build conscious AI if we do want to build conscious AI then following you know addressing these challenges is one way possibly to get


01:03:02
there if we don't want to build conscious AI which which is entirely possible and these are things that we may want to avoid in our language models either way it's going to be important to know what kind of features of language models are likely to be conducive to Consciousness I mean it may well be that for example if it turns out there's a way to build an artificial philosophical zombie um systems that behave like humans without Consciousness that may in many contexts be be ethically preferable so I


01:03:29
think all of the philosophical issues here are difficult but I think there's a lot we need to be thinking about thank you [Applause] [Music] thank you