I've been working through Pope Leo's encyclical on AI this week. It's a fascinating, thorough, and well-reasoned document, arguably the first major 'statement' on AI from outside the tech world.

Leo rightfully focuses most of his attention on systemic issues: that AI, and technology in general, be developed conscientiously so as to promote human flourishing, rather than being a tool of accumulation and domination.

But there's an interesting ambiguity when he turns, in paragraphs 98 and 99, to the nature of AI itself.

Paragraph 98 is all epistemic humility, admitting that there are core aspects of AI that we don't fully understand or control. LLMs:

"are more 'cultivated' than 'built,' for developers do not directly design every detail, but instead create a framework within which the intelligence 'grows.' As a result, fundamental scientific aspects — such as the internal representations and computational processes of these systems — remain, at present, unknown."

We have to be careful making definitive statements about what AI is because we could be proven wrong tomorrow.

Paragraph 99, though, discards that humility to confidently declare what AI is not:

"[t]hese systems merely imitate certain functions of human intelligence ... do not undergo experiences ... do not know from within what love, work, friendship or responsibility mean."

AIs simulate certain aspects of humanity, but they do not possess them.

It makes sense for Pope Leo to take this position given that he is the foremost representative of the tradition of Christian thought which regards human beings as created in the imago dei, and thus superior to all other creatures, and certainly to made things. It's also worth noting that the Pope, in spite of the criticism he's been receiving, here aligns with the consensus in the tech world that LLMs are best understood as tools rather than a new class of entity.

But it's hard for me as a reader to not see paragraph 98 as undermining the confidence with which paragraph 99 declares what AI is not and can never be. Given how carefully written the text is, it's hard not to believe this effect was intentional.

It's an interesting wrinkle at the heart of the text, one that's maybe easy to gloss over given the urgency of so much of what Leo's talking about.