Finding a North Star for UN AI Governance

Humanity is already shaken by AI. However, AI’s impact on the exercise of human rights should not be a matter of chance, but rather a matter of choice. The key question is who is called to make those choices? AI governance cannot mean product safety alone and it certainly should not mean leaving the protection and fulfilment of human rights and the preservation of human dignity, to isolated, unverifiable, and unevenly applied voluntary commitments. As the outgoing UN Secretary General has stressed “this requires genuine international cooperation and a coordinated global effort”.

The Independent International Scientific Panel’s Preliminary Report and the Co-Chairs Summary of the Global Dialogue on AI are building blocks of what should be the trajectory of the AI governance roadmap inside the UN. What do they tell us about convergences and priorities? How useful are they for understanding the gaps in evidence and consensus that need to be filled? How do they interact (or not) with the recent public calls on AI development pacing and voluntary commitments on external oversight?

Starting with the priorities and areas of convergence

Children first. The Co-Chairs’ Summary highlights the Secretary-General’s AI Child Safety Pledge and the new Coalition for Children’s Rights and Protection in the Age of AI. The Scientific Panel backs this with hard data on sexualised deepfakes, AI-generated abusive content, and sycophantic interactions leading to suicidal behaviours. The key open questions here are what concrete governance levers need to be developed to tackle these differentiated impacts for children, where in the AI life cycle they need to be placed in order to be effective, and who in the multilateral system will be leading this work. 

Environmental cost as a governance question, not a footnote. While the report states that the rapid expansion of AI is driving demand for digital infrastructure, increasing energy and water consumption, greenhouse gas emissions, pressure on critical mineral supply chains and e-waste, it points to the lack of standardised evidence collection to measure those impacts. The co-chairs summary elevates environmental sustainability as a key theme raised during the dialogue as a governance question. Therefore, treating environmental impacts as an integral part of AI governance is a positive step, but it requires standard evidence collection to enable meaningful accountability. 

Agentic AI is stress testing governance thinking. The Co-Chairs’ Summary captures the accountability challenges from autonomous agents as a rising governance challenge heard during the human rights cluster discussions. The Panel, by contrast, devotes a full section of its report to this, citing evidence like AI coding agents being tricked into running malicious commands, systems that have violated shutdown instructions in lab settings or produced exploits escaping human control. The Panel has also published a thematic brief on AI agents’ risks in complement. While both recognise the issue of agentic AI as distinctive, what they are less effective at unpacking is the question of avoiding the specificities of agentic AI overtaking the broader AI governance agenda. In other words, agentic AI systems as unpredictable products might deserve regulatory approaches closer to traditional methods of dangerous product regulation such as market recalls or strict liability regimes for providers that might not be necessary or adapted for other types of AI, but those distinctions risk being lost amid the hype around frontier and agentic AI, dangerously narrowing the focus of governance debates at the expense of a more comprehensive AI governance strategy. 

Now, moving into the gaps in evidence and consensus:

The place of human rights framework in AI governance. The Co-Chairs’ Summary treats human rights law as the normative foundation that governance instruments should be built on or interoperate around. They are positioned as a viable answer to the interoperability question that remains open. However, the Panel treats human rights framework more as one category of useful governance frameworks (praised for offering tools for due diligence, impact assessments and remedy) but putting a question mark in its effectiveness for the lack of evidence. None of them question the value of the human rights framework but they fail to make the case on its effectiveness for AI governance. 

This is  concerning, but not new. The gap here is related to the lack of concrete examples in which the human rights framework is effectively used to pursue AI accountability. In this sense, it is disappointing that the Co-Chairs Summary does not sufficiently account for the human rights AI red lines call that was repeatedly shared during the dialogue, precisely as a way forward to demonstrate what the human rights framework has to offer as baseline for AI accountability, by enforcing prohibition or moratoriums of AI systems and capabilities that are impossible to operate in a manner compatible with human rights. 

Multilateral role to provide interoperability rather than harmonization. Even if both, the Co-Chairs’ Summary and the Scientific Panel report lean towards interoperability of AI governance regimes, they both miss the mark in spelling out what kind of elements would allow that interoperability and more importantly who would have the responsibility for developing them. As GPD has been arguing, it is precisely that role that the UN system is particularly well suited for. The UN is a natural house for setting the parameters for: incident and transparency reporting standards, best practices in testing and auditing capabilities, and mechanisms of exchange for building technical capacity to govern AI and to develop AI in a manner that is adapted to local needs and context. Abandoning this responsibility in companies cannot provide the independent assessment and the transparency that is required for the governance to be effective. 

The interoperability can be built in an incremental manner (as the experience of dealing with nuclear energy and commercial aviation demonstrated in the past). As a building block, what seems critical today is moving from the patchwork collection of snapshots of evidence by civil society, academia or the press reporting incidents to creating methodologies for accounting for lived experience, particularly in the Global Majority, in a way that is useful for filling the evidence gaps identified by the Panel. What isn’t yet measured creates gaps for interoperability when systems are designed and produced in one jurisdiction but deployed in a different one. Methodologies are also needed to measure how individual-level AI impacts aggregate into societal outcomes (epistemic erosion, civic participation, social cohesion). Interoperability can be built incrementally in evidence collection, technical capabilities and normative requirements. 

Then there is the elephant in the room: the concentration of power. Both documents make almost identical arguments: the Co-Chairs Summary describes a shift “from access to capacity,” while the Scientific Panel report frames the AI divide as “not just about access, but about capacity to influence AI development”. Both note that countries dependent on foreign models and cloud infrastructure gain access to AI use while entering blind on assessment and safeguards. Why is there no direct call in the Co-Chairs’ summary to the market power concentration impact in governance? The word concentration is used twice in the summary, one time a reference to the Panel’s Preliminary Report findings and the other summarising interventions on the Cluster 4 on human rights. 

While attending the Dialogue we did hear repeatedly market power concentration being raised as an issue by many actors. The co-Chairs’ summary chose to cover it only through the lens of a divide and equity issue for nation states. But no reference to how this threatens democratic life (electoral interference, national policies pressure, service dependency). There is also the absence of discussion of how that dominance harms the rights or livelihoods of people. The Scientific Panel’s Preliminary Report took a rather bolder approach by capturing the market power concentration in its findings as a risk that needs to be understood and managed. It is defined as a risk that impacts democratic accountability not just an equality problem. 

Under pressure to deliver 

As the Human Rights Commissioner responds to the industry announcements, “AI safety must mean more than the reliability of the technical product in question. It means actively protecting people, communities, institutions and future generations”. States and the multilateral system are responsible for delivering this, not private companies driven by profit and raising capital investments. 

How do we get there? By the time the Global Dialogue reconvenes in May, political will should focus on three incremental priorities: first, the standardisation of evidence collection for AI impacts; second, the willingness to unpack and enforce AI red lines when systems, uses and performances are incompatible with human rights obligations; third, starting to build national strategies and cooperation mechanisms to produce AI aligned with the public interest, walking away from a technology exclusively determined by commercial interests, and reducing dependency on systems that today are 90% anchored in only two jurisdictions.