Nobody's fault, everyone's problem?
This was originally meant as a talk for Fediday 2026 but I got ill and had to cancel my participation. Since everything was prepared, I figured I’d change the talk in to a blog article.
Hello, my name is Roel Roscam Abbing. I am a design researcher based in Sweden. I’ve also been a co-administrator of a Mastodon instance since 2018.
One thing you might know me from is Low-Tech Magazine’s solar powered site and server, which I helped design and maintain to this day. If not, that doesn’t matter because it is not relevant to today’s story.
However, I recently completed a PhD in Interaction Design at Malmö University. There I looked at the effects of how online federation works in the Fediverse. I studied this by working with stakeholders to prototype new communities using fediverse software. It is based on these experiences that I’m here to talk today.
I believe there is a problem whose contours are becoming visible. My hypothesis is that there is a class of sociotechnical vulnerabilities that sits at the intersections between three things:
- The way people run their instances
- The specific affordances of different fediverse software
- The way data is made to circulate in the federation
These issues are often nobody’s fault. Or at the very least fall between areas of responsibility. But because they exist in a federated system, they may become everyone’s problem.
I will sketch the contours of these issues through three examples.
My first example is a design pattern that exists more broadly in the fediverse. It is a way of implementing federation that results in content rapidly being copied to different servers from where it becomes discoverable or accessible to the public.
Peertube is a good example of this. Peertube instances can follow each other. Once a Peertube instance follows another one, it will make the content of remote instances available locally. So if you want to make your Peertube instance appear lively, you follow a lot and get content for free! This has a snowballing dynamic as well. It used to be that Peertube instances would even automatically accept follow requests, though nowadays you can prevent that by having to manually allow being followed. Peertube is designed so that every instance becomes a portal into the entire network, and this opportunistic circulation of content is probably meant to aid discovery.
In 2023 researchers found videos from deplatformed conspiracy theorists and far-right activists across the Peertube network. Often this material was quite prominently displayed on landing pages. Because of this dynamic of opportunistic circulation, they found that a small number of channels were receiving significant amplification by large instances that were densely connected with other ones. By now that is somewhat mitigated through moderation, but the underlying dynamic remains. This dynamic also occurs in other software such as Lemmy or through Mastodon relays.
Example 2. This one is similar effect but a different motivation. I found this when doing a pilot research project together with Brendan Howell as part of SWRs Reinvent Social Platforms fediverse Fellowship. In that project we looked into media publishers’ attitudes towards long-form content support in the fediverse. For different reasons, we found publishers were not so enthusiastic about the prospect of federating full articles. If professional media are not so enthusiastic about that, we wondered who could be. Looking at the existing long-form fediverse ecosystem more closely, we found out who was: Startlight Princess!
Starlight Princess is SEO spam for an online casino. She is a symptom of a broader phenomenon where Fediverse long-form platforms like Write Freely, Plume and Write.as combine open sign ups with a reader view that publicly shows all posts. The current design of many fediverse servers means that remote content is cached, viewable, and searchable on different instances.
In the case of Plume instances, these posts are even aggregated from across the federation. That is an SEO spammer’s paradise. The Starlight Princess example is annoying but relatively harmless. The same goes for shopping tips for pyrolysis machines.
I wanted to keep this example light-hearted with Starlight Princess and her friends, so I searched for WriteFreely statistics to find more examples. You can quickly find out where to find such materials by looking at instances where user growth does this. Spoiler: this is not the fediverse suddenly becoming popular.
Yet the recent examples I found were not so lighthearted. Right now there is a wave of bots posting Islamophobic articles, or articles about Christians being murdered. All of this is interwoven with links to men’s rights activism, the ressamblent national and, as you can see here, the identitarian movement. Speaking of good contextual ad placements, it even features everyone’s least favorite distro, omarchy. This just goes to show that these issues and the sociotechnical vulnerabilities they reveal are serious in the end.
My third example is actually the work of Jaz-Michael King at IFTAS. I follow his work closely as part of co-administrating a Mastodon instance, and this is something I recommend you do too. In this example he found a very interesting pattern of messages, which he suspects is part of a Russian influence operation.
These are automated accounts that sign up to fediverse servers and then start posting about current events. They look rather normal on the surface, and this is probably also why they slip through the cracks when requesting accounts.
The general current events posting is aimed at gathering followers. The thing they really want to spread though is the occasional link to Russian regime-aligned media and / or Telegram channels. However, the signature move of these these bots is that they only follow one account: namely Bridgy Fed.
That is because the goals of these accounts are not to spread things in the fediverse, but to do it in the ATmosphere. This is such an instructive example because the attack chain involves signing up on a fediverse server, enabling the bridge, and then posting to the ATmosphere.
So I come back to this image. The three examples I discussed show different aspects of it. The first example shows that the affordances of fediverse software and how data is made to circulate through online federation can be exploited to amplify questionable content. The second example shows how instance policies (or lack thereof) together with the dynamics of the first affect systems outside of the fediverse: namely search engines. However, it also means the quality of material available on that part of the fediverse is just bad. The third example then shows an attack chain combining the three. The more important thing is that it was an exploit that just used things the way they were meant to be used.
However, at the same time, all the examples are also quite self-contained, despite interoperability between different types of fediverse platforms. Peertube content generally stays on Peertube and long-form spam is invisible to anyone on Mastodon because there is not yet any infrastructure for circulation. But it is likely that someone will build it at some point.
So where does this leave us? We need an integrated, cross-platform, ecosystem-wide approach to Trust and Safety. Because this whole situation has some specific effects:
-
Emergence : Threats are emergent and, to an extent, unknowable beforehand. Yet threat modeling and foresighting can uncover and mitigate issues before they appear.
-
Ecosystem effects : Threats resulting from ecosystemic interactions are not likely to be mitigated in a single software. This requires coordination.
-
Weakest links : Quality of experience of the best applications may be impacted by the worst. This is a core risk to fediverse adoption as well as growth.
-
Out of band : Fediverse may impact or be impacted by developments outside the fediverse ecosystem. GenAI accounts signing up is one thing, but the reputational damage to the fediverse for being an unmoderated breeding ground for spam or worse is another.
-
New ground? : Best practices from centralised systems are only partially applicable to federated environments. No one has a full view of threats; you cannot necessarily trust other instances. It is a specific environment. As a researcher I will always say, “More research is necessary”!
So what are some next steps? First there is the W3C ActivityPub Trust and Safety group, which is short of hands. I started to get involved with it, and one of the things I am working on is an overview of T&S “best practices” that developers and operators are currently implementing.
The second would be to follow and support the work of IFTAS. They are doing incredibly valuable work in elevating the baseline safety level of the entire ecosystem, and they need support.
Third, I’m looking for funding to get a sustained and comprehensive research project for this off the ground. Are you interested in this, either to collaborate or fund it? Let’s talk!
