ACTRIAL: Wikipedia volunteer leadership should be recognized

The Wikimedia Foundation (WMF) is inviting commentary on how to recognize and encourage informal leadership in the volunteer community. (The consultation runs from September 20 through October 16, 2016.) This is a welcome initiative; the Wikimedia movement has not done well, over the years, at capturing stories of volunteers who successfully focus attention on important areas, and who do good work building consensus and forging and executing plans.

There are exceptions. The announcement linked above hails achievements by Liam Wyatt and Vassia Atanassova; and the WMF has consistently highlighted successful volunteer-driven projects on its blog and elsewhere. Still, many who have taken on big challenges and risks to advance our shared values and vision go unheralded.

Below, I consider an important 2011 initiative and conflict, known informally as ACTRIAL (short for “Article Creation Trial”). I learned of it many months after it took place; I had to jump through jargon-filled discussions in multiple venues before I began to understand what had happened. It’s an important story, though. In recent years, WMF staff have often asserted that Wikipedia’s volunteers are “change averse,” and incapable of generating or agreeing on new ideas. But the characterization is neither fair nor accurate, and is typically asserted out of mere political convenience.

ACTRIAL, however, provides a clear and valuable counterexample, in which Wikipedians self-organized to advocate for change, and WMF staff blocked the effort. This post highlights an important piece of Wikimedia history, and offers a little recognition to unsung heroes.

The Blade of the Northern Lights, a Wikipedia volunteer, wanted to address a persistent problem that irks many regular Wikipedians: brand new Wikipedia volunteers who write articles that are far from meeting Wikipedia’s content standards.

The Blade proposed that the creation of new articles be restricted to users with a bit of experience (“autoconfirmed”: 4 days, 10 edits), and then guided more than 500 English Wikipedia volunteers in considering and ultimately approving the proposal.

ACTRIAL was designed as a six-month experiment rather than a definitive policy change. That point is an important one; the WMF often encourages volunteers and affiliate organizations to test hypotheses before making long-term commitments. Volunteer Rich Farmbrough, in spite of his skepticism about the proposed change, praised ACTRIAL’s design as an experiment in 2014, and explained the significance:

I am against preventing article creation by IPs let alone non-autoconfirmed users. But this trial might well have provided compelling evidence one way or the other.

Once the English Wikipedia community had agreed to move forward with ACTRIAL, Scottywong, another Wikipedia volunteer, formally requested a necessary technical change. In a haphazard discussion driven by WMF staffers, the request was denied. Apparently ignoring the extensive deliberation that had involved hundreds of volunteers, one WMF employee stated: “this entire idea doesn’t appear to have been thought through.” Several seemed to agree that the proposal was at odds with the strategic goal of improving editor retention, though no clear argument supporting that position was advanced.

To date, the WMF has not explained this extraordinary rejection of a good-faith, volunteer-driven initiative. The closest approximation to an explanation was a mailing list discussion in 2014. In that discussion, then-WMF staffer Philippe Beaudette asserted that the WMF had ultimately solved the underlying problem in another way:

What I remember was that a pretty good number (~500) of [English Wikipedia] community members came together and agreed on a problem, and one plan for how to fix it and asked the WMF to implement it. The WMF evaluated it, and saw a threat to a basic project value. WMF then asked “what’s the problem you’re actually trying to solve?”, and proposed and built a set of tools to directly address that problem without compromising the core value of openness. And it seems to have worked out pretty well because I haven’t heard a ton of complaints about that problem since.

However, Beaudette’s statement had several problems (edited: see note below):

  • If there was indeed an evaluation, it was never made public.
  • While several individuals argued that a “basic project value” was at risk, no decisive case was made, nor any formal conclusion presented. Others disagreed, and the matter was never resolved decisively.
  • If the WMF had asked “what’s the problem you’re actually trying to solve?”, the question was not (as far as I can tell) posed in a public venue.
  • It’s unclear what “set of tools” were developed; but regardless of what that was referring to, any claim that it “worked out pretty well” should have been evaluated by a more robust process than listening for complaints. As volunteer Todd Allen said: “You haven’t heard more complaints, because the complaint was pointless the first time and took a massive effort to produce.”

When I requested clarification, Beaudette did not respond; but James Alexander, another WMF staff member, did step in. Alexander speculated that the software inspired by ACTRIAL was the Page Curation tool. He may have been correct, but no other staff member confirmed it in that email thread; and neither the page on Page Curation nor its parent page on the Article Creation Workflow make any mention of the discussion of the 500+ volunteers that may or may not have have inspired them.

The sequence of events around ACTRIAL has not been publicly documented by the Foundation – and accordingly, five years later, the leadership shown by the volunteers who guided the discussion remains unrecognized. The initial reactions of WMF staff, therefore, loom large in volunteers’ memory. As Rich Farmbrough opined: “The dismissal as a ‘we know better’ was a bad thing.”

Now that the Foundation seeks to explore stories of leadership in the Wikimedia movement, it would do well to look into the story of The Blade of the Northern Lights and Scottywong. These two played important roles in guiding the English Wikipedia community to define a problem, and to map out a viable (if unimplemented) way to explore a solution. Moreover, if any Wikimedia Foundation staff worked with the volunteer community to make something worthwhile out of those deliberations, their leadership merits recognition as well.

Note: This blog post does not cover the role of Erik Moeller, then the Deputy Director of the WMF. He made a couple of substantive comments in the discussion. Some of them speak to Beaudette’s points, and the extent to which the WMF engaged with the problem surfaced in the volunteer community. I will update this post or follow it up with more detail when I have time. -Pete

About Pete Forsyth

Pete Forsyth is the principal of Wiki Strategies, and a Wikipedia expert. Full bio here:
This entry was posted in governance, history, leadership, Uncategorized, wiki, Wikimedia Foundation, Wikipedia. Bookmark the permalink.
  • Federico Leva

    WMF didn’t do anything, I take responsibility of this WONTFIX and I confirm I was right, based on research.

  • Can you expand on that? You had some worthwhile comments in the initial discussion, would love to hear your take now that some years have passed.

  • Scotty Wong

    Glad to see that the legacy of the ACTRIAL debacle still resonate throughout the halls of Wikipedia. (I’m the Scottywong mentioned in the article.) The article is an accurate portrayal of what happened. The ironic part is that WMF claimed to reject our request for a trial because they feared it would have a negative impact on new editor retention, yet they neglected to realize the effect that their disrespect of the volunteer community would have on existing editor retention.

    After this incident, I eventually became an administrator on WP (and technically still am), so I think it’s fair to say that I was a member of “Wikipedia leadership” at the time. I have since retired from Wikipedia and rarely make any edits there any more. One part of the reason (but certainly not all, or even a majority) was the feeling of disillusionment that ACTRIAL provided to many of the editors involved.

    I remember trading a few emails with Jimmy Wales at the time. He actually agreed with ACTRIAL and thought that something along those lines should be allowed to happen. He disagreed with some of the details of the implementation, but still thought that the trial should happen. (He preferred to see some kind of tutorial system get developed, where new editors couldn’t create a new article until they progressed through a tutorial and passed a quiz, rather than tying new article creation to autoconfirmed status). He also admonished Brandon Harris (a.k.a. Jorm) for his histrionics and rude comments at the bugzilla request, and disagreed with many of his assertions.

    In the end, neither the trial nor Jimmy’s tutorial idea were ever implemented. The WMF overruled everyone and implemented the page curation tool. I was so disgusted with the whole thing that I never really tried the page curation tool, and completely withdrew from patrolling new articles and trying to solve the problem, and moved on to other problems that didn’t require developer access.

    It’s heartening to see that the WMF is starting to realize that they need to recognize their volunteer leaders, praise them, and listen to their ideas. If only they had figured that out half a decade ago.

  • Scotty Wong

    There was never any research done to determine whether or not WMF was right or wrong. ACTRIAL was the proposed research. The only way to get real data about its effects was to do the temporary trial. Whatever research you think you did (whoever you are) is nonsense.

  • Federico Leva

    I don’t think this is nonsense, for instance:

  • Scotty Wong

    First of all, that research was done a full 2 years after the ACTRIAL request. Therefore, that research couldn’t possibly have been used at the time as evidence that ACTRIAL would have caused damage to the project.

    Secondly, many of the conclusions of that research are actually in support of ACTRIAL, and suggest that it would still be helpful. For instance, “the more experience an editor has, the more likely their articles are to survive”. ACTRIAL was an attempt to force new users to gain a minimal level of experience before attempting to create a new article in mainspace, in the hopes that it would increase their chances of success.

    Your research also lauds the effect of “WP:Articles for creation” on the success rate of new editors. ACTRIAL was designed to specifically point new users to AFC if they really wanted to create a new article before becoming autoconfirmed.

    None of this research proves anything about whether ACTRIAL would have caused harm or good. Again, ACTRIAL was the actual research project that would have been able to prove that one way or another. ACTRIAL was a temporary trial, not a permanent change.

    Again, I don’t know who you are, but your attitude is very reminiscent of the “shut up because WMF knows best” attitude that caused this whole issue in the first place.

  • Thank you for linking that research, @federicoleva:disqus. I had missed it in researching this blog post; there’s still IMO a lot of work to be done to connect the dots so that Wikimedians can see how this stuff fits together. This blog post was meant to be a step in that direction, but ideally I think it’s a significant enough event that there should be an overview at – I may start it myself, but I’d rather see somebody who knows the topic better involved.

    I basically agree with the things that @scottywong:disqus said in response, though I should qualify that by saying I haven’t read Aaron’s report in full yet, and might not get to it for a while. But he appears to state the problem well and take a sensible approach to exploring it, and I have a lot of respect for his research work in general.

  • I want to strongly disagree with your final paragraph. I think Federico and I might have a significant disagreement on this topic (still assessing that), but apart from that, he is one of the most careful and independent thinkers I know in the Wikimedia world, and has not to my knowledge ever worked for WMF. I don’t want to presume it’s OK to link s account, but it’s no big secret…a little web searching should get you the answer.