CrowdTangle is gone. The replacement story is a warning about tool lifespans

September 11, 2026 · ThreadGrab

Every so often a story lands that is nominally about somebody else's product and is really about a structural risk. This week's is Nieman Lab's September 9, 2026 report on Arbiter, an AI tool from the nonprofit SimPPL that is trying to occupy the space Meta vacated when it shut down CrowdTangle in 2024. I read the whole thing expecting a straightforward product profile. What I got was a clear-eyed account of what it costs to depend on a tool you do not control.

That is the thread I want to pull, because it applies well beyond journalism. Anyone who monitors, captures or cites public social content is running the same risk, and the industry has spent the last three years proving the point repeatedly.

What CrowdTangle was, and what replaced it

CrowdTangle was the tool journalists, fact-checkers and researchers used for more than a decade to watch how content spread across Facebook and Instagram. Meta disinvested slowly, then announced in March 2024 that it would cease operations in August. A petition signed by more than 100 watchdog groups warned, in the words Nieman Lab quoted, that "almost all outside efforts to identify and prevent political disinformation, incitements to violence, and online harassment of women and minorities will be silenced."

Meta did ship a successor: the Meta Content Library, released in November 2023. The reception has been mixed at best. Nieman Lab notes it has been criticized for restricting journalists' access and for removing the automated insights and sophisticated search features that made CrowdTangle useful in the first place. Duuya Baatar, co-founder of the Nest Center in Ulaanbaatar, put the difference in plain terms for the piece: if you want to look at a specific post or narrative, the Meta Content Library is fine; if you want to analyze how an information ecosystem is changing, it is nowhere near CrowdTangle.

That gap is what Arbiter is aimed at, and the shape of the tool is worth understanding because it tells you where the hard parts are:

Why the founder built it independently, in his own words

Here is the part that should interest anyone who has ever built a workflow on top of somebody else's data access. Swapneel Mehta, co-founder and president of SimPPL, explained his reasoning to Nieman Lab, and it is a design constraint stated outright:

"CrowdTangle is not a service that was mandated or regulated to continue being offered at any point. Platforms could decide that this was not a net positive and stop offering it."

He follows it with the position he is building toward: "These are public conversations, and collecting public conversations in the public interest should not be illegal."

There is a second quote that tells you how the pipeline actually works under load, and it is refreshingly unglamorous about the role of AI:

"Anyone who's dealt with more than 100,000 pieces of data knows that throwing them at an LLM first blows up your credit card bill and then gives you garbage outputs that you can't control."

Arbiter's answer is not one model but eight to ten of them, each doing a different job — encoding queries, retrieving the right posts, identifying coordinated networks, measuring semantic similarity, mapping networks. Some layers are model-agnostic and newsrooms can bring their own LLM provider. That is a sensible architecture, and it is also a reminder that "we added AI" is not the same as "we solved retrieval."

The uncomfortable part: the access is borrowed

Even a well-built independent tool is downstream of the platforms. Arbiter went through months-long review processes to get API access to Facebook and Instagram, and it remains subject to Meta's rate limits — using the API alone, it can only pull so many posts at a time. To widen that aperture, Mehta described working with third-party data providers holding vendor agreements, identifying carve-outs for collecting public data without violating terms of service, and leaning on data donations from academics and civil society organizations.

He also does not romanticize the arrangement. Mehta told Nieman Lab that these processes often start favorably because a small organization does not look like a management concern, and that "as you grow, the kind of work you do changes." The team has lawyers reviewing its API workarounds for vulnerabilities, which is a reasonable precaution given the history: X sued the Center for Countering Digital Hate in 2023 over alleged scraping and use of a third-party listening tool, a case that was later dismissed.

The UK report says the quiet part out loud

Two days before the Nieman Lab piece, Tech Policy Press published a September 8, 2026 essay tied to a UK Social Platforms Data Access Taskforce report. It names the condition directly, describing what it calls a post-API environment and a growing mismatch between the demand for independent evidence about platforms and the routes available to produce it. The same essay records the timeline: X withdrew its free research API for academic researchers in 2023, and Meta closed CrowdTangle the following year.

Its conclusion is the one worth carrying forward for anyone who archives social content, whatever their profession:

"Web scraping is already an established research method and remains one of the few ways researchers can independently collect publicly accessible platform data."

Read that next to the Nieman Lab reporting and you get a coherent picture. The official doors have narrowed. The remaining doors — scraping, vendor agreements, data donations, third-party providers — are the ones being leaned on, and they are the ones with the most legal and technical uncertainty attached.

So what should an archivist actually do differently?

Nothing about this story changes the fact that monitoring tools are useful. Arbiter's ability to flag an emerging narrative weeks early is genuinely valuable — the piece recounts a Mongolian newsroom that used it to spot accounts pushing a "libel and defamation" framing of a parliamentary process before the working group was announced. That is real signal.

The lesson is about which layer you treat as durable. Concretely:

1. Separate discovery from the record. A monitoring tool's job is to tell you something is happening. Your archive's job is to hold what was actually said. Those are different systems with different lifespans, and only one of them is under your control.
2. Store the post text, not just the pointer. A saved search result or a permalink is a reference into somebody else's system. The words, the author and the timestamp are the evidence.
3. Assume any access route can close. CrowdTangle was load-bearing for more than a decade and it closed. A free research API closed. Rate limits tightened. None of that was announced as a catastrophe; it was just a decision someone else made.
4. Keep the archive portable. Files in a format you can read without a vendor's dashboard are the difference between a record you own and a record you are borrowing access to.
5. Re-verify before citing. If a citation depends on a token, a handle or a tool that existed when you captured it, check that it still resolves the way you described.

A small script captures the distinction better than prose. ThreadGrab exposes public profiles as JSON, which you can flatten into Markdown that survives the tool that found it:

# Capture the posts themselves, not a pointer to a dashboard
curl -s https://threadgrab.com/api/profile/USERNAME \
  -H "Accept: application/json" \
  | jq -r '.posts[] | "## " + .created_at + "\n\n" + .text + "\n\nsource: " + .url + "\n"' \
  > archive/public-thread-USERNAME.md

# Record the observation date, so the capture is a dated claim
date -u +"captured_at: %Y-%m-%dT%H:%M:%SZ" \
  >> archive/public-thread-USERNAME.md

The second command matters more than it looks. A file that records when it was captured is a statement about a moment. A file that does not is a claim about the present that quietly expires.

And if you already have a folder of captures whose provenance you are unsure about, this check finds the ones that depend entirely on a live external service:

# List captures that store only a link and no post text of their own
for f in archive/*.md; do
  if ! grep -q '^## ' "$f"; then
    echo "pointer-only capture: $f"
  fi
done

# Anything printed above depends on a service staying up.
# Re-fetch each one and store the post text alongside the link.

What to watch next

• Whether Arbiter's revenue model holds. All the newsrooms trialing it are doing so for free, and Mehta told Nieman Lab he is conscious that many of the organizations that need it are cash-strapped. A free public-interest tool with a nonprofit behind it has a different durability curve than a vendor product — better in some ways, worse in others.
• Whether platform API access widens or narrows. Arbiter's coverage depends on carve-outs and vendor agreements that are not guaranteed to persist. The UK taskforce work is the policy-side expression of the same question.
• Whether scraping liability gets tested in court again. The CCDH suit was dismissed, but the legal perimeter of collecting public conversations is still being drawn, and tool builders are the ones finding out where the lines are.
• Whether Meta Content Library access improves. If it does, the urgency of independent alternatives drops. If it stays restricted, expect more SimPPL-style projects.

The limitations of this article

• Everything here rests on secondary reporting. I did not test Arbiter's output, and I cannot verify its coverage claims independently. The quotes are reproduced from Nieman Lab's September 9, 2026 piece and Tech Policy Press's September 8, 2026 essay.
• Pricing and access terms are moving targets. Nieman Lab reports that all current newsroom use is free and that SimPPL is still exploring revenue streams. That can change; re-check before relying on it.
• "Post-API environment" is an argument, not a measured statistic. It is a fair characterization of a series of narrowing decisions, but it is a framing rather than a metric.
• I have not used CrowdTangle or the Meta Content Library. My description of what each did well comes from the reporting and from named practitioners quoted in it, not from hands-on comparison.

Frequently asked questions

Last verified: September 11, 2026. Primary sources: Nieman Lab, "Two years ago, Meta killed CrowdTangle. Can a new AI tool fill the void?" by Andrew Deck (September 9, 2026); Tech Policy Press, "Why the UK Needs Better Social Media Data Access to Research Online Harms" (September 8, 2026); SimPPL project pages at simppl.org and arbiter.simppl.org; and Nieman Lab's 2024 coverage of CrowdTangle's shutdown. All named quotations are reproduced from the Nieman Lab and Tech Policy Press reporting linked above. Tool pricing, access terms and platform API availability should be re-checked before relying on them.