Are Chainalysis’ Tools Scientifically Sound? Executive’s Statement Casts Doubt

An executive for on-chain analytics firm Chainalysis was the source of controversy following a recent statement. According to a court filing, the company’s Director of Investigation Solutions for government applications, Elizabeth Bisbee, provided testimony in the case against Roman Sterlingov.

The alleged leading operator of Bitcoin Fog, a platform accused of enabling money laundering services via the cryptocurrency. Sterlingov is accused of money laundering, fraud, and other alleged criminal activities, and the crypto analytic platform is playing a pivotal role in supporting the case against the defender.

Chainalysis Methodology Under The Spotlight

Bisbee was a former agent of the Drug Enforcement Administration (DEA) and their department for cryptocurrency investigations. The executive has been working with the crypto company since 2021 and spoke to the court about its methodology and the validity of its tools.

Chainalysis uses different services and tools to dive deeply into the Bitcoin blockchain, mainly de-anonymized transactions. These tools allow their clients to connect on-chain activity to real-life individuals.

However, Bitcoin users have found ways to avoid these tools, which is at odds with Chainalysis and Bisbee’s statement. The company uses clustering transactions, co-spend heuristics, and others to analyze BTC transactions.

While these tools have been used by the U.S. government and other major entities, Bisbee stated:

Chainalysis clustering methodologies have not been peer-reviewed in the sense that an academic paper would get peer-reviewed with data and methodology(ies) reviewed in a separate study by other scientists.

The executive clarified that their system had been reviewed by “numerous data scientists, intelligence analysts, and investigators” within the company. In addition, Bisbee claims that other entities can independently review their data.

Is The Biggest Crypto Surveillance Company Making Mistakes?

The court also inquired about Chainalysis’s margin of error, false negatives, and other measures of potential mistakes. The company stated the following via its representative:

Historically, Chainalysis has not gathered and recorded in a central location false positives /false negatives because there is design to be more conservative in the clustering of addresses. (…) Chainalysis is looking into the potential of trying to collect and record any potential false positives and margin of error, but such a collection does not currently exist.

This statement suggests that the company never implemented a system to keep track of potential mistakes/false positives. This claim and many others have been the source of controversy; Chainalysis has constantly stirred debate in the Bitcoin community due to its activities and alleged government ties.

Developer of OXT, the on-chain analytics tool, LaurentMT classified the crypto company’s statement as a “huge pile of bullshit.” Like many other Bitcoin developers, Laurent believes Chainalysis tools “lack scientific rigor.”

Via Twitter, Laurent stated:

(…) this declaration is a huge pile of bullshit trying to hide the complete lack of science behind the heuristics used by blockchain analytics platforms…

Another pseudonym on-chain analyst and researcher, ErgoBTC, added the following about Chainalysis potential intentions:

It’s not about building trust(TM), it’s about grovelling at the feet of the bureucrats in permanent security state and building a multibillion dollar cash cow on the backs of the taxpayer.

As of this writing, Bitcoin (BTC) trades at $29,300 with sideways price action over the past week.

Cover image from Unsplash, chart from Tradingview