Your Backtest Is Lying: How Cohort Data Exposes the Gaps

You ran the backtest. Open interest spiked, funding flipped negative, your signal fired, and the simulated P&L came back positive. The Sharpe looks clean. The win rate holds across 200 trades. So you ship it live, and within two weeks the strategy is bleeding.

The problem is rarely the signal itself. It's what the signal can't tell you. A funding flip is a funding flip regardless of whether the wallets pushing it are historically profitable traders or serial liquidation victims. An OI spike looks identical whether it's driven by Smart Money cohort wallets accumulating a position or by Giga-Rekt wallets doubling down on a losing thesis. Your backtest treated both scenarios the same, because it had no way to distinguish them.

That distinction is the gap cohort data fills. HyperTracker classifies every wallet on Hyperliquid into one of 16 behavioral cohorts, 8 by account size (Shrimp through Leviathan) and 8 by all-time PnL (Money Printer through Giga-Rekt). When you layer that classification into a backtest, you stop asking "did the signal fire?" and start asking "who is on each side of this trade?"

The Blind Spot in Standard Backtests

Most backtests on Hyperliquid use price, volume, open interest, and funding rates. These are the inputs available from the exchange's public API, and they're good enough to build directional signals that test well in isolation. The issue surfaces when you look at what they miss.

Consider a simple long signal: OI rises sharply while funding stays flat or slightly negative. The standard interpretation is that new money is entering long positions without paying excessive carry, which suggests genuine conviction rather than leveraged speculation. In a backtest, this pattern produces a modestly positive edge over hundreds of trades.

But aggregate OI doesn't tell you whose conviction you're seeing. If the OI increase comes primarily from the Money Printer cohort (wallets with $1M+ all-time profit, segment ID 8) and the Smart Money cohort ($100K-$1M profit, segment ID 9), you're probably looking at informed positioning. If the same OI spike is driven mostly by Giga-Rekt wallets (below -$1M lifetime, segment ID 15) and Full Rekt wallets (-$1M to -$100K, segment ID 14), the "conviction" is more likely capitulation or revenge trading.

Both scenarios produce the same OI chart. Both fire the same signal. But the expected outcome is meaningfully different, because the quality of the capital behind the move matters.

What Historical Data Covers (and Where It Stops)

Before you can add cohort context to a backtest, you need to understand what data is actually available and how far back it goes. HyperTracker's historical endpoints have different lookback windows depending on the data type.

The deepest history is in positions. Individual position snapshots go back roughly 10 months to April 2025, which gives you the raw material to reconstruct what the market looked like at any point. Fills (individual trade records) cover about 6.5 months from July 2025, useful for building price series and analyzing trade-level execution.

The cohort-specific data has a shorter window. Per-coin cohort metrics, the endpoint that tells you how each segment was positioned on a given asset, offers about 4 weeks of lookback per API call. This is the critical constraint for cohort-based backtesting: you can't retroactively query what the Money Printer cohort was doing on BTC six months ago unless you collected that data at the time.

Cohort bias, which measures directional lean across all assets for a given cohort, is even shorter: a rolling 12-hour window. Order snapshots (stops, take-profits, limit clusters) go back roughly 3 weeks. Heatmaps and leaderboards are current-only, with no historical archive.

Builder tip: The practical implication is simple. If you want to run cohort-aware backtests with meaningful depth, start collecting the data now. A cron job calling the cohort metrics endpoint every 5 minutes and writing the responses to a database gives you a month of data in a month, three months in three months, and so on. The API's 4-week lookback window is a rolling window, so earlier data falls off unless you store it yourself.

Layering Cohort Filters into an Existing Backtest

You don't need to rebuild your backtest engine from scratch. The cleanest approach is to add a cohort filter as a post-signal gate. Your existing signal generates candidates, the cohort filter decides which candidates to execute and which to skip.

The filter works in three stages:

Stage 1: Raw signal fires. Your existing logic, whatever it is, identifies a trade opportunity. OI spike, funding divergence, price breakout, moving average cross. The signal doesn't change.

Stage 2: Cohort lookup. At the moment the signal fires, query the cohort metrics for the relevant asset. You're looking at two things: which cohorts are positioned in the direction of your signal, and which cohorts are positioned against it.

Stage 3: Conviction gate. If the historically profitable cohorts (Money Printer, Smart Money, Consistent Grinder) are aligned with your signal's direction, execute the trade. If the historically unprofitable cohorts (Giga-Rekt, Full Rekt, Semi-Rekt) are the primary drivers and profitable cohorts are positioned against you, skip it. If the picture is mixed, reduce size or pass entirely.

Here's what the lookup and gate logic looks like in practice:

import requests

BASE_URL = "https://ht-api.coinmarketman.com/api/external"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

# PnL-based cohort segment IDs
PROFITABLE = [8, 9, 10]   # Money Printer, Smart Money, Consistent Grinder
UNPROFITABLE = [13, 14, 15]  # Semi-Rekt, Full Rekt, Giga-Rekt

def fetch_cohort_metrics(coin):
    resp = requests.get(
        f"{BASE_URL}/cohort/metrics",
        headers=HEADERS,
        params={"coin": coin}
    )
    resp.raise_for_status()
    return resp.json()

def cohort_conviction(metrics, signal_direction):
    """
    Returns 'pass', 'skip', or 'reduce' based on
    whether profitable cohorts align with the signal.
    signal_direction: 'long' or 'short'
    """
    profitable_net = 0
    unprofitable_net = 0

    for cohort in metrics:
        seg = cohort.get("segmentId")
        long_n = cohort.get("longNotional", 0)
        short_n = cohort.get("shortNotional", 0)
        net = long_n - short_n  # positive = net long

        if seg in PROFITABLE:
            profitable_net += net
        elif seg in UNPROFITABLE:
            unprofitable_net += net

    # Determine alignment
    if signal_direction == "long":
        aligned = profitable_net > 0 and unprofitable_net < profitable_net
        opposed = profitable_net < 0
    else:
        aligned = profitable_net < 0 and unprofitable_net > profitable_net
        opposed = profitable_net > 0

    if aligned:
        return "pass"
    elif opposed:
        return "skip"
    else:
        return "reduce"

In a live backtest loop, you call cohort_conviction() every time your signal fires and use the result to gate execution. Trades that get "pass" execute at full size. Trades that get "reduce" execute at a fraction (you pick the fraction). Trades that get "skip" never execute.

Building the Collection Pipeline

Since cohort metrics have a 4-week lookback window, any serious backtesting effort requires continuous data collection. The pipeline is straightforward: a scheduled script calls the API, parses the response, and writes it to persistent storage.

import requests
import json
import sqlite3
from datetime import datetime

BASE_URL = "https://ht-api.coinmarketman.com/api/external"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}
COINS = ["BTC", "ETH", "SOL", "HYPE"]

def collect_cohort_snapshot(db_path="cohort_history.db"):
    conn = sqlite3.connect(db_path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS cohort_snapshots (
            timestamp TEXT, coin TEXT, segment_id INTEGER,
            long_notional REAL, short_notional REAL,
            total_notional REAL, raw_json TEXT
        )
    """)

    ts = datetime.utcnow().isoformat() + "Z"

    for coin in COINS:
        resp = requests.get(
            f"{BASE_URL}/cohort/metrics",
            headers=HEADERS,
            params={"coin": coin}
        )
        if resp.status_code != 200:
            continue

        for cohort in resp.json():
            conn.execute(
                "INSERT INTO cohort_snapshots VALUES (?,?,?,?,?,?,?)",
                (ts, coin, cohort.get("segmentId"),
                 cohort.get("longNotional", 0),
                 cohort.get("shortNotional", 0),
                 cohort.get("totalNotional", 0),
                 json.dumps(cohort))
            )

    conn.commit()
    conn.close()

Run this every 5 minutes via cron or systemd timer. At 4 coins per call, you're making 4 API requests per cycle, which is 1,152 calls per day. The free tier's 100 calls/day won't cover continuous collection, but the Pulse tier at $179/mo gives you 50,000 calls per month, enough for 24/7 collection across multiple assets with room to spare.

After a few weeks of collection, you'll have enough historical cohort data to start running backtests that actually account for who was on each side of every move.

Three Questions Your Backtest Should Answer

Once you have cohort data layered into your backtest, you can ask questions that price-only backtests cannot:

1. How does win rate change when smart money aligns?

Run your base strategy twice. Once without the cohort filter, once with it. Compare the win rates. If the filtered version has a meaningfully higher win rate but fewer total trades, the cohort filter is doing its job: removing low-conviction setups that drag down average performance.

The tradeoff is always the same. Filters reduce trade count. Fewer trades means slower compounding but also fewer drawdowns from low-quality setups. If the filtered Sharpe ratio improves even as the total P&L stays flat, the strategy is more capital-efficient and easier to size up.

2. Which cohort alignment matters most?

Not all cohorts carry equal predictive weight. Test each cohort independently as a filter. Does alignment with the Money Printer cohort alone improve results? What about the Leviathan cohort (wallets with $5M+ in perp equity, segment ID 7)? Size-based cohorts and PnL-based cohorts measure different things, and the one that matters most depends on the asset and timeframe you're trading.

For high-cap assets like BTC and ETH, PnL-based cohorts tend to be more informative because the market is deep enough that size alone doesn't predict directionality. For mid-cap and long-tail assets, size-based cohorts can matter more because a single Whale or Leviathan wallet entering a position represents a larger share of total open interest.

3. Does cohort divergence predict drawdowns?

One of the highest-value uses of cohort data in backtesting is drawdown analysis. Go through your backtest's worst losing streaks and check what the cohort composition looked like at entry. If the worst drawdowns consistently coincide with moments where your signal was aligned with unprofitable cohorts and opposed by profitable ones, you've found a structural weakness in the strategy that the standard backtest would never reveal.

This is the real payoff of cohort-aware backtesting. You're not just improving average performance; you're identifying the specific conditions under which your strategy is most vulnerable. That's worth more than a few extra basis points of average return.

Common Pitfalls When Backtesting with Cohort Data

Cohort data adds a powerful lens, but it introduces its own failure modes.

Overfitting to a specific cohort. If you optimize your strategy to follow only the Money Printer cohort and it backtest beautifully, ask yourself: are you trading a real signal or just fitting to a small sample of wallets that happened to be right during your test period? The Money Printer cohort represents a narrow slice of the market. Any strategy that depends entirely on one segment's positioning is fragile.

Ignoring cohort regime changes. Wallets move between cohorts over time. A wallet that was Smart Money in January might be Semi-Rekt by June after a string of losses. HyperTracker reclassifies wallets based on all-time PnL, so the cohort labels are dynamic. A backtest that assumes static cohort membership will overestimate the signal quality, because some of the "smart money" positioning it's following was actually generated by wallets that were smart then but aren't anymore.

Latency assumptions. Our data refreshes every 5 minutes, with 15 to 20 minutes for full state refreshes. In a backtest, you have perfect data at every timestamp. In live trading, you're always looking at data that's a few minutes old. Build that lag into your backtest by using cohort data from the previous snapshot rather than the current one. If the signal only works with current-bar cohort data, it doesn't work in production.

Confusing correlation with causation. Smart money positioning doesn't cause price to move in a particular direction. It's a coincident indicator: profitable wallets tend to be positioned correctly more often than unprofitable ones. The relationship is probabilistic rather than deterministic. Treat cohort alignment as a filter that improves expected value over many trades, not a guarantee on any single trade.

Start Collecting Cohort Data for Your Backtests

The free tier gives you 100 API calls per day. Enough to prototype your collection pipeline and validate the signal quality before scaling up.

Get Your Free API Key

Every backtest is a simplified model of reality. The question is which simplifications cost you money. Treating all capital as equal, ignoring who's behind the flow, and assuming that OI is OI regardless of source: these are simplifications that look harmless until you ship the strategy live and watch it underperform the simulated curve. Cohort data doesn't make your backtest perfect. It makes the lies smaller, which is enough to change the outcome.