Comparing an Indian fund with a dollar index, one date at a time

I wanted to know whether a large- and mid-cap Indian mutual fund had actually beaten its market. The obvious hurdle is the S&P India LargeMidCap Total Return index. The catch is that the public page reports returns in US dollars, while a fund’s NAV is in rupees, and the two sides publish on different days. CompareMF is a small FastAPI and React app that makes those two series comparable and says so when it can’t.

The problem

A fair comparison needs three things to line up. The periods have to cover the same dates. The benchmark has to be in the same currency as the fund. And every input has to be traceable, because a number from a scrape that failed last week looks exactly like a fresh one.

None of the sources help with this on their own. AMFI publishes today’s NAV for every scheme. MFapi.in has history. S&P shows 1, 3, 5 and 10-year annualised USD returns for a single “as of” date. Frankfurter serves historical USD/INR reference rates. The app’s job is mostly bookkeeping between them.

Architecture

There are two processes built from one Docker image, sharing a SQLite file on a named volume.

The scheduler container runs the daily ingestion for AMFI and S&P. The web container runs FastAPI, serves the built React app, and computes comparisons on request. SQLite holds five tables: schemes, nav_observations, benchmark_observations, fx_observations and ingestion_runs. It runs in WAL mode with a busy timeout, which is enough for one writer and a few readers on a single host.

Anchor everything to the benchmark date

The key decision is that the S&P as_of date drives every other lookup. The benchmark only exists for one date, so the fund has to come to it, not the other way round.

The end NAV is the latest observation on or before as_of. For each period the service computes the anniversary and takes the latest NAV on or before it. If that NAV is more than ten days early, or missing, the period is reported as insufficient_history rather than stretched to fit. From app/comparison.py:

target = years_before(as_of, years)
start_nav = self.repository.nav_on_or_before(scheme_code, target.isoformat())
if start_nav is None or date.fromisoformat(start_nav["nav_date"]) < target - timedelta(days=10):
    periods.append({"years": years, "status": "insufficient_history"})
    continue

years_before clamps the day to the target month’s length, so 29 February maps to 28 February in a non-leap year instead of raising.

If the fund has a NAV newer than as_of, it is not used. The response includes a warning naming both dates, and an is_date_aligned flag says whether the end NAV fell exactly on the benchmark date.

Converting the benchmark to rupees

A USD return is not a rupee return. I undo the annualisation, apply the change in the INR-per-USD rate over the period, and annualise again:

def inr_adjusted_return(usd_annualized: Decimal, fx_start: Decimal, fx_end: Decimal, years: int) -> Decimal:
    if fx_start <= 0 or fx_end <= 0:
        raise ValueError("FX rates must be positive")
    cumulative = (1 + usd_annualized) ** years * (fx_end / fx_start)
    return Decimal(str(float(cumulative) ** (1 / years) - 1))

The fund side is the NAV ratio annualised over the same number of years. The difference between the two is what the UI shows as alpha.

FX rates are cached by requested date. The rate Frankfurter returns can be dated differently from the day asked for, so the table stores both requested_date and effective_date, and the API returns the effective one next to each rate.

Ingestion that runs once a day

The daily command has to be safe to call twice. Before any network request, it claims the pair (source, calendar date) with an insert that does nothing on conflict. From app/repository.py:

cursor = connection.execute(
    """INSERT INTO ingestion_runs(source, run_date, status, started_at)
    VALUES (?, ?, 'running', ?) ON CONFLICT(source, run_date) DO NOTHING""",
    (source, run_date, started_at),
)
return cursor.rowcount == 1

If the row already exists, the run is skipped, even if the earlier attempt failed. That stops a retry loop from hammering a source that is blocking us. --force bumps attempt_count and runs anyway. Each run finishes as succeeded or failed with a detail string. AMFI and S&P are claimed separately, so one failing doesn’t stop the other, and the command exits nonzero if either fails.

Benchmark rows are keyed on as_of and inserted with ON CONFLICT DO NOTHING. S&P shows Friday’s date over the weekend, and repeated scrapes of the same day don’t create duplicates.

The scheduler is a loop that polls at most every 60 seconds. It runs the jobs once the configured local time has passed and it hasn’t run that day yet, so a container that restarts after the scheduled time catches up straight away. I kept it out of FastAPI on purpose. Inside the API it would run once per worker.

When the S&P scrape fails

The scraper uses headless Chromium through Playwright, and S&P blocks it from my environment. Before it reads any numbers, the scraper checks that the heading is “S&P India LargeMidCap” and that the page says “USD Total Return”, so a changed page layout fails loudly instead of returning the wrong index.

To keep the app usable, it ships one manually supplied observation for 2026-06-19. It is stored with origin = 'fallback'. Every benchmark row carries an origin of live, fallback or unknown, enforced by a CHECK constraint. Rows that existed before the column was added are migrated to unknown, and the fallback values are recognised and relabelled. The comparison always uses the newest as_of, so a successful live scrape replaces the fallback without anyone touching it.

Warnings travel with the result:

• a fallback or unknown benchmark gets a warning.
• a benchmark more than three days old, counted in India’s calendar, gets a warning.
• an IDCW or dividend plan gets a warning, because NAV returns leave out cash paid out.
• a scheme outside the large- and mid-cap category gets a warning, because this index is the wrong hurdle for it.

What I learned

The first commit computed returns and showed them. The second one, titled “make benchmark data trustworthy”, added provenance, date alignment and freshness warnings. The third fixed a null dereference when a scheme had no latest NAV. The calculation was the easy part. Most of the code is there to record where each number came from and when.

Storing money values as TEXT and reading them into Decimal kept the database free of float rounding. The exponent still goes through float, because Decimal doesn’t do fractional powers cleanly, and I accept that at the last step.

A scraper you don’t control is not a data feed. The fallback keeps the app working and labelled honestly, but it doesn’t make the data fresh.

What’s next

• A sanctioned source for the S&P numbers. My uncommitted branch adds a fail-fast check for S&P’s block page and a local command that reads the page from a Chrome window I open myself. That helps me recover by hand. It is not a production answer.
• 10-year comparisons. The 10-year field is already stored, but the service only compares 1, 3 and 5 years.
• Total-return fund data, so IDCW plans can be compared without a caveat.

Past performance is not investment advice.