We run a small fleet of reporting bots: scripts on GitHub Actions that pull from the ad platforms each morning, render a card as an image, and post it to Telegram. No dashboards to log into, no CSVs. The team reads them with coffee. One of them is a weekly conversion digest, and one Monday it was wrong by three quarters.

It reported 47 conversions and about 2,600 dollars for the week. The week actually had 200 conversions and about 22,000 dollars. Nothing errored. The report looked exactly like every other week, only smaller, and a smaller week is not suspicious on its own.

What the API did

The stats endpoint returns at most 50,000 rows per page, and it does not tell you when it has hit that limit. You get 50,000 rows of real data and no flag. Our seven-day pull, at the full width of dimensions we were using, produced exactly 50,000 rows, and they covered days one and two. The other five days were simply not there.

The cause was dimension cardinality. We were asking for date, campaign, banner, page, country and media. Country and media multiply the rows by roughly ten. The same seven days at date, campaign, banner and page came back as 14,861 rows, all seven days, well under the cap. We were paying the row cost for two columns nobody read.

Rules the pipeline follows now

  • Only request dimensions you actually use. Every column you add multiplies rows.
  • Page every pull that covers more than a day. Loop with an offset until a page comes back short. Never trust a single page.
  • Treat a page of exactly 50,000 rows as a failure, not a result. The script now aborts and alerts if it sees one.
  • Reconcile totals against a second, low-cardinality pull: date only. If the two totals disagree by more than a few percent, alert instead of publishing.
  • Rate limits are real: ten stats calls a minute. The script waits between calls rather than retrying into a lockout.

Why the bots at all

Because reports that require someone to go and look do not get looked at. A card that arrives at seven in the morning, in the channel people already read, gets read. The fleet shares one renderer, so every report looks the same: dark card, tabular figures, the same accent colour, generated as an image by headless Chromium and uploaded as a photo. Adding a new report is mostly writing the query.

The 50,000-row rule is now the first line in the memory file every one of our agents reads before it pulls stats from that platform. It cannot be forgotten, because nobody has to remember it. The under-report was caught because a person noticed the number felt low and checked the platform by hand. That is the honest version. The fix is that the pipeline now checks itself, so the next silent truncation gets caught by the script and not by luck.