Reading competitor signals without fooling yourself

Most small SaaS teams already watch competitors. Someone checks the pricing page on Monday, skims the changelog, notices a new "Senior Backend Engineer" listing, and posts a screenshot in Slack. The habit is fine. The problem comes later, when a number in your own dashboard moves and the screenshot becomes the explanation.

This post covers three practices that keep the habit honest: record changes with their evidence, treat any match with your own metrics as a correlation, and say how confident you are before you act.

1. Keep a record, not a memory

A remembered observation ("they changed pricing sometime in spring") can't be checked. A useful record has four fields for every entry:

Two details matter. First, the timestamp is when you observed the change, not when it happened. A page you check weekly can change any time in the seven days before your visit, so your record has a window, not a moment. Write the window down. Second, keep the evidence itself, not only your summary of it. Pages get edited back, listings get taken down, and a month later the link alone proves nothing.

Also record what you did not see. A source you could not load on a given day is a gap, not a "no change".

2. A match is a correlation

Suppose Company A cuts a plan price on March 3, and your trial signups dip in the following two weeks. It is tempting to write "their price cut hurt our signups". The record does not support that sentence. It supports a narrower one: the price change lined up with a dip in our signups.

Many other things could produce the same picture:

• Your own changes. A release, a pricing edit, or a paused ad campaign in the same window.
• Seasonality. Signups often dip around holidays and quarter ends regardless of competitors.
• Shared causes. Both companies may react to the same market event, so their moves line up without either driving the other.
• Chance. With many competitors, many sources and many metrics, some pairs will line up by luck. The more comparisons you scan, the more likely a striking one is noise.
• Timing choices. If you pick the lag after seeing the data, you can usually find one that fits.

A useful habit is to write down the alternatives before you look for supporting evidence, and to check one of them directly: did anything change on your side in that window?

3. Label confidence, and mean it

A confidence label is a statement about how much the evidence supports a claim, written so someone else can disagree with it. A simple three-level scheme works for a small team:

• Low: one competitor change and one metric move, a single occurrence, plausible alternatives not ruled out.
• Medium: the pattern has repeated, for example the same kind of competitor change followed by a similar move in your metric more than once, and you have checked for your own changes and seasonality.
• High: reserve this for results from a controlled experiment of your own. Observation alone rarely earns it.

These labels are judgments, not measured probabilities. Do not attach percentages to them unless you have a method that produces real ones. Next to the label, write what would change your mind.

4. Turn the pattern into an experiment

The way out of correlation is to run something where you control the assignment. If you suspect a rival's lower entry price is pulling visitors away from your pricing page, you can test your own response instead of guessing:

1. State the hypothesis and the single metric, such as pricing-page-to-trial conversion.
2. Randomly assign visitors to control or variant.
3. Decide the sample size, duration and stopping rule before you start, and do not stop early because the numbers look good.
4. Measure the difference with an interval, not only a point estimate. A small test often produces an interval wide enough to include "no effect", and that is a legitimate result.

The test tells you about your change and your visitors. It does not tell you what the competitor's change did, and that is fine. Your decision is about your own page.

Where Ephor fits

Ephor is a tool for this workflow. You configure public sources for the companies you follow (pages, pricing, feeds, releases, job boards). It checks them at the collection interval you set for each source (your plan sets the shortest interval allowed), keeps the page text and a source-linked history of detected changes, and lets you import your own metrics. It compares those series, evaluates association models against held-out data, and records the randomized experiments you run. Its error bands are descriptive, not calibrated confidence intervals, and it does not claim that any competitor action caused an outcome. Coverage depends on which sources you configure and what they publish; JavaScript-only page content is not rendered, and public activity does not reveal private sales.

You can start the 14-day no-card trial, or see the plans. A spreadsheet with the four columns above is also a perfectly good place to start.