Scoping an AOI before a broad area similarity sweep

You've got one site. You know what it looks like, down to the roofline and the yard layout. What's missing is the boundary: how far the search should run before you hit the button.
Draw the AOI too tight and you miss the facility that's two valleys over, outside your assumed radius. Draw it too loose and you're back to the problem the sweep was supposed to solve: a review queue too long to get through before the brief is due.
Start from the siting logic, not a radius
The instinct is to buffer the known site by some round number, 50 km, 100 km, and call that the AOI. That works if the thing you're looking for is sited randomly. It almost never is. Whatever put that site where it is, a supply route, a water source, a particular terrain type, a border distance, a grid connection, will have put the others there too.
So before you touch the drawing tool, write down what you actually know about why the site sits where it does. A processing facility near a river crossing. A compound set back from a paved road but within reach of one. A pad built on flat ground above a certain elevation because the surrounding terrain doesn't allow it below that line. Each of those constraints either expands or shrinks the honest search extent, and they rarely produce a circle.
A terrain-constrained site might warrant a narrow AOI that follows a valley floor for 300 km and ignores the mountains on either side entirely. A site tied to a transport network might warrant a buffer around road and rail corridors instead of a blanket polygon. If you can state the constraint, you can usually shape the boundary to match it, and a shaped AOI returns a cleaner set of candidates than a bounding box ever will.
Check what the sweep will actually be comparing
An AOI sets the geographic extent of your search, and it also sets the background the sweep will score your query tile against. A query tile pulled from flat agricultural land, searched against an AOI that's mostly dense urban fabric, is going to surface a lot of visually odd matches that have nothing to do with the site type you care about. If the terrain character inside your boundary doesn't resemble the terrain the known site sits in, expect false positives. The method is doing what it was asked: comparing things that were never going to look alike.
Sanity-check this before the sweep runs. Pull up the known site and scan the terrain and land cover across your proposed boundary. If half the AOI is open water or dense forest canopy that the site type couldn't be sited on anyway, cut it. That saves compute, and it saves the review pass at the other end, where every extra square kilometer of irrelevant background becomes tiles a person has to look at and rule out by eye.
Run it narrow first, then widen
The first pass tests your reasoning about siting logic. Run the sweep against a boundary built from your best read on where the site type can plausibly sit, see what comes back, and look at where the near-misses fall. If matches cluster near the edge of your AOI, that's a signal the boundary was drawn too tight and the true extent runs further than you assumed. Widen along that edge and run again, rather than redrawing the whole thing from scratch.
It's worth knowing what a similarity sweep against high-resolution imagery can and can't tell you before you commit a day to building the perfect boundary. If you're scoping this kind of search for the first time, Broad Area Search is built around exactly this workflow: mark the known site, set the extent, and get back the tiles across it that resemble it.
Get the AOI right and the rest of the sweep does the work you built it to do.