Vetting creators without letting the model make things up.

Before a brand pays a creator, someone has to find out who this person is and whether there is anything in public view that conflicts with the campaign? Done by hand, that means an afternoon of searching. Done badly by a language model, it means a confident paragraph about a namesake, citing a URL nobody fetched. I built Influencer Vetting to do the searching while keeping the model away from two jobs it does badly. It guesses which person a name means, and it invents sources.

The live demo takes a brand, a campaign summary, and one to eight creator names, handles, or profile URLs. The source is a Python 3.13 FastAPI service on Google ADK with a Vite and TypeScript client.

The shape of the pipeline

The root agent is a custom ADK BaseAgent subclass, InfluencerVettingAgent in src/agent/vetting/agents.py. It owns six LlmAgent specialists and calls them in a fixed order that I wrote out by hand instead of letting a planner choose:

1. batch_intake extracts the brand, campaign, and creator list.
2. identity_researcher resolves who the current creator is.
3. vetting_director writes the search queries.
4. reputation_researcher runs the searches and extracts evidence.
5. brand_policy_analyst maps evidence to the campaign.
6. evidence_auditor checks support and wording without searching again.

Steps 2 to 6 run once per creator, sequentially. Each specialist writes a Pydantic-typed result to session state through output_key, and the next one reads it through template variables like {identity_result}.

Identity comes before evidence

The most common failure in this domain is vetting the wrong person. So the loop stops for a creator whose identity did not resolve:

identity = _state_object(ctx.session.state.get("identity_result"))
if identity.get("confidence") == "low" or not identity.get("selected"):
    reports.append(_deterministic_report(ctx.session.state))
    yield self._activity_event(ctx, self.name, "identity_unresolved", ...)
    continue

No research runs and no claims are evaluated for that creator. The report comes back FLAGGED with low confidence and a condition to verify the official profile before contracting.

The schema backs this up. Early on, IdentityResult raised a validation error when the model returned medium confidence with no selected identity. That failed the whole batch over one ambiguous name. The fix in commit 718fcb5 made the validator downgrade instead of throw:

@model_validator(mode="after")
def normalize_confidence_without_selected_identity(self) -> IdentityResult:
    if self.confidence != Confidence.LOW and self.selected is None:
        self.confidence = Confidence.LOW
        if not self.blocker:
            self.blocker = "No creator-specific identity could be selected"
    return self

The identity search tool, exa_people_search, uses Exa’s people category for plain names. If the input contains @ or ://, it keeps the locator and appends official-profile context instead, because the people category is the wrong tool for a handle. The tool counts calls in session state and refuses a third one.

Every source gets an ID, and only IDs count

The reputation step runs one bundled tool, comprehensive_reputation_search in src/agent/vetting/research_tools.py. It fires four searches in parallel with asyncio.gather(..., return_exceptions=True): Linkup for broad reputation, Exa news for corroboration, Exa for audience metrics on the selected profile, and Linkup for reporting on fake followers or bought engagement. A failed provider becomes a string in research_provider_failures rather than an exception, so one outage produces a partial result, not a crash.

Every URL the providers return is canonicalised and hashed into a stable ID. Canonicalising lowercases the host, drops www. and the trailing slash, and strips tracking params like utm_* and fbclid.

def _source_id(url: str) -> str:
    digest = hashlib.sha256(canonicalize_url(url).encode()).hexdigest()[:12]
    return f"src_{digest}"

The tool then opens the top five discovered URLs per purpose through Exa’s contents endpoint and marks them content_fetched. Reputation and audience sources are stored in separate catalogs so a follower-count page cannot turn into a reputation claim.

After the model extracts evidence, Python checks it against the catalog. _normalize_reputation_result looks each item up by source_id, falls back to canonical URL, and drops anything that matches neither. _normalize_policy_result does the same for policy findings: a finding with no matching registered evidence is removed.

The verdict is not a model output

The model produces findings. It does not produce the verdict. _deterministic_report reads identity, policy, and audit state and checks rules in order. First an unresolved identity, then a high-severity hard block, then manageable or medium findings, then any other source-backed finding. Any of those gives FLAGGED. Otherwise it is APPROVE, and confidence drops from high to medium if a provider failed, coverage was empty, or the audit did not pass.

I made two rules stricter than the prompt.

• Audience-integrity findings count toward the verdict only when audience_evidence_strength is corroborated, meaning an official or creator-statement source confirms it, or the supporting sources come from at least two domains. One article about fake followers gets shown to the reviewer, but it does not flip the verdict on its own.
• Audience findings are dropped entirely unless the evidence was fetched and there is a single validated target profile.

Every report lists the same three limitations. One says this is due diligence, not a legal background check, and that public metrics cannot definitively detect bots.

Streaming progress without leaking internals

The client posts to ADK’s /run_sse endpoint. I didn’t want the browser to render raw tool arguments or evidence payloads mid-run, so the agent emits a separate ui_activity state delta at each stage with only the agent name, stage, creator name, and index. The final event carries a sanitised ui_report, which is also what the history view loads from the ADK session later.

What I learned

• Hand-written orchestration was the right call here. With a fixed order I can reason about what state exists at each step, and the per-creator reset is one dictionary.
• Per-tool call budgets in session state are cheap and effective. The identity tool refuses a third call, and the research tool refuses a second, whatever the prompt says.
• Validators that throw on model output are brittle in a batch. Normalising to the safe value and recording why worked better.
• The client’s anonymous web- user ID lives in local storage. The frontend README says that is fine for a local prototype, and that production must derive the user ID from server-side authentication.

What’s next

Authentication comes first, so review history belongs to a user and not a browser. Then I want the auditor’s checks, which live in a prompt today, to become deterministic rules like the verdict already is. The repository also removed its Python CI gates in commit e1f9649, and those should come back before the code grows further.