Barclays Live discovery
Expand
Expand the personalized research entry point.
High-intent users reached relevant reports faster, with entitlement and quick-bounce guardrails stable.
Redacted Representative Artifact
Weekly CRO Experiment Readout
A portfolio sample showing how I turn product experiments into clear decisions, quality guardrails, and reusable learnings.
This is a decision artifact, not a status update.
The readout compresses the week into three calls: what to expand, what to keep learning from, and what to rework before rollout.
Ask Mara answer trust
Continue
Keep the citation-first answer test running.
Citation opens improved, but feedback volume and review outcomes are not clean enough yet.
Generic onboarding funnel
Rework
Rework the shorter checklist before rollout.
Completion moved up, but the test hid one education moment users needed before their first key action.
What I would instrument first.
Research engagement quality
qualified_research_action / active_user
Up
AI answer trust
citation_opened + answer_saved + positive_feedback
Mixed
Quality guardrail
irrelevant_click + negative_feedback + support_ticket
Flat
ReadUse the KPI to understand behavior, not to decorate the update.
DecideEvery experiment gets a call: expand, continue, rework, or stop.
ReuseEach learning enters a findings library so the next PM is not starting cold.
The event model keeps the read honest.
The taxonomy separates usage from quality. That matters in research and AI products, where a click can mean curiosity, confusion, or trust.
research_home_viewed
Top-of-funnel denominator with client segment, region, device, and entitlement tier.
research_search_submitted
Search discovery read with query type, asset class, ticker count, and experiment variant.
report_opened
Research relevance and content-quality signal by source module, analyst team, and topic.
ask_mara_query_submitted
AI assistant adoption and answer-quality analysis by query intent and answer type.
citation_opened
Trust behavior and citation usefulness by answer, source rank, report age, and variant.
Three experiments, three different calls.
A good readout does not force every test into a victory story. It shows judgment under uncertainty.
Experiment 1
Barclays Live Research Discovery Modules
If returning users see role- and topic-aware research modules earlier, they should reach relevant reports faster and take more qualified research actions.
VariantReplaced a static landing section with saved-topic, followed-analyst, and entitlement-aware modules.
MetricQualified research actions per active user, with report opens and saved reports as secondary signals.
GuardrailNo increase in irrelevant clicks, entitlement errors, or quick bounces.
DecisionExpand to a larger cohort after segmenting by asset class, region, and entitlement tier.
Experiment 2
Barclays Live Ask Mara Citation-First Answers
If AI-generated research answers make citations and source freshness more prominent, users should trust the answer enough to continue working.
VariantMoved citations above the fold, exposed source confidence, and shortened the path to the full report.
MetricCitation-open rate after an Ask Mara answer, with follow-up questions and answer saves as secondary signals.
GuardrailNo increase in negative feedback, unsupported-answer flags, or low-confidence answer completion.
DecisionContinue and hold rollout until feedback volume and answer-review outcomes are cleaner.
Experiment 3
Generic Onboarding Checklist Compression
If a multi-step checklist is compressed into fewer visible tasks, more users should finish setup without sacrificing feature discovery.
VariantCombined two low-value checklist steps and moved optional education into a later tooltip.
MetricOnboarding completion rate, time to complete, first key action, and week-one return rate.
GuardrailNo decline in first meaningful feature use after setup.
DecisionRework before rollout by restoring one contextual education moment before the first key action.
The backlog changes because the read changed.
Validate that Barclays Live relevance lift holds across asset class and region.
Separate answer engagement from genuine trust and citation usefulness.
Help users personalize research discovery without forcing setup too early.
Repair feature-discovery softness from the generic onboarding compression test.
Findings worth carrying forward.
Personalization works when relevance is explainable.
Expand personalization, but keep entitlement and relevance checks visible in analysis.
Answer engagement is not the same as trust.
Treat AI answer changes as quality experiments, not just engagement experiments.
Shorter flows can hide the product's value path.
Pair simplification with one well-placed education moment tied to the user's next action.