Midtrans-Middleware/.bmad/bmm/workflows/1-analysis/research/checklist.md

Market Research Report Validation Checklist

🚨 CRITICAL: Source Verification and Fact-Checking (PRIORITY)

Source Citation Completeness

• EVERY market size claim has at least 2 cited sources with URLs
• EVERY growth rate/CAGR has cited sources with URLs
• EVERY competitive data point (pricing, features, funding) has sources with URLs
• EVERY customer statistic or insight has cited sources
• EVERY industry trend claim has sources from {{current_year}} or recent years
• All sources include: Name, Date, URL (clickable links)
• No claims exist without verifiable sources

Source Quality and Credibility

• Market size sources are HIGH credibility (Gartner, Forrester, IDC, government data, industry associations)
• NOT relying on single blog posts or unverified sources for critical data
• Sources are recent ({{current_year}} or within 1-2 years for time-sensitive data)
• Primary sources prioritized over secondary/tertiary sources
• Paywalled reports are cited with proper attribution (e.g., "Gartner Market Report 2025")

Multi-Source Verification (Critical Claims)

• TAM calculation verified by at least 2 independent sources
• SAM calculation methodology is transparent and sourced
• SOM estimates are conservative and based on comparable benchmarks
• Market growth rates corroborated by multiple analyst reports
• Competitive market share data verified across sources

Conflicting Data Resolution

• Where sources conflict, ALL conflicting estimates are presented
• Variance between sources is explained (methodology, scope differences)
• No arbitrary selection of "convenient" numbers without noting alternatives
• Conflicting data is flagged with confidence levels
• User is made aware of uncertainty in conflicting claims

Confidence Level Marking

• Every major claim is marked with confidence level:
  • [Verified - 2+ sources] = High confidence, multiple independent sources agree
  • [Single source - verify] = Medium confidence, only one source found
  • [Estimated - low confidence] = Low confidence, calculated/projected without strong sources
• Low confidence claims are clearly flagged for user to verify independently
• Speculative/projected data is labeled as PROJECTION or FORECAST, not presented as fact

Fact vs Analysis vs Speculation

• Clear distinction between:
  • FACT: Sourced data with citations (e.g., "Market is $5.2B [Source: Gartner 2025]")
  • ANALYSIS: Interpretation of facts (e.g., "This suggests strong growth momentum")
  • SPECULATION: Educated guesses (e.g., "This trend may continue if...")
• Analysis and speculation are NOT presented as verified facts
• Recommendations are based on sourced facts, not unsupported assumptions

Anti-Hallucination Verification

• No invented statistics or "made up" market sizes
• All percentages, dollar amounts, and growth rates are traceable to sources
• If data couldn't be found, report explicitly states "No verified data available for "
• No use of vague sources like "industry experts say" without naming the expert/source
• Version numbers, dates, and specific figures match source material exactly

Market Sizing Analysis (Source-Verified)

TAM Calculation Sources

• TAM figure has at least 2 independent source citations
• Calculation methodology is sourced (not invented)
• Industry benchmarks used for sanity-check are cited
• Growth rate assumptions are backed by sourced projections
• Any adjustments or filters applied are justified and documented

SAM and SOM Source Verification

• SAM constraints are based on sourced data (addressable market scope)
• SOM competitive assumptions cite actual competitor data
• Market share benchmarks reference comparable companies with sources
• Scenarios (conservative/realistic/optimistic) are justified with sourced reasoning

Competitive Analysis (Source-Verified)

Competitor Data Source Verification

• EVERY competitor mentioned has source for basic company info
• Competitor pricing data has sources (website URLs, pricing pages, reviews)
• Funding amounts cite sources (Crunchbase, press releases, SEC filings)
• Product features verified through sources (official website, documentation, reviews)
• Market positioning claims are backed by sources (analyst reports, company statements)
• Customer count/user numbers cite sources (company announcements, verified reports)
• Recent news and developments cite article URLs with dates from {{current_year}}

Competitive Data Credibility

• Company websites/official sources used for product info (highest credibility)
• Financial data from Crunchbase, PitchBook, or SEC filings (not rumors)
• Review sites cited for customer sentiment (G2, Capterra, TrustPilot with URLs)
• Pricing verified from official pricing pages (with URL and date checked)
• No assumptions about competitors without sourced evidence

Competitive Claims Verification

• Market share claims cite analyst reports or verified data
• "Leading" or "dominant" claims backed by sourced market data
• Competitor weaknesses cited from reviews, articles, or public statements (not speculation)
• Product comparison claims verified (feature lists from official sources)

Customer Intelligence (Source-Verified)

Customer Data Sources

• Customer segment data cites research sources (reports, surveys, studies)
• Demographics/firmographics backed by census data, industry reports, or studies
• Pain points sourced from customer research, reviews, surveys (not assumed)
• Willingness to pay backed by pricing studies, surveys, or comparable market data
• Buying behavior sourced from research studies or industry data
• Jobs-to-be-Done insights cite customer research or validated frameworks

Customer Insight Credibility

• Primary research (if conducted) documents sample size and methodology
• Secondary research cites the original study/report with full attribution
• Customer quotes or testimonials cite the source (interview, review site, case study)
• Persona data based on real research findings (not fictional archetypes)
• No invented customer statistics or behaviors without source backing

Positioning Analysis

• Market positioning map uses relevant dimensions for the industry
• White space opportunities are clearly identified
• Differentiation strategy is supported by competitive gaps
• Switching costs and barriers are quantified
• Network effects and moats are assessed

Industry Analysis

Porter's Five Forces

• Each force has a clear rating (Low/Medium/High) with justification
• Specific examples and evidence support each assessment
• Industry-specific factors are considered (not generic template)
• Implications for strategy are drawn from each force
• Overall industry attractiveness conclusion is provided

Trends and Dynamics

• At least 5 major trends are identified with evidence
• Technology disruptions are assessed for probability and timeline
• Regulatory changes and their impacts are documented
• Social/cultural shifts relevant to adoption are included
• Market maturity stage is identified with supporting indicators

Strategic Recommendations

Go-to-Market Strategy

• Target segment prioritization has clear rationale
• Positioning statement is specific and differentiated
• Channel strategy aligns with customer buying behavior
• Partnership opportunities are identified with specific targets
• Pricing strategy is justified by willingness-to-pay analysis

Opportunity Assessment

• Each opportunity is sized quantitatively
• Resource requirements are estimated (time, money, people)
• Success criteria are measurable and time-bound
• Dependencies and prerequisites are identified
• Quick wins vs. long-term plays are distinguished

Risk Analysis

• All major risk categories are covered (market, competitive, execution, regulatory)
• Each risk has probability and impact assessment
• Mitigation strategies are specific and actionable
• Early warning indicators are defined
• Contingency plans are outlined for high-impact risks

References and Source Documentation (CRITICAL)

References Section Completeness

• Report includes comprehensive "References and Sources" section
• Sources organized by category (market size, competitive, customer, trends)
• Every source includes: Title/Name, Publisher, Date, Full URL
• URLs are clickable and functional (not broken links)
• Sources are numbered or organized for easy reference
• Inline citations throughout report reference the sources section

Source Quality Metrics

• Report documents total sources cited count
• High confidence claims (2+ sources) count is reported
• Single source claims are identified and counted
• Low confidence/speculative claims are flagged
• Web searches conducted count is included (for transparency)

Source Audit Trail

• For each major section, sources are listed
• TAM/SAM/SOM calculations show source for each number
• Competitive data shows source for each competitor profile
• Customer insights show research sources
• Industry trends show article/report sources with dates

Citation Format Standards

• Inline citations format: [Source: Company/Publication, Year, URL] or similar
• Consistent citation style throughout document
• No vague citations like "according to sources" without specifics
• URLs are complete (not truncated)
• Accessed/verified dates included for web sources

Document Quality

Anti-Hallucination Final Check

• Read through entire report - does anything "feel" invented or too convenient?
• Spot-check 5-10 random claims - can you find the cited source?
• Check suspicious round numbers - are they actually from sources?
• Verify any "shocking" statistics have strong sources
• Cross-check key market size claims against multiple cited sources

Structure and Completeness

• Executive summary captures all key insights
• No placeholder text remains (all {{variables}} are replaced)
• References section is complete and properly formatted
• Source quality assessment included
• Document ready for fact-checking by third party

Research Completeness

Coverage Check

• All workflow steps were completed (none skipped without justification)
• Optional analyses were considered and included where valuable
• Web research was conducted for current market intelligence
• Financial projections align with market size analysis
• Implementation roadmap provides clear next steps

Validation

• Key findings are triangulated across multiple sources
• Surprising insights are double-checked for accuracy
• Calculations are verified for mathematical accuracy
• Conclusions logically follow from the analysis
• Recommendations are actionable and specific

Final Quality Assurance

Ready for Decision-Making

• Research answers all initial objectives
• Sufficient detail for investment decisions
• Clear go/no-go recommendation provided
• Success metrics are defined
• Follow-up research needs are identified

Document Meta

• Research date is current
• Confidence levels are indicated for key assertions
• Next review date is set
• Distribution list is appropriate
• Confidentiality classification is marked

---

Issues Found

Critical Issues

List any critical gaps or errors that must be addressed:

• Issue 1: [Description]
• Issue 2: [Description]

Minor Issues

List minor improvements that would enhance the report:

• Issue 1: [Description]
• Issue 2: [Description]

Additional Research Needed

List areas requiring further investigation:

• Topic 1: [Description]
• Topic 2: [Description]

---

Validation Complete: ☐ Yes ☐ No Ready for Distribution: ☐ Yes ☐ No Reviewer: {reviewer} Date: {date}