Member Of Technical Staff (Data Scientist/Engineer, Online Metrics)
Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and specialized data sources. The Answer Quality team ensures that our prompts, tools, search, and specialized datasets, combined with both frontier and in-house models, create the best possible
What this role actually needs.
Perplexity serves tens of millions of users daily with reliable, high-quality answers grounded in an LLM-first search engine and specialized data sources. The Answer Quality team ensures that our prompts, tools, search, and specialized datasets, combined with both frontier and in-house models, create the best possible Responsibilities: - Discover and validate online signals from user interactions that serve as reliable proxies for true answer quality - Design and implement novel online metrics to be tracked both in A/B testing and on product health dashboards, ensuring alignment with ground-truth evaluations - Analyze experimental results to validate these metrics, ensuring they accurately predict user satisfaction and drive product decisions - Build and maintain the data pipelines that calculate these metrics at scale, delivering actionable quality signals to Search, Product, and model training teams - Communicate findings and bring clarity through close collaboration with Product and Search teams - Operate in a small, high-impact team where your work directly shapes how Perplexity measures and improves Answer Quality Requirements: - MS in a technical field or equivalent experience - 4+ years of experience working as a Data Scientist, Analytics Engineer, or related role - Experience working on search, recommendation, or LLM-based products, with an emphasis on designing online metrics and analyzing A/B experiments - Strong proficiency in Python and SQL (expected to write production-grade code) - Deep knowledge of statistical analysis - Experience with Business Intelligence (BI) tools for visualization and reporting Company context: Perplexity is an answer engine — an AI-native search product used by millions of professionals daily.
Day-to-day expectations
Perplexity lists these responsibilities for the Member Of Technical Staff (Data Scientist/Engineer, Online Metrics) role.
- Discover and validate online signals from user interactions that serve as reliable proxies for true answer quality
- Design and implement novel online metrics to be tracked both in A/B testing and on product health dashboards, ensuring alignment with ground-truth evaluations
- Analyze experimental results to validate these metrics, ensuring they accurately predict user satisfaction and drive product decisions
- Build and maintain the data pipelines that calculate these metrics at scale, delivering actionable quality signals to Search, Product, and model training teams
- Communicate findings and bring clarity through close collaboration with Product and Search teams
- Operate in a small, high-impact team where your work directly shapes how Perplexity measures and improves Answer Quality
What a strong candidate brings
These requirements are extracted from the source listing and normalized for UpJobz readers.
- MS in a technical field or equivalent experience
- 4+ years of experience working as a Data Scientist, Analytics Engineer, or related role
- Experience working on search, recommendation, or LLM-based products, with an emphasis on designing online metrics and analyzing A/B experiments
- Strong proficiency in Python and SQL (expected to write production-grade code)
- Deep knowledge of statistical analysis
- Experience with Business Intelligence (BI) tools for visualization and reporting
Why this listing is more than a copied job post.
Member Of Technical Staff (Data Scientist/Engineer, Online Metrics) is framed against UpJobz source checks, country scope, compensation visibility, and work-authorization signals so candidates can make a faster go/no-go decision.
United States tech market
United States roles on UpJobz are filtered for high-tech relevance, source freshness, and actionable employer detail before they are allowed into SEO surfaces.
Compensation read
$200K - $320K is visible before the click, so candidates can compare the role against local market expectations before applying.
Work authorization read
Current extracted signal: Open to TN, H-1B, and OPT candidates already in the United States. UpJobz treats this as a search signal, not legal advice, and links visa-sensitive roles back to the relevant visa hub where possible.
Location read
Hybrid roles in New York City should be compared against commute, local salary bands, and nearby employer demand.
Browse similar jobs
Turn this listing into an application plan.
This is the first pass at the premium UpJobz layer: a fast brief that helps serious applicants move with more clarity.
Next moves
- Tailor your resume around ai and llm instead of sending a generic application.
- Use the first two bullets of your application to connect your background directly to member of technical staff (data scientist/engineer, online metrics) is a high-signal hybrid role in new york city, and it is most realistic for open to tn, h-1b, and opt candidates already in the united states.
- Open the role quickly if it fits and bookmark three similar jobs before you leave the page.
Interview themes
Watchouts
- $200K - $320K is visible, so calibrate your application around the posted range.
- Use open to tn, h-1b, and opt candidates already in the united states as part of your positioning so the recruiter does not have to infer it.
- Show concrete examples of succeeding in hybrid environments.
Keywords to match against your background
Use these terms to decide whether your resume, portfolio, and recent projects line up with the role.
Apply through the employer source
Open the source listing from jobs.ashbyhq.com, confirm the role is still active, then apply on the employer or ATS page.
Source: jobs.ashbyhq.com · Source ID: 768efd34-3854-4c9b-9686-2a128ab35405 · Confidence: 94/100 · Last checked: May 7, 2026
How UpJobz verifies job sourcesContinue browsing tech jobs