The Optimization group within the Analytics team provides recommendations to the business for opportunities, runs the A / B test lifecycle, and measures results through a variety of reporting tools.
You will be responsible for working with partner teams providing strategic support in running a robust A / B testing program.
You will also be responsible for developing standardized testing metrics and reporting for the organization that keep us informed, and are the fingers on the pulse of our business.
The ideal candidate for this role will be highly motivated and will use their technical skills to quickly make an impact ensuring that our growth curve extends into 2018 and beyond!
Own and execute full A / B test lifecycle work closely with Product Managers to assess opportunity, develop requirements, schedule, drive test implementation, monitor, and measure / validate results.
Analyze results using internal reporting tools, BI tools, and Web analytics tools Mixpanel, Optimizely, and Google Analytics.
Work with Product Managers to quantify revenue potential for all tests; prioritize these tests to optimize for strategic initiatives as well as incremental revenue.
Use technical knowledge and testing best practices to ensure optimal test setup and performance.
Monitor testing dashboard daily for unusual trends and work with teams to resolve in a timely manner.
Understand test variables and determine what drives short and long-term fluctuations in results. Proactively cut off or extend tests based on monitoring of results.
Ensure all tests results and summary are completed and shared in a timely manner to external teams and executive leadership, and communicate results effectively by telling the story behind the data.
Communicate results and potential financial impact of A / B tests to external teams and executive leadership.
Perform ad-hoc analysis for testing opportunity and results using SQL.
Work closely with product analysts to improve test reporting, validation, reducing noise, and statistical modeling / significance.
Research and ideation for product improvements with UX team.
Training PMs and external stakeholders in interpretation of A / B test results using internal reporting tools.
Documenting and advocating for optimization best practices.
Bachelors in engineering or another quantitative field.
A minimum of 2 years of professional experience in an analytical role.
Experience implementing, running and measuring A / B tests using testing tools like Optimizely, segmentation, and traffic acquisition tools like Mixpanel and Google Analytics.
Experience writing SQL queries.
Able to prioritize and manage multiple stakeholders.
Experience working with large datasets.
Creative thinking and problem solver.
Ownership / entrepreneurship mentality.
High level of organization.
Experience in a SaaS-based business.
Professional experience with statistical analysis.
Beginner level proficiency with programming of some kind, preferably Python.
Having been a project lead, SME, or performed training / onboarding.
Broad exposure to tech through a diversity of internships.