Apptimize YCombinator Application

yc.apptimize.com/admin

Apptimize lets you AB test mobile applications. You keep the native experience without needing to push changes blindly or rely on users to update. There’s a web interface to manage experiments, and a WYSIWYG interface for non-programmers. Apptimize removes the pain of designing a controlled experiment, serving variations, collecting results, and calculating statistical significance. Right now you have to be a developer and statistician to AB test a mobile app, but we make it so that non-programmers can AB test too. Apptimize makes optimization as easy for mobile as it is for web. Apptimize technology could transform the process of testing and pushing changes and be integrated into 100% of apps.

We met a couple years ago through mutual friends and started working together when Jeremy convinced Nancy to leave NYC for the Bay.

We prototyped an app called Firesale that helps people sell unwanted stuff. To create a market of buyers, we brought on full-time Craigslist market makers. The Craigslist expert users complained about the process of being first to email a poster, so we optimized the messaging to make transacting as fast for them as possible. They also complained about Craigslist lacking a reputation/identity system, so we implemented one. We put Firesale on hold to work on Apptimize.

Nancy: trader who ran the Fixed Income Quantitative Strategies team at GETCO (GETCO grew from 100 to 500 people to become the premiere algorithmic trading company); world class expert in Fixed Income trading and exchanges. Jeremy: owned IndexedDB (the emerging w3c standard for storing data in a browser) within Chrome; edited the spec, worked closely with Mozilla and Microsoft on the design, and wrote most of the initial implementation in Chrome/WebKit; simultaneously started the London Chrome team.

Nancy wanted to work in the Middle East but there wasn’t a culture of internships. Nancy discovered if she didn’t mention she was just a sophomore she could interview as a consultant (and get a company car and phone). She was the first student ever hired for Mercury’s R&D office in Israel (a load testing company acquired by HP). At Google, Jeremy became an expert in free travel. After getting on shortlists for university recruiting, he positioned himself as a datacenter expert and visited many across America. After targeting developer relations, Jeremy got on the shortlist for places like Moscow, Berlin, Manila, Singapore, Sydney, and Tokyo, giving talks, meeting partners, and exploring- all for free.

Apptimize works and we just launched our private beta this week! We have 100+ signups but we only accepted 2 friends this week because we are working closely with our first customers to shape the future of our product. The beta has the Android library, a website dashboard to manage experiments, and a results page showing statistics and conclusions. The WYSIWYG interface will be ready in a few weeks. Our research suggested starting with Android because Android developers rely on freemium (compared to iOS who make a lot off premium) and want to AB test to optimize in-app purchases, etc. Our iOS version is coming in a few weeks.

We started in January, and Apptimize is currently ~8K lines of code (not including libraries, html, or css) and works end-to-end. The frontend is JS, CSS, and Angular. We’re on EC2 mainly using PostgreSQL, nginx, and Netty/Java.

We picked this idea because Jeremy had looked for a mobile AB testing solution when working on Drawchat, but couldn’t find one. Three 50+ people companies, 3 YC companies, and 10+ indie developers have signed up to beta test our product. All the programmers/contractors we’ve interviewed have also asked to sign up for our private beta. This is an immediate need for most mobile companies. Nancy is an expert in experiment design and data analysis. Jeremy is an expert in mobile and has built many efficient, scalable backends. We both love being data driven and view life as an experiment.

Most wait for app store approval and push many changes simultaneously. They eyeball the results and haphazardly rollback suspect changes. Desperate people resort to basic, home-grown solutions. Because of other projects, Switchboard and Clutch.io evolved incomplete solutions (we noticed errors: randomization mistakes that mess up the experiments, poor error handling, malformed responses that’d crash your app!). There hasn’t been much focused effort towards creating a seamless AB testing experience for native apps. AB testing for mobile is a technologically harder problem than for websites due to challenges particular to mobile devices (ie. intermittent internet, lack of cookies/iframes, users running different versions). Existing solutions ignore complexity whereas we view handling it as our core business.

Our competitors are developers building for other developers, so most only offer programmatic interfaces. We understand often the goal setters and decision makers aren’t programmers. Apptimize makes it simple for non-technical owners, product managers, designers, and marketers via a WYSIWYG interface and a website to control and create experiments. Our experimental setup, results, and analysis will be superior. Stanford PhD’s helped with our statistics by pointing out problems with competitors’ setups (ie. fixed sample sizes, small data set handling). We’ll target companies who don’t monetize through app sales, instead using apps for branding, coupons, other off-app conversions. Although our first users are indie developers, most profitable apps make <$2K per month, so we’ll grow to targeting corporations like United, Starbucks.

Several companies very recently entered the game. Swrve has so far focused on games. Pathmapp is focusing on overall analytics (pretty different from our approach). Abstate is unlaunched. Artisan and Arise.io have buggy, immature products. A risk is that Visual Website Optimizer or Optimizely will decide to focus on expanding from websites into native apps. Native might be a natural next step for them since they offer web app support in premium plans, so we’ll grow aggressively. We think there’s no dominant player because nobody has made anything good yet. Our goal is to be the best.

The plan is a monthly subscription. We’ll offer customers help with experiment design. If we charge premium customers $1K per month and get 200 customers (less than 2 sales a week) over 2 years we’d make ~$2.4MM per year 2 years in. Artisan (launched this month) claims to charge $1K-$10K per month, so that’s possibly a better price. Ultimately we want to be the default way people change their apps. Everyone would use Apptimize to test each idea, and then use Apptimize to deliver the change to users. 100% of apps would use our library to reduce time to propagate changes and tighten the app development cycle. We’d help erase the line between apps and the web.

Our first customers are our friends’ startups. To target our next customers, we downloaded their apps and their competitors’ apps and are designing experiments for them. If they find the pre-designed experiments useful, they can easily start testing with those the instant they sign up. We’ll offer customer referral rewards such as temporary premium memberships. We also want to make it easy to see and implement case study results by suggesting experiments to potential users. For marketing, we will ask and answer stackoverflow and Quora questions regarding how people AB test on mobile. We could partner with companies in related fields like App Annie or Parse.

EEG machine to read babies’ minds. We like playing with our Emotiv machine, know prominent MIT/Stanford researchers, and see parallels between EEG analysis and high frequency market data for financial instruments (both systems produce massive amounts of data that seem random but aren’t). A page-less browser using crowdsourcing. It’d show logical dependencies, assumptions, relationships between ideas, and best arguments for and against each belief.

People think it’s red, but no one knows the best button color.