Debunking the Parse API limit

So one of the things that all DBaaS sights are not good at is giving clear pricing information. Or maybe it is more accurate to say they are very good at making it difficult to determine what your actual price will be. Parse.com is no exception. What is strange, is I would actually think they would get more users if they actually told us up front since we would then be more likely to use them instead of shying away from them.

So I signed up for a Parse.com account to do some testing. First off their API is pretty easy to work with, especially if you are already accustomed to working with code blocks instead of callbacks. There were a few things I wanted to test. The second is how many API requests would I actually be using. The first is what would happen if I overloaded my API/s limit.

What actually counts?

Here is what counts towards your API limit (note: this is not an exhaustive list of operations that might count against your API limit, just the common ones):

  • Create an object
  • Update an object
  • Get a specific object
  • Query a number of objects

What doesn’t count against your API limit (note: this is also not an exhaustive list)?

  • Creating an account
  • Logging in
  • Analytics (such as app launches)

Basically, it seems like only API calls that have to do with data are counted against your quota. Analytics don’t have anything to do with data directly so they don’t seem to count. So lets take this one at a time and see how many API calls we are talking about.

Create an object
PFObject *obj = [PFObject objectWithClassName:@"Test"];
obj[@"Value"] = @(42);
[obj saveInBackgroundWithBlock:nil];

Each create object call such as this costs one API usage.

Update an object
PFObject *obj = <some existing object>;
obj[@"Value"] = @(42);
[obj saveInBackgroundWithBlock:nil];

Each time you save an object with one of the save… methods, it costs you one API call.

Note: There are class save… methods that you can use to save multiple objects at one time. While you would think this would be a way to save on your API calls. Unfortunately it doesn’t. If you have 50 objects in your array and you perform a class level save… call, then you are dinged for 50 API calls.

Get specific object
PFObject *obj = [PFObject objectWithoutDataWithClassName:@"Test" objectId:@"some-object-id"];
[obj fetch];

The first line of this code will retrieve a “placeholder” object in memory. This doesn’t actually trigger an API call, and can be used as a reference for object relationships. In order to actually do anything with the object, you need to call a fetch… method to fill in the data, this does cost an API call.

Query a number of objects
PFQuery *query = [PFQuery queryWithClassName:@"Test"];
[query whereKey:@"Value" lessThanOrEqualTo:@(50)];
[query findObjectsInBackgroundWithBlock:^(NSArray *objects, NSError *error) {
    for (PFObject *obj in objects)
        NSLog(@"Value = %@", obj[@"Value"]);
}];

The code above will cost only a single API call. It seems the library can load multiple objects via query at once, but not save them.

So lets take a useful example. We want to keep a running tally of the number of games a player has one. So we have an object class called Games, and it has a column called Won.

PFQuery *query = [PFQuery queryWithClassName:@"Games"];
[query whereKey:@"User" equalTo:[PFUser currentUser]];
[query findObjectsInBackgroundWithBlock:^(NSArray *objects, NSError *error) {
    if (objects.count) {
        PFObject *obj = objects.firstObject;
        [obj incrementKey:@"Won"];
        [obj saveInBackgroundWithBlock:nil];
    }
}];

This will cost us a total of 2 API calls. One to query/retrieve the object and another to save the data back to the database.

But 30 API/s is not very much!

may_i_have_some_moreYour right, it isn’t. Let’s say we have a game of 4 players going. Let’s also assume that the game ends for each player at the exact same moment and they all want to increment a “play count”, that means 2 API calls each times four players, or 8 total API calls in one second. If our app is even remotely popular that could easily break the 30 API/s free tier.

The 30 API/s seems to be a soft limit. I easily blew past this limit in a test by creating 700 objects, putting them in an array and then doing a bulk save operation to save all 700 at once. Looking at my API counts for the day, it did indeed increase by 700.

First off, the limit seems to be averaged over a minute of time. I ran the 700 save and on the dashboard it showed 11.683 API/s, which is equal to 700 calls / 60 seconds. So to max out our API limit, that should be 60 * 30 == 1,800 API calls in a single minute. So lets run that and see what happens. In truth, we are going to run it for 2,000 just to be sure.

Bingo, got some errors! And for that matter, I got an e-mail at the same time saying my app had hit it’s API limit. The error message even says: This application performed 1802 requests within the past minute, and exceeded its request limit. Please retry in one  minute or raise your request limit. That to me is a strong indication that they do indeed average your requests per second over a minute. And actually looking at the e-mail, it warns at 75% capacity, but since they all happened at once, who could tell the difference.

So really their free tier is 1800 API requests per minute. But lets face it. In marketing terms, 30/s sounds like you need to upgrade so they probably get more money this way. And downloading the graph data shows that the system did indeed drop some of my requests.

So 1,800 API/m. Now we are talking. See the problem I had from the first moment I heard about Parse.com is the per second limit. I can totally see hitting that 30/s limit. Let’s use a common scenario. Let’s say we get fairly popular and have a few thousand users. Let’s say that we post on Twitter, that if they login within the next 1 hour they get $5 of in-app-currency for free.

Okay for each user that means a minimum of two API calls, one to retrieve their money record and one to save it again. What are the chances that 16 users will happen to open the app within a single second and push us over our limit? I’d say pretty good chance that out of 5,000 users 16 of them might tap the app icon at the same time. Now, what are the chances that 900 users will all open the app within the same minute? I would say much slimmer. See now we are talking about 18% of our users all opening the app at the same time. I feel pretty comfortable about that ratio.

Bottom Line

While we haven’t committed 100% to using Parse.com just yet, I feel much better about them. Their 30 requests/s scares me. Their 1,800 requests/m, not so much.

Hopefully this information is helpful to you. While I can’t guarantee that other DBaaS providers do the same and average their burst limit, it would not surprise me if something similar is happening there too.

Retina – Why you so big?

Yea so last week I got a new MacBook Pro retina. I had been waiting for them to upgrade the MacBook Air to include retina, but with the introduction of the entirely useless 12″ MacBook, I don’t see that happening. So I went ahead and purchased the MBPr 13″ instead. My last laptop was a 15″ and I finally decided that was just a wee bit too big for what I use it for now.

The first thing I did, after trying to shove a CD in the non-existing drive, was to finish up last week’s blog and click the post button. After which I took a look at the homepage to make sure everything look good with the post, only to discover how hideous our site looked all of a sudden. I mean, those graphics were nasty and pixelated. Matt and I spent about an hour rectifying retinafying the situation. I feel so much better now. Rather small thing to do, but helped me feel like I actually accomplished something.

Okay so why mention Retina? Well, aside from the fact that I can rub my WoW resolution of 2560×1600 in Matt’s face, it made me take a look at our current graphic sizes. Up to that point we had been including graphic images of the cards at 900×1260. We wanted something large enough that it would look good on an iPhone 6+ and iPad Retina when the cards were zoomed up large. Thankfully our images compress really well since there is so much space on the card that is just a big block of color.

Even still, our entire deck (just one deck) was taking up about 8MB. Since we had just started talking about having potentially four decks at launch time, that was going to be a rather large download. Yes, there are lots of apps and games much larger, but I would like to keep our download size as small as possible. Especially since that is just the size of the card decks. That isn’t counting things like the menu images location themes, etc. Needed to shrink all that down a bit.

The first thing I did was rework how I access the card images. Originally I had been making multiple images for each card, like you would normally do, so that the image of the appropriate size is loaded on the appropriate device – no resizing needed. Eliminating all those extra images cut off about 3MB from the deck size. So instead we now have a single full-size 900×1260 image and then we resize it on the fly. Well I know that would slow things down and drain battery, so I implemented a simple cache mechanism.

When I am ready to load/draw a card image, I call a method that requests that image at a specified size. If I have a cached copy of that image on “disk” then I load it up and return it. Otherwise I load the full-size, resize it and save it to disk before returning to the caller. Next time I need that image it’s instant. What about the cache folder, isn’t it going to be huge? I thought it might, so I put in some temp code/UI to report the size of the cache folder. After playing a dozen or so games (so nearly all the cards should be cached at the various sizes I need), the cache folder on my iPhone 5 is only about 1.6MB. Larger devices like the iPhone 6+ and iPad will probably be a bit larger, but probably only 2MB maybe.

So that got us down to 5MB. I still wanted more. Remember I said they were 900×1260 pixels? Well, the largest device we have is the iPhone 6+, in terms of raw pixel count. It runs at a scale of 3x, so that means I am only 300×420 in actual drawing. The 6+ runs at a (scaled) resolution of 736×414 landscape resolution. So that means at our current size a card could fill the screen vertically without any scaling. I’m probably never going to show a card that big. I figure at most I will draw a card at 3/4 height, probably a bit less. With that in mind, I took the cards down to 600×840, or 200×280 scaled resolution. So I reduced the resolution by 33%, but my file size for the entire card deck dropped to 2.8MB, which is about a 45% decrease. I’m feeling pretty good about that size.

Apple and others are constantly pushing the DPI higher and higher on our devices. As a user I love it. The text and images are so crisp on my phone and laptop it makes everything much easier. And the super hi-res dragon images on my desktop background are simply gorgeous. But the downside to all that is the size required to download things keeps going up. Some apps I download on my phone are 50-60MB, and yet really have no graphics that I can see. It boggles the mind sometimes how they can be so bloated.

Mexican theme anyone?

I can’t tell you how happy it made me when Matt starting talking about the rice card. Not because I love rice that much, but because he was having such a hard time making rice look like rice with “low quality” vector graphics. I laugh not just because I like to see Matt have a hard time, though there is some of that too, but because it’s one of those things you can’t help but think, “it’s just so stupid.” I mean seriously, how hard can it be to draw rice? Well after seeing a few of his attempts I must agree, it’s really hard.

The upside to the effort he put into the rice is that he also completed our second deck of cards. Over the week as he was pumping out the cards I was rebuilding the game every night with whatever new cards he had (and a few bugs I included of my own) so that we could try those cards out in game. As he finished up the Taqueria deck I was putting the finishing touches on being able to fully theme the game board based on what food establishment, i.e. level, you were playing.

I must say, there is a real sense of accomplishment at being able to, for the first time, run the game and actually pick which level you want to play. And have it show a completely different theme depending on which level you go with. Makes me feel like we are really starting to make progress.

Odds and Ends

Actually would like to say a lot more, but I should really put in some programming time too. But very briefly, I mentioned above some bugs I included. Part of the image resizing process I did completely and utterly broke the game. To be honest I don’t even remember what it was at this point, but it sure put a damper on Matt and Evan being able to test the game after I uploaded the latest binary and put my computer away for the night. It was so broke, you couldn’t play a single card. Oops. Guess I should have actually done some simple testing.

I mentioned last week that we have Parse tracking some basic analytical data about each game played. Well after a week of tracking we looked over the current results and were rather surprised. Over the course of the week, we (mostly Evan) played nearly 70 games. The average length of the games was 3.5 minutes, with a win rate of only 49%. Meaning one of the computer AIs won half the time. That was surprising to me since I hadn’t done a ton of work on the AIs and figured they had a long way to go still. Apparently I need to dumb them down some more! I already dumbed them down a bit because Matt’s wife was having a hard time with them. I kind of want to crank them up to smartest again and see how our win rate drops.

At any rate, that gave us some good numbers to begin figuring out exactly how long we can expect it to take a person to reach a goal of X to unlock another level. With the current averages, that means that each win would take about 7 minutes. So if we want them to play for 2 hours before unlocking the next level (probably not that long) they would need to play around 34 games, which would give them 17 wins. In truth we are going to have to dumb down the AI a bit more so that we can make them smarter as they progress through the game. Hopefully that will keep the interest up for people when they first start.

Week 4 : Daniel