This is how advertising should work

My new landlord sent me an email today to let me know the dimensions of my windows in the apartment I'm moving into. But what do I know about drapes, or blinds, or whatever one uses to cover windows.

But what's that at the top of my webmail inbox?

Now I could treat this as an invasion of privacy or being over exposed to advertisements. But honestly, this is an ad that makes my life easier.

I haven't looked at their product/prices yet so in no way should you take this as an endorsement. However, point one in their favour is that they are paying for me to receive information. Clearly, my receiving this info is mutually beneficial yet they are incurring all of the costs. That's the right attitude as far as I'm concerned.

Eating Crow

If I'm going to write a prediction post it's only fair that I write a follow up to rate myself.

1. The product will be too expensive. Obviously I was very wrong here. There were a lot of leaks that the price point would be a lot lower than people expected and I clearly ignored them. But let's not pretend the unit is $500 either. 16GBs is clearly not enough storage to use the iPad in any reasonable way. And personally I'm resistant to paying for another monthly data bill. But we aren't talking about apple hifi or iPhone gen 1 prices here.
2. Most developers will be locked out, except perhaps in a ridiculously restrictive way. Honestly, my expectation was that the only way you would be able to get applications onto the device was through the app store. But I over stated because I didn't expect iPhone apps to be compatible (and I assumed the API would be released at WWDC). I figured even if there were app store applications available immediately it would seem pretty pathetic because I was expecting a device that looked more like a mac than an over-sized iPod. All that said, I'm still going to give myself half points for this one, rather than no points. Just because we've gotten used to the idea that you have to get permission to put an application on your own device doesn't mean it's not ridiculously overly restrictive.
3. Content channels will be incredibly locked down. Again, I was expecting a more general/mac like device instead of an over-sized iPod. If this had been a mac tablet then it would have seemed absurd that it couldn't play divx. As a large iPod we've been conditioned to expect it not to play divx. But I think I really have to give myself a zero on this prediction simply because apple went with a standard and used epub. However, here are some things to consider. My guess is apple will treat epub like aac: you will be able to install non-drm'd epubs from anyone but only apple will be allowed to offer drm'd epubs. Where does this put amazon? I'm sure the kindle app will still be available but there will be advantages to using the native iPad book reader. Amazon's content deals won't let them sell drm free books and apple won't let amazon sell drm'd books into the iPad book reading app. So the only major seller of content of books for the iPad will still be apple.
   
4. There will be some incredibly obvious feature that is inexplicably missing. Really this is a silly prediction. Of course features are inexplicably missing. Because the explanation is available to those who are working in the real world who are dealing with timelines and pricing restrictions. So yeah, there is no gps in the non-3g version, no camera, no multi-tasking, and no flash. And there are probably many more. So I get full points but how could I not.
   

So why were my predictions so wrong? Am I just an idiot? I think all my predictions fairly cited apple's history for first gen products. Perhaps what I missed is that maybe this isn't a first gen product. This really is just a big iPod touch. Book store is new and so are these iWork apps but really they are just riffs on current businesses. And we haven't seen how they will pan out yet.

Careful overextending that model

In my current work I spend a lot of time working with models that try to predict the likelihood of a customer's future action. From time to time we have a need to model something new quickly and it is very tempting in those cases to take a different but similar model and see if it can simply be re-aligned to fit the new circumstance. Sometimes this works fine but it is interesting how often this fails dramatically.

But I have to be careful about writing about specific examples from work. So let's consider a hypothetical example instead to see the danger of assuming a model extends.

How about consider a software company who has started a pilot program of giving potential clients six week trials of their product. Now they would like to know as soon as possible what sort of customer is most likely to convert so they can focus their sales force most effectively. Rather than wait six weeks for results they decided to try to observe who will convert in the first week (ie a target of one week conversion). That way after one week, plus development time, they will have a model that they can start using. While this model won't have much accuracy in predicting the probability of converting in six, surely it will rank order correctly with a model that used a full observation period. After all, if a customer that doesn't convert in week one has similar characteristics as someone who does then it seems fair to conclude they just need a little bit more of a push to become more like their cohorts.

However, this model fails disastrously. Those who were supposed to be the most likely to convert ended up being the least likely. So the model is revisited to see what could have gone wrong. It is discovered that the biggest drivers of the model were

• lots of use of the software
• lots of interaction with the sales department (asking questions about price, etc)
• indicates on questionnaire that the software is to be used on an urgent project
• asks a lot of questions about the features that are locked by the trial.

And that's when the analyst hypothesizes his mistake. Instead of targeting for the desire to convert, he suspects that he's targeted for the need to make a decision quickly. So those who were rated highly by the model and didn't convert in the first week, didn't convert because they had decided in that week that they would never convert.

To test this theory the analyst checks whether those who did not convert, but were predicted to, had bought the competitor's software (the trial included some spyware so he can check for such things). And there is the answer. Not only had they bought the competitor's software in large numbers, but usually with in the first week of the trial. Those that were considered the best leads had, unbeknownst to the company, had already ruled themselves out of being future customers.

Now real life examples are rarely this clean cut. In reality, the population with a high likelihood of converting in the first week would probably be made of up a mixture of types. Instead of moving as a group to being the worst converters, you would probably some sort of indeterminate result.

So how could this mistake have been prevented. Well first by validating that there was some justification that the model could be extended. Perhaps he could have validated against the second week to find out sooner that he had made a mistake. But more importantly, the analyst waited too long to try to understand his variables. When making a model like this it is important to try to understand why each of the drivers of the model are actually predictive. And then test the relationships that you believe exist. That is to say construct a narrative that explains why your variable predicts the way it does, consider the other consequences of this narrative, and see if these other consequences exist in your data.

Even though we aren't wearing lab coats it is a good idea to keep in mind that we are still doing science. The scientific method makes a pretty good guide.

Implement US health care reform at a state level

Slate asks how the Democrats can still implement healthcare reform. This was my response:

Consider the major points of the senate bill: state level exchanges, individual mandate, no refusal based on pre-existing conditions, and subsidies paid for by Cadillac insurance tax. Assuming the tax and subsidy is distributed equally, the benefits for any one state implementing this bill is not affected by how many other states implement this bill.

And since health insurance companies are not allowed, and would not be allowed, to operate in one state and insure someone in another, there is no issue of a loss in cost savings due to smaller pools being insured.

So if there is no advantage to implementing this nationwide, instead of in just the states that can pass it, why not just pass the bill at a state level in states where it can pass. A domestic coalition of the willing.

There is a non-zero sum game in game theory called stag hunt which has two defining characteristics: a single defector is able to lower the payoff of the co-operators, and the single defector lowers his own payoff by defecting (unless there is another defector). It is this situation that justifies modern democracy where the majority is able to enforce the co-operation of defectors (whose defection wouldn’t even be in the defector’s own self interest).

But this isn't stag hunt. A single defector, or many defectors, has almost no affect on those that co-operate. So implementing at a state level may even be more democratic in this case. After all, shouldn’t individual choice be preserved where possible.

There may be two migration issues to worry about in this strategy: those with pre-existing conditions moving into states that implement this bill, and healthcare providers moving out. However, there are lots of other ways that entitlements vary from state to state, so if migration isn’t already a strategy for those seeking extra entitlements there’s no reason to think this entitlement would be any different. And we’ve been told that this bill won’t hurt healthcare providers so there should be no reason for them to leave.

Finally, consider our healthcare in Canada. While we have broad laws at a federal level requiring single payer and transferable health insurance, healthcare itself is actually implemented at the provincial level. This is because (a) the provinces were considered too varied to manage all of healthcare from Ottawa and (b) it was thought that trying to manage the healthcare of 30 million people in a single institution is a task that only a crazy person would attempt. 300 million is a bit more.

My pessimistic/realistic apple tablet predictions

So I wanted my next blog post to be a little less controversial but I think I'm about to kick the biggest hornets' nest yet: apple fan-boys. But the next apple announcement is around the corner and I don't have much longer to be a wet blanket.

Note, that I'm basing these predictions completely on apple's track record with first gen devices. This isn't what I want to see happen, just the pattern I'm used to from apple. Also be warned; I'm a bit rant-y.

1. The product will be too expensive. What did the iPod, iPhone, apple speaker and appleTV all have in common on day one? They looked fun but they were way too expensive to consider actually buying under normal circumstances. In each case multiple hundreds of dollars to expensive. Now, they come down in price eventually (assuming the product isn't discontinued first). And they can get away with the high price because there is a cluster of consumers at the high end of the demand curve that are sufficient to buy all of their initial supply, which is usually pretty small. But I don't think it makes sense to expect a sane price.
2. Most developers will be locked out, except perhaps in a ridiculously restrictive way. I've heard various predictions on how developers should be excited because they will have a whole new way to make a ton of money. But apple has never let developers make money on their products on day one. And when you are allowed it has to be exactly on their terms. When the iPhone came out it was just assumed that anyone who was buying one would jailbreak because apple had locked it down so much. This is still true with the appleTV. This is the company that made an iPhone that had a recessed headphone jack so the vast majority of third party headphones wouldn't work with it. And the company that disallowed the iPhone podcaster app, allowed it, then re-disallowed it, put it in a three month penalty box, then re-allowed it again.*
3. Content channels will be incredibly locked down. In fact all content will go through apple. This one seems obvious to me. The appleTV is really the only set top box left that can't stream netflix in the US. Neither the iPhone nor the appleTV have an approved way to play divx files. Their may be a crack in this lockout similar to podcasts on the iPod. But if you want to charge for your content you are going to need to go through apple to get onto their device. And then you'll have to wait (and as I understand it, wait and wait and wait) to get paid by apple. So I expect predictions that this will be the perfect universal content device to be very very wrong.
4. There will be some incredibly obvious feature that is inexplicably missing. The appleTV doesn't have a tv tuner, and no approved way to add one. The iPhone has had bluetooth from day one but no way to use an external bluetooth keyboard. The iPhone took multiple generations to get cut and paste. The apple mouse still doesn't have a simple second button. (No, that doesn't count. I said a "simple" second button.)

So might these predictions be wrong? Sure, and I hope they are. But if they are wrong it's because apple has decided to alter their behaviour. There are those who expect this to be the perfect universal device that can be crafted into whatever they need a tablet for. I have no idea why they think such a device could ever be a first gen apple product.

*Ok, I need to rant about this a little more. Let me be clear: apple hates developers. I used to think they were generous because they gave away xcode. But then I realized that they aren't giving away xcode, they are exclusively bundling the only IDE that can develop for their platforms with their computers. So as a developer, you are allowed to develop for the iPhone, but to do so you need to buy one of their computers, only develop on that computer, and then seek approval for what you've done to get it into the store. There, I'm done ranting.

When criticizing Israel is anti-semitism

A friend of mine linked to a story in the tyee that I responded to on facebook. I've decided to copy that response here.

While it is certainly true that not all criticisms of Israel are acts of anti-semitism, there are those who call themselves critics of Israel who really are spouting hate and anti-semitism. To understand this you have to appreciate just how our people, our land, our history, our customs and our spirituality are interwoven in Judaism. Somewhere just under half of all Jews are Israelis; and a high majority of Jews in the Diaspora self identify as supporters of Israel.

Absolutely you can find examples of Jews who reject Israel - even groups of them. You can also find Jews for Jesus. I don't think anyone would say that because of this the Jewish people are ambivalent on whether Jesus has a role in Judaism. It is simply noted that there are exceptions.

Of course it is reasonable to question and criticize actions of the government and the state. Ex-prime ministers of Israel do this. The current Israeli president has done this in the past. And of course Israelies don't vote as a monolithic block. In fact in a first past the post like system Netyahu would probably not be Prime Minister right now; and Lieberman would certainly not be a serious player.* (This is not to say I think that this government is illegitimate in anyway.)

What crosses the line into anti-semitism is when people say (and I've met these people in university) "I don't have a problem with Jews; it's just Israelis I hate" (and they felt comfortable saying this to my face). And it is easy to find examples of people crying "Zionist thieves" at rallies. To be clear, when you express hate for Israel you are expressing hate for Jews. When you call Zionists thieves you are calling me and my family criminals.

So yes, not all criticisms of Israel are anti-semitism. But why let the anti-semites hide amongst you? Why be an apologist?

*This is one of my personal complaints of the Israeli government; in attempt to be more in line with academic theories of "democracy" Israel has a proportional representation system. Because of this voters aren't required to seek out a compromise with their vote in order to influence the government. This has resulted in Knessets made up of bizarre and unstable coalitions with those on the edge getting to hold the balance of power. First past the post would solve this. STV would not.

New job/blog title

I recently changed my blog title, and most people I know already know why. It's because I recently changed my career which I think affects my perspective, and probably the topics I will blog about.

I'm now a credit risk analyst at a Canadian bank. Which means I look at the behaviour of our credit card customers and work on creating scores that try to predict their future behaviour, and the future behaviour of applicants. I won't mention the name of the bank because I don't want to speak for them and I certainly don't want to pop up in searches for them.

In the process I'm building up my modelling chops, learning about companies like fico and sas, and getting a new look at how various people use their credit. First big lesson, various people vary a lot. There's no such thing as a perfect or invincible score because people of all types and behaviours go bankrupt all of the sudden. For some behaviours that might be a 1 in 100,000. Which sounds like an absurdly low risk, but for the individuals that realize that risk it is a 100% reality.

Anyways, I feel like I'm back with my mathematical roots, though I'm taking more of a statistics bent this time.