It’s not every day that a so-called FinTech darling has to tweet out that they do not engage in “phrenology/physiognomy” but that is exactly what self-proclaimed insurance disruptor Lemonade did back in May. How this came about is that the individual managing the firm’s Twitter account typed out a tweet thread talking about how Lemonade is ‘different’ from traditional insurers. The following is the – now-deleted – offending tweet as screen-shotted by @GrowthLike:
Now, Lemonade has since put out a blog post about how the firm allegedly does not use AI to decline claims but rather uses its facial recognition abilities “to flag claims submitted by the same person under different identities.” In addition, the firm suggested that those who are interested in the firm’s AI policies and philosophy ought to look at a post entitled “AI Can Vanquish Bias” written by their CEO Daniel Schreiber. In the piece, Mr. Schreiber is adamant that “machines can overcome the biases that contaminate their training data if they can continuously calibrate their algorithms against unbiased data.” As such, it does not matter if the initial parameters of software are biased so long as they have the ability to learn from, and correct, their mistakes. Which, sounds decent enough right?
Well, first off, let’s ignore the disputed realm of AI ethics, as well as arguments to the contrary of Mr. Schreiber’s claims, and assume that AI can be made unbiased. Secondly, let’s ignore that Mr. Schreiber and I have exactly the same level of formal AI training – zero. Thirdly, let’s ignore the fact that Lemonade’s premise is that – according to Schreiber in an interview with Forbes – insurance “has gone largely untouched by technology for more than a century” which ought to come as a surprise to insurance companies such as Allstate who have been increasingly leaning on technology over the past decade. For example, these efforts were obvious enough for a Harvard MBA student to write a brief case study about them in 2017, and ironically Forbes ran an interview with Allstate’s CEO a year before Schreiber’s in which they declared that the company is now “a data and technology company.” And lastly, let’s ignore Evan’s law which states that any company publicly pumped by noted human-cantaloupe crossbreed Scott Galloway is probably eight shades of dog shit.
What’s really the issue here is that Lemonade, like many AI/FinTech firms, requires two propositions to be true in order to justify their operational and business claims. These propositions are:
Human behavior – and thus business operations – overwhelmingly consist of computation and pattern recognition.
Although technology is never perfect, the mistakes it makes relative to human errors are either fewer, or the cost savings involved with technology adoption are high enough that added mistakes are more than compensated for.
While both of these propositions are true to an extent, they are less true than the techno-optimist crowd – and assumably the management of Lemonade – would like us to believe.
Let’s start with number one. In assuming that human intelligence consists of computations and pattern recognition, the AI/FinTech crowd makes the common mistake of applying a methodology beyond its useful limits. I like to call this short-coming the Charles Murray Trap.
For those who don’t know him, Murray is an emeritus chair at the American Enterprise Institute who holds a Ph.D. in political science from MIT. His books include Losing Ground and Coming Apart which respectively deal with American social policy and the economic trajectory of low-income white Americans. However, if you Google Mr. Murray you’ll quickly find a page on him from the South Poverty Law Center that paints him as a racist pseudo-scientist.
The reason for this is that in 1994 Murray, along with Richard Herrnstein wrote a statistically focused book called The Bell Curve in which there are two chapters entitled “Ethnic Difference in Cognitive Ability” and “Ethnic Inequalities in Relation to IQ.” As you can probably imagine, Murray and Herrnstein did not take the color-neutral approach. Instead, they basically argue that black people are less intelligent than white folks by about a standard deviation.1 Obviously, these findings are easily thrown into the trash but the thing is that Murray and his co-author did not just pull numbers out of their asses. Instead, they used professionally accepted data sources, ‘ran the numbers,’ and then came up with their conclusions. In doing so though, Murray – as he’s done in follow on work – made a categorical mistake in terms of matching research questions to methodologies.
You see, Murray’s idea is that statistics don’t lie and that they speak to seemingly natural laws. Except that there is no reason to think that freshman-level statistical methods can answer questions surrounding unequal performance in structurally different social contexts. Murray’s analysis might present the shape of an issue – but it does not have the ability to put forth a cause as to why so instead he chose a weak-tea pseudo-Darwinian explanation because it ‘fit the curve’ so to speak. Not to mention that in order to take Murray’s work seriously you have to look past arguments such as Nassim Taleb’s which posit that the entire notion of IQ is “largely a pseudo-scientific swindle.”
While Murray might be an extreme example of using the wrong tool for a given situation, he is not alone in terms of prominent folks falling into this trap. For example, take UC Davis economist Gregory Clark and his rather prominent, Princeton-published, work A Farewell to Alms. In Farewell, Clark looks at the ‘rise’ of ‘middle class values’ and decides that the proper explanatory framework is Darwinian evolution. To make clear that I am not misjudging Clark’s thesis allow the following quote from the work regarding why England was the first to undergo the Industrial Revolution:
“The characteristics of the population were changing through Darwinian selection. England found itself in the vanguard because of its long, peaceful history stretching back to at least 1200 and probably long before. Middle class culture spread throughout the society through biological mechanisms.”2
According to Clark, rich people possess a genetic predilection for bourgeois behavior – AKA ‘middle class culture.’ These people in turn, because they are rich in a world of scarcity – can have more children who survive to adulthood. In turn, this means that the successful merchants out-reproduce their peasant counterparts and bippity boppity boo each generation, in turn, contains more and more people of the bourgeois persuasion. While you might at first glance expect then that there would only be rich people left after enough generations Clark notes that just because one has a middle-class mindset does not mean that they aren’t morons, profligates, or unlucky. Thus, the values ‘descend’ through the classes over time.
There are several issues here. The first is that it assumes a pretty damn quick evolutionary timeline – peasants to debutantes in a few centuries. Secondly, it requires centuries of general peace which occurred in more than enough locales outside of England prior to the period noted by Clark. Thirdly, it’s an untestable theory due to the fact that there is no ‘middle class values’ gene. Oh, and fourthly, Clark’s argument is a soft-form of eugenics – if such a thing is possible – because in Clark’s world economic progress relies on the rich having the ability to mate like rabbits. Thus, if you want to maximize growth and optimize the ‘growers’ then the most pertinent policy would be to sterilize the not so bourgeois while having a middle class harem on every street corner.
How does this apply to the world of AI? Well, the previous two examples are just some of the more egregious versions of the trash that ensues when seemingly reasonable3 folks approach serious matters with the wrong tools. Where this comes into contact with the realm of AI is that given the field’s tendency to view the human mind as consisting of computational and pattern-recognition capacities, all the algorithms and platforms have to do is simply count faster and notice tendencies more quickly in order to kick ass and take names. This may work when spotting tumors on x-rays and developing mathematical proofs but there is no reason to assume that it works in areas where non-quantifiable data is pertinent such as . . . wait for it . . . insurance claims or even corporate lending. The recent poster child of the latter is the multi-billion dollar failed factoring firm Greensill Capital.
Greensill waddled around the world claiming to be “democratising” finance through the usual buzzwords of “Big Data,” “Machine Learning,” and “Artificial Intelligence.” The firm’s business was good ol’ fashioned factoring – AKA invoice financing – wherein they gave clients cash upfront for the right to receive an invoiced amount from the client’s client. It’s not rocket science. It’s not new. And it’s not sexy. However, the premise of ‘well, AI will tell us the likelihood of getting paid and therefore we can lend more at lower rates’ turned out to be hogwash. Even ignoring the fact that insiders have come forward saying that this innovative AI was little more than ‘spreadsheets or other basic programs’ there is little in the way of reasonable evidence to suggest that an algorithm or collection of platforms, or whatever you want to use, can replace human judgment to the extent that Greensill thought it could.
For example, no AI was going to tell Greensill that their largest client Sanjeev Gupta was going to allegedly use some potentially fraudulent invoices to garner cash prior to the alleged fraud occurring. Why? Because the AI can’t recognize a pattern as it occurs without having seen it previously. In a unique situation, there is no pattern to discern. It just is what it is. The sun rises and sets every day like clockwork yet people and firms accused of doing what Mr. Gupta and his network of firms are accused of are not celestial bodies. Rather, they are more like the true scale of a 100-year flood when all you have is 23 years’ worth of data.
This brings us back to Lemonade. Again, assuming that the firm is being honest when they say that they do not use “non-verbal cues” – i.e. the shape of a dude’s skull – to decline claims, there is the question of why the person running the Twitter account believed that AI has such capabilities. Assumably, they did not merely pull the idea out of their backside. We don’t just assume these sorts of things. Sure, a racist person will hold stereotypes about this or that group but they don’t tend to think that AI will discern such things. My guess is that there are folks in the AI/FinTech space who actually believe that a video analyzing algorithm can prove that someone is committing insurance fraud based on these magical ‘non-verbal cues.’ And, that there are enough people in the space thinking it for a lowly social media jockey to have absorbed the line of reasoning via osmosis.
Let me be clear though. I am not insinuating that the AI/FinTech folks are racist. Instead, I think that more than a handful of them have merely fallen into the Charles Murray Trap. They are trying to tackle a complex problem with the wrong tool. And this incorrect tool is leading them to make a hell of a lot of assumptions and claims that they can neither justify nor prove to the standards we hold human agents accountable to. The assumption here being that the human mind does little more than calculation and pattern recognition. In short, there’s much more to the human mind than what is taught in Comp Sci departments.
A more realistic paradigm – coming out of the humanities – would be to view the human mind, and subsequent intelligence, as the mixture of “memory, understanding, and will” as opposed to mere ones and zeros zipping about.4 Unless AI can match these aspects – which I would argue is impossible – then AI always differs from human intelligence. Even if the form of intelligence that AI possesses works better in many situations, there will always be those in which human intelligence is better suited. The problem is knowing when and where the two forms are best suited.
Alright, moving on to the second proposition we spoke of way back at the top. This proposition has less to do with technology and more to do with the business of the AI/FinTech world. Basically every single god damn FinTech firm has the same value proposition and it goes something like this:
“Currently industry X relies on bureaucratic processes dependent on human judgement and error. Every year industry X spends a very large sum of money paying the humans involved in the process. By automating the processes via our AI platform we cut out the overhead and improve margins which in turn allows us to undercut the existing players and yada yada winner takes all.”
For example, take personal loans startup Affirm. Their business model is that they offer loans to would-be consumers in order to purchase products through partner retailers. The consumers then pay off the loan over the course of a couple of months or a year. The idea being that Affirm loans the money upfront, pays the retailer, and then charges an interest rate on loans ranging from a promotional 0% to more credit card-like rates of 10-30%. The firm’s CEO claims that this is a better situation than a credit card because Affirm tells the consumer what the cost is for them in terms of dollars as opposed to compounding percentages. These loans are made based on the recommendation of “Affirm’s proprietary credit model.”5 Thus, instead of using a credit card or getting a loan from a bank [for larger purchases], a consumer can just use Affirm.
While Affirm is what it is, and I can’t really see any super sketchy aspects of it when compared to the universe of short-term lenders. Its entire premise is based on the arbitraging of technology costs and human labor. Whereas I might have to call my local bank and talk to a real person in order to get a new credit card, higher limit, or even a loan – not that they would give me one for purchasing a Peloton – Affirm will just run their ‘proprietary credit model’ and tell me whether I’m good or not. Rather than walking through the 5 C’s of Credit, Affirm runs a credit check, a couple of add-ons [thus making it ‘proprietary’], and then that’s it. Is Affirm likely to make mistakes? It does not matter according to the business case because the delta between technology costs and human labor is large enough to account for any added losses due to sub-par QA/QC.
At the end of the day, companies like Affirm find opportunities to capitalize on that delta. They aren’t reinventing the wheel. In fact, Affirm is not in a sexy business. Yea, they have a nice-looking website, but when you peel back the marketing it’s obvious that they’re making what are likely sub-prime loans. Why? Because who needs their product? People who can’t afford to purchase goods upfront. And who are those people? People who do not have money lying around with which to make discretionary purchases. Or in layman’s terms, poor people. Sorry, but if you need Affirm to purchase a Peloton then your fat ass does not need a Peloton. What you need is some restraint. Run outside as I do for the low, low cost of free. God made you a gym, go use it. Seriously, any business that requires its customers to pay up to 30% interest when the S&P 500 returns about 6% is not a high street business.
But is that a problem? I don’t think so. The company is fulfilling a need but let’s not pretend that it’s an innovation shop – it’s an arbitrage firm. Nothing more. Nothing less. The same goes for Lemonade. Sure, they might claim that their AI is a razzle-dazzle platform that hyperventilates the Rubik’s Cube but their actual value proposition is that they might be able to automate away the labor costs of the insurance industry while limiting the cost of ensuing mistakes caused by the misuse of AI. Strip away the buzzwords, think pieces, and bullshit LinkedIn posts and that’s all you got.
We’re at just over 2500 words at this point so how about we land this plane?
Lemonade, they done goofed on their social media account. But they apologized and probably took the intern out back and put them out of their misery. But the initial tweet thread laid bare an uncomfortable fact about the AI/FinTech crowd. At their core, the various firms in the space overwhelmingly assume that human behavior is nothing more than 1’s and 0’s while at the same time putting that assumption to work in the form of technology-labor arbitrage. If they’re right, good for them. If they’re wrong, oh well. In the meantime, spare me the evangelism.
Richard Herrnstein and Charles Murray, The Bell Curve: Intelligence and Class Structure in American Life, (New York: The Free Press, 1994), pp.276-277.
Gregory Clark, A Farewell to Alms: A Brief Economic History of the World, (Princeton: Princeton University Press, 2009), p.259.
I’m being generous here.
Augustine of Hippo, The Trinity, 2nd ed., trans. Edmund Hill, (Hyde Park: New City Press, 2012), X:18.
https://www.inc.com/sonya-mann/max-levchin-affirm-misunderstood.html#:~:text=Affirm%20lends%20money%20to%20make,but%20less%20risky%20for%20customers.&text=To%20its%20critics%2C%20though%2C%20Affirm,things%20they%20can't%20afford.