Zillow has been the punching bag of many of late, and perhaps its simply backlash from the large amount of free publicity they got after launch. It generated a lot of buzz to the point where bloggers were so sick of talking about them that they swore to abstinence (from Zillow) for a while.

People have had a chance to use the site for a while now and are beginning to realize how inconsistent the results are. Very accurate in some markets, very inaccurate in other markets and a lot of markets in between so you don’t quite know when and where the results are accurate. The results can be like snowflakes: pretty to look at, but the results for each search can provide a random result. They publish the ratings but I am sure the consumer doesn’t refer to them very often. And seriously, 52% accuracy in NYC means the results should not be published, should it? I have commented on this before since I am in the valuation business.

Zillow is such a neat concept on paper, but in the end, all data goes into a black box and spits out a number. They over promised and should have moved slowly over the country as they felt comfortable in each market. The logarithms that create the “Zestimates” are proprietary and perhaps thats what drives the suspicion, combined with pretty significant inaccuracy ratings.

Despite what I think are best efforts on Zillow’s part, however, I suspect we are seeing the first of several attempts to shut them down or change their model, refuting the argument that since they disclose their inaccuracy, they can continue business as usual. Because of their high visibility, its likely to become a public relations coup for anyone that goes after them.

Damon Darlin writes about the complaint filed by the Community Reinvestment Coalition in his article A Home Valuation Web Site Is Accused of Discrimination.

In a letter sent by the National Community Reinvestment Coalition to the Federal Trade Commission last Thursday, the group asserted that Zillow’s Web site misrepresented home values and placed residents in low-income neighborhoods “more at risk for discriminatory and predatory lending practices.”

Here’s the complaint [pdf]

I don’t really follow CRC’s logic for taking this action since all markets are subject to various degrees of inaccuracy but its going to be interesting to see what the FTC does. I have seen this from my own personal experiences from family members in different parts of the country who have used the site.

Complaints are filed against appraisers for inaccuracy too, but unless there is negligience or fraud commited, its simply an opinion. I know first hand that data collecting in my urban market is often very challenging, and more difficult than in many surrounding suburban markets. Besides record keeping issues, one overlooked reason for the inaccuracy of urban areas is the challenge of valuation in a “vertical” market. Condos and co-ops stacked on top of each other is very difficult to automate.

Whats also really interesting, is the fact that they have been marketing to consumers in order to bypass the real estate professional, but recently the orientation has changed to try to be broker-friendly. This attempt to placate the brokerage community backfired recently and the pr spokesman for the NYT article contradicted themselves by saying:

[UPDATE]A Zillow spokeswoman, Amy Bohutinsky, said the site’s valuations, which it calls Zestimates, were intended for consumers and had never been marketed to real estate professionals. The company sees the tool as a way to empower consumers who in the past would have to rely on a real estate agent to make an estimate based on the sales of comparable homes in a neighborhood.

Is Zillow’s market ignorance in certain areas bliss? Thats up to the FTC, apparently.

Tags: , ,


21 Responses to “Zillow Gets FTC’d Despite Randomness Of Their Inaccuracy”

  1. Lex says:

    I will guarantee that if Zillow was erring on the high side, you wouldn’t hear a peep out of anyone.

  2. Hi Jonathon, it’s David G from Zillow.com.

    I need to clarify that the NYT snippet at the end of this post is not a quote and that no-one representing Zillow.com said that — if you read that paragraph again in context, you should note that the reporter was merely drawing their own conclusions.

    Zestimates are not competitive to CMA’s and appraisals. Nearly all of the consumers we serve rely on real estate professionals and are often looking to meet them when they use Zillow.com. Our industry partners are critical to our site’s success (including its ad-driven revenue).

    I found this NYT article most interesting in that the NCRC apparently refused to supply evidence to support these unsubstantiated claims or to disclose any details regarding the data and methods used in their widely quoted and horribly inaccurate analysis of Zillow.com accuracy.

    Thanks for blogging.

    David

  3. Lex -

    It’s David again from Zillow – I just saw your comment.

    As the guy that listens to the “peeps”, I can let you know that we actually do hear a few peeps when Zestimates are (too?) high — usually from buyers and their agents. Last time I analyzed this feedback, the ratio of feedback reporting Zestimates too-high to that reporting too-low was about 7:3.

    That said, most people “get it” and we see significantly less feedback about Zestimates now that owners can publish their estimate on Zillow.

    David

  4. Jonathan J. Miller says:

    Thanks David – I added the beginning of the paragraph – While I see what you are saying, the spokesman is saying it is a tool for the consumer in the part I omitted, which seems to be consistent with the way Zillow was originally promoted – as a tool for consumers. I see your point about inferring that real estate agents are cut out of the process.

    However, I wasn’t really referring to that nuance. It was more that Zillow was consumer orientated and yet has been courting the Realtor community as of late because it needs them to succeed, I assume for advertising. I think the idea was originally that consumers would be armed with more information and this would be brought to the broker when marketing a home for sale. It sort of evolved into an idea of the brokers being replaced (which is silly because Zillow is not selling houses).

    Anyway you look at it right now, its a pr nightmare.

  5. Jose says:

    Thank god its a Zestimate. Thats why I tell my clients not to look on Zillow. Zillow has gotten inaccurate. House we sold for 650k is zesimated for 790+k.

  6. BubbleAnalyst says:

    Some random comments …

    I appreciate Zillow as a resource because I understand what they are trying to achieve and the limitations of the system. Let’s face it, today they are the best source for this information.

    The NCRC Complaint can be summarized as: the general public is bad at math so sites offering a statistical analysis of the housing market should be banned.

    The Complaint mentions “true market value” and I hear that phrase a lot when it come to criticisms of Zillow. As I see it, the housing market is driven by (i) parties who have a lack of information, are often driven by emotion, and receive a flurry of disinformation, and (ii) interested middlemen (and women). Realtors are not striving to run a system designed to close a home at a true value based on perfect information. Just a thought, but (admitting statistical error in Zillow’s estimates) is Zillow woefully inaccurate, or just exposing the randomness of an illogical real estate market?

  7. Jonathon -

    Yep, not having much fun right now.

    Thanks for the addition; I think I’m extra-sensitive to that nuance. Which I guess is your point :-) Zestimates were built for consumers. And yes, Zillow’s a great place for most brokerages (and soon, most individual Realtors) to advertise, but Zillow’s partnership with (and value to) the Realtor community is more important than that.

    Many Realtors are now helping sellers update their home’s facts (where necessary) and publish revised estimates as part of their marketing strategies. Makes sense — buyers are on Zillow. And Zestimates are now a common conversation starter between homeowners and their Realtors. I’ve even heard from owners whose Zestimate reminded them to get an appraisal. We love the opportunity to get in front of a large Realtor community and answer their questions about Zillow. It makes sense (to us) to put effort into equipping Realtors to help their clients understand Zillow, demonstrate their expertise and turn those conversation into $$$’s.

    This partnership (with Realtors) is also a primary reason that I personally blog — I know I’m reaching Realtors with useful information about Zillow that they in turn can pass on to their clients.

    Jose -

    That does sound odd. I’d appreciate it if you could e-mail me with the address [davidg at z].

  8. Jonathan J. Miller says:

    Thanks David – appreciate your insight and candor. The disconnect between the brokerage community and your service, measured by the CAR conference specifically has always puzzled me since you are not providing a competing brokerage service, and helping the consumer is what drives the Internet.

  9. UrbanDigs says:

    I have to be honest when reading all this but I think Zillow is just a fluff site to be used for entertainment purposes for anyone trying to find out what their home is worth via an online ‘authority’; if I can even say that.

    Everyone should know that these zestimates are not fully accurate and therefore not really useful at all. I mean, who would buy or sell a home at a certain price because of a zestimate they saw on Zillow?

    I respect what Zillow is trying to do but they should have seen these troubles from a mile away when they first devised their business model offering price estimates for 67 million homes using an algorithm.

    So many factors contribute to the price of a home that no set algortithm could ever be useful in the real world; at least that is my opinion.

    In the end, the consumer should be able to figure out that this service is nothing more than internet fluff and the data that is provided be used as such.

  10. There is a significant core of Realtors® that is feeling, like David G expresses for Zillow lately, ‘extra-sensitive’. So they react … badly, in ways that may be short-sighted (Ex A will be the quotes attributed to Alan Dalton about being ‘offended’, but there are a few alphabets worth of other examples out there.) Zillow contributes to the extra-sensitivity for many Realtors® by suggesting that agents are overpaid for what we do (can’t find the quote right now).

    I don’t exactly like that, either, but I am less bothered by what else bothers a lot of Realtors®: the evolution of information about transactions away from only the people in the club. That level of paranoia about Zillow seems to be Canute-like, since the tides favor “information wanting to be free”.

    When Zillow takes “publicly available” information that used to be readily available only to those in The Club, then plays with that information through proprietary algorithms Zillow steps into a huge information gap that members of The Club prefer to leave alone. So paranoia sets in.

    I think the NCRC complaint is ridiculous (what part of “estimate” don’t they understand?) but the FTC may be lame-brained enough to run with it.

    More Realtor® paranoia claims to be based on two “principles”: (1) you are not really paranoid if they are out to get you, and (2) Zillow has too much money (in theory) to be just in it for advertising revenue, so they must (by application of possibly paranoiac process of elimination) be heading towards a Lead Aggregation system. Realtors® HATE lead aggregators, as all they seem to do is use Realtor® listing data to collect names and contact info of consumers that hey then sell to Realtors®. Or so the theory goes.

    Myself, I wonder about those algorithms and how there can be any credibility in a market in which there is a 52/48% chance of getting a Zestimate that is within 10% of sale price (the New York-Northern New Jersey-Long Island, NY-NJ-CT-PA CMSA “market”), but the market place will deal with that Zillow’s Bright People don’t fix it first.

    THX for jumping into the fray David G.

  11. skep-tic says:

    I find it pretty funny that realtors and sellors are crying about zillow’s inaccurancy while their homes sit on the market with zero interest for months due to wildly inflated asking prices.

    can realtors claim better than 50% accuracy? judging by the number of price reductions and relistings, the answer seems to be no.

    what is the scientific pricing method of realtors? looking at recent comps and then basically guessing a price! yes, this is way more scientific and reliable than zillow

  12. Susan says:

    It seems to me that Zillow won’t be able to please anybody. The housing markets in major population areas are going through a hard transition, and in some places home prices are trending down rapidly. However good Zillow’s number crunching is, its estimates will lag what is happening out on the street in real time – they will come in too high.

    Had Zillow been around 2-3 years ago would the opposite been true – Zillow valuations coming in too low because the markets were on warp speed up?

  13. UrbanDigs says:

    11 – Fine, so your saying Zillow is basically an ‘asking price’ site that tells you what your property should start out asking on the open market?

    Or should the price go beyond past comps where the broker can judge the target audience of that specific price point in their local market and combine that with renovations, natural sunlight, views, noise level, exposures offered, difference in monthly costs, building policy’s, financing allowed, school districts, and others that all influence both the value of the property and the size of the buyer pool that it is marketed to?

    You know your crying:

    “I find it pretty funny that realtors and sellors are crying about zillow’s inaccurancy while their homes sit on the market with zero interest for months due to wildly inflated asking prices.”

    ..when that doesnt even seem to be related to this conversation? I have an exclusive that is 3 weeks on the market that I think is slightly overpriced for the sole reason that my clients wanted to ‘test the market’ for their property before allowing me to reduce to attract more activity. Is this strategy my fault? Not at all. In fact I bet if you were selling your home you would skip a broker and NOT even offer the buyer the saved money in brokers commission via your asking price. You would probably price your property just as high as others on the market and go through the same process.

    Its human nature. We are greedy. So, we learn the hard way and brokers get stuck with other people’s homes that only get reduced if there is a time pressure. In the end, the market dictates the homes true value. We as brokers, can just be a guide and market the property as best as possible with the client’s best interests in mind.

    I’m so sick of everyone bashing brokers. Its pathetic. I know a lot of good hard working brokers whose clients thank them at the end of a deal for getting them a price that exceeded what ‘in their gut’ they would get. Sure, everyone prices higher and tests the market. So what. What matters is if the home can carry the weight of the higher asking price. Will the incredible light and views prove to actually be worth more to someone looking for just that type of property. Will the waterfront home get that extra 100K, because a perfect buyer happens to be at the right place at the right time to place a bid?

    Zillow’s data is not threatening to me as an agent in any way. I am just offering an opinion. Do I think Zillow will steal my buyers? NO. Do I think Zillow will steal my sellers? NO WAY. So, why should I be afraid of them? Because my client goes onto the site and asks why zillow.com says their house is worth $50K more than what I think?

    I know the local market. I know a ballpark # that a buyer will pay. Almost always will the seller originally price much higher than this number. And almost always they hit a point during the sales process that they come down to reality, and cut the price to move the property. Especially is there is a pressure (time).

    Good blogging.

  14. This analogy might not be spot on but I think it’s close. I use Zillow like I use Mapquest — as a reference point. It’s a possible route to my destination but not necessarily the fasest or most reliable.

  15. jf.sellsius says:

    I agree with Mr.Digs on virtually all his points. Well said.

    The only risk to Z is the nature of the disclaimer, since even fortune teller sites are legal.

    Racial & economic bias claims are losers unless the secret sauce is demographics, which Z denies.

    And I agree with you, Jonathan, the threat is not to brokers, it’s to another profession :)

  16. skep-tic says:

    in other markets, assets are priced all of the time using some kind of rigorous analysis. why is the idea that the same can be done in real estate so threatening? at the end of the day, pricing for any kind of unique asset is really a judgment call, but a judgment based on ascertainable facts seems prefereable to a broker’s gut feeling.

    maybe a broker’s gut was good enough back when apts were selling for $75k. But when you start doing deals in the millions routinely, you’ve got to come out of the stone age

  17. It kind of feels like the broker doth protest too much. If they’re not feeling threatened, why are they lending such credibility to zillow by denouncing it so loudly? Certainly, I’ve worked with brokers and agents who have operated with complete integrity, added value to my buying or selling effort, and truly earned the considerable commission they were paid. I’ve also had a house listed for four months with a broker and ended up selling it myself after advertising for a week, and coming out $22k ahead of what I would have received had I gotten the broker’s listing price. As with many industries, there are good agents and there are bad ones. There is plenty of room for bashing, and the good brokers keep enhancing their wealth at the expense of the bad ones, as it should be. Zillow can provide a tangible discussion point for buyers/sellers and their agents or brokers. Used properly, it will aid the agent in a discussion of the importance of differentiation within a market. Caveat emptor for those dealing with brokers who are dismissive of independent information, whether it lands within 10% of the selling price 50% of the time or 90%.

  18. [...] Joe started to feel the pressure and wanted to write a post and comment on a few blogs. Eventually he caved. He left bloglines and went straight to many of our favorite blogs directly. Say it ain’t so Joe. Although difficult, I still held the line as my partner went to the darkside. Not only did he read the comments, but he actually posted one in the Matrix. Oh no. Would agent Smith come after us now? Joe said, “Don’t worry. I’ll be back in a jiffy.” He found a great post by Jonathan Miller of the Matrix Real Estate Blog. Deja vu? Maybe. He thought that some of the answers we were looking for could be found here. Was Miller the almighty oracle? He decided to post a comment to find out. We waited for an answer but the oracle did not provide one. [...]

  19. jf.sellsius says:

    To Kevin

    The threat to brokers is only a perception, while to appraisers it’s a reality. Having someone give you a value, whether a computer or an appraiser, is just one component necessary to complete a real estate transaction. The value communicator does not sell or buy a house. So instead of waiting for the appraiser to clear the file, Zillow.com will do it. Just MHO.

  20. Evan Gould says:

    Zillow is another tool for consumers to use when considering to buy or sell a home. Personally, I think it’s great to give more information to homebuyers/sellers. It creates a dialogue that can only help brokers and consumers alike.

  21. It think that Kevin got it about right. There are brokers who earn their keep and those who do not. I just wish a zestimate could start a healthy discussion about values, but at a 52% accuracy rate, it’s not healthy its disfigured; that’s the real problem. It’s great to give everyone access to real data, you get better, more educated buyers and sellers that way. It’s another to give them access to information that is about as accurate as a wild guess.

    It’s hard to really understand a property’s value just by pinging the city records. Knowing what’s special about a home is something that an information retrieval system can’t evaluate. Missing out on a home that you really want because of inaccurate data, or pricing a home for sale wrong, is bound to have a severe cost at the end of the day.