References to the JDPower Customer Satisfaction Index prompted me to go and read the report on the Whatcar website, at www.whatcar.com/News_SpecialReport.asp?NA_ID=20780...3
It's intersting reading, because the data seems to tell a very different story to the press coverage.
Here are some of the observations I made about the data:
1/ All cars score very well
* The highest score is only 88.4% (for the Honda Jazz), so even the best model still has quite a few failings.
* The lowest score is 69.3% (for the Fiat Bravo), so even the worst result is far from abysmal.
When I was at university, an exam mark of over 70% was graded as a "first". If we apply that grading here, even the worst result only narrowly misses the top grade. Second-last place goes to the Renault Espace on 72.8%, which is a clear first-class ranking.
The best cars get very similar results
The top ten cars have ratings so close that they are probably within the margin of error.
First placed car, gets 88.4%; tenth-placed gets 84.1%. Considering that these are figures based on subjective measurements, I wouldn't let that small a difference influence my purchasing decision.
3/ The top 50% of cars are very closely bunched
The first placed Honda Jazz gets 88.4%, the joint 60th-placed Vauxhall Omega 80.0%. That's probably outside the margin of error, but it's not a huge difference: the Omega is probably less satisfying, but not by much very much. If the ratings are accurate, the difference is unlikely to spoil your day.
(Conversely, if the Omega really is a bad car, then the ratings system does not reflect that)
4/ Whatcar's star-system may be misleading
Take superminis. First placed Honda Jazz on 88.4% gets five stars, as do the second-and third-placed cars on 85.x%, but the four-star Hyundai Amica gets a very close 81.9%. The two-star Fiat Punto is very close to the Amica, on 77.3%, while the in-between three-star Citroen C3 gets 79.4%.
With scores that close, the Amica, C3 and Punto really deserve to be in the same bracket ... but instead, the Amica appears to do twice as well as the Punto. I wouldn't use this star-system as a guide.
Conclusion
I don't know enough about how JD Power builds its ratings to really judge how useful the scores are. But if we assume that they are a reasonable measure of something, then far too much attention is paid to them -- because they basically paint a picture of manufacturers achieving broadly similar outcomes.
|
I like some of the judging and scoring criteria. Things like: People say they dont like the way the car looks, or not enough space, or the performance is not good enough.
Then why did they buy the car in the first place?????
|
|
|
I was looking at the results of the JD Power survey some years ago and I was struck by the paucity of responses for some of the vehicles.
I was interested particularly in the Omega, which was not highly rated. I don't remember the exact figures, but there were very few who responded for the Omega, whereas the responses for those that were highly rated were enormous in comparison. I'm not a statistician, but I would have thought that the Omega responses, and no doubt others, should have been regarded as insignificant, and therefore invalid.
|
I was looking at the results of the JD Power survey some years ago and I was struck by the paucity of responses for some of the vehicles.
Whatcar says that:
"Any car which generates fewer than 50 complete surveys is rejected; such a small number isn?t enough to guarantee a representative sample, and might give a misleading result."
I dunno how respondents are selected, and self-selecting surveys can be very misleading ... but assuming the selection is okay, a lower threshold of 50 looks fine to me.
|
Excellent post, NoWheels.
Confirms what I've often thought about so many of the scare stories which emerge on this site, too. Just because a collection of people seem to have a car with a similar ailment, that does not mean the car itself is therefore poor and to be avoided.
Also, nearly all cars, these days, are exceptionally reliable. If I think about the number of journeys I have made in the last 20 years, the percentage resulting in a breakdown is miniscule. Its inevitable, however, that servicing will result in expenditure because parts wear out. That's always painful but to be expected.
Splodgeface
|
|
|
|
What a sensibly sceptical attitude to statistics No Wheels.
Although the repsondents are "invited" I don't think they tell us what the response rates are - the lower the higher the self-selection and the less useful the result?
They also say they give most weight to reliability and quality. The latter is clearly very subjective as reported by owners and related to expectations. Experience of listening to car owners talking about their cars suggests that the seemingly more objective area of reliability is also likely to be distorted in the same way by reliance on owner reports - what one owner reports as a problem may be regarded as perfectly acceptable by another. I once asked a Jaguar owner (it was a long time ago) whether his one year old car had been OK, as the previous one had been a real dog. His answer was yes, it had been fine. Then he said, without irony, that the gearbox and rear axle had both been replaced but that had only been due to a "bad batch" and didn't reflect on the car which he regarded as absolutely top notch - what would he have reported? If you rely on people's memories for fault reporting you will generally get under-reporting anyway, and the more pleased they are with the car the more likely they might be to forget.
Having said all that, if you invert the "satisfaction" ratings then they imply that the Omega leaves over 50% more owners unsatisfied than the Jazz (20% vs. 11.6%)- a clearer margin when expressed in those terms than saying only 10% more Jazz owners are satisfied than Omega owners (88% vs 80%).
|
|
|
When I was at university, an exam mark of over 70% was graded as a "first". If we apply that grading here, even the worst result only narrowly misses the top grade. Second-last place goes to the Renault Espace on 72.8%, which is a clear first-class ranking.
Don't believe all you were told at Uni! I bet the JD Power marking is more reliable...
I think the truth is not that all cars are doing well, but that many cars are quite similar in what they offer to the consumer in their sector. An unreliable car (used to be only 30% of total mark) can do OK in JD if it offers something in its design you can't get anywhere else for the same money.
Same for What Car, their ratings reflect desirability, not overall quality. For example the Tigra used to get 4 stars as it was the only car in its sector and by the end of its life their was more competition from cars like the Puma, so it dropped to 2 stars. It was still the same under-developed little mess.
|
|
I take all these survays with pinch of salt, I think they are largely meaningless
|
I recall the JDP surveys of some 3-4 years ago showing Subaru as the most reliable make. Now we get the Reliability Index showing Subaru as the second or third most expensive car for warranty repairs. I know Subaur suggest thsi is due to the large volume of grey imports and the use of a low octane fuel (most of the repairs are due to engine blowups apparently).
So there must be some question over either the period of the JDP survey or its legitimacy concerning low volume cars - which Subaru were 3-4 years ago.
I suggest you cannot look at any one year study in isolation but over a number of years: which tend to show trends . For example, the fact that Toyota tend to be consistently 1 or 2 in JDP surveys is no coincidence.
But a survey showing a manufacturer has suddely shot for example from 100 last year to 10 this year means nothing if the next year they fall to 50.
I suggest individual years should be ignored: just look at the trends...
madf
|
Damn lies and statistics.
'If any car generates less than 50 entries - it's not included"
So 51 entries is deemed as being representative of the model of car - I don't think so!
I'd accept the results if they polled 1000 registered owners at random - far more representative.
Take a look at this forum and you'll notice that most views are polarised towards either love it or hate it.
I suspect this trend impacts the JD Power survey as I suspect most contributors aren't dissimilar to the people on this forum.
|
|
I found the Auto Express survey more revealing (albeit my response is very subjective). The placings for the individual categories are given, and you can look at the overall picture.
|
My take on JDP surveys is 'mildly interesting but statistical tosh'. The sample isn't randomised and as a result the respondents are probably biased. And some of the things rated are incredibly subjective - it's quite possible that other people would think in a totally different way.
Those likely to respond are likely to be a) over the moon with their car (not many?) or b) disgruntled in some way. I remember in one JDP survey that the VW Sharan, Ford Galaxy and Seat Alhambra had wildly different results (and not obviously related to dealer performance either). Yet these are the same cars from the same factory!
Draw your own conclusions. Most cars are pretty good. Some - mainly Japanese - are very good. The luxury brands may not be all they're cracked up to be. And there are some dogs. But a person's view on a car is very much influenced by expectations...
|
|
It is a bit confusing. If for example I would be considering to purchase a Toyota, but would still be deciding which model to buy, I may consult the JD power survey. It suggests that the Yaris is better than the Avensis and corrola. However, the problem with this is that the confounding variable is that a Yaris driver would do a lot less miles than an avensis owner, hence would have a better opinion of the car as there would be less evidence of wear and less failures. In reality, the avensis and corrola should be better in terms of durability, reliability etc. The same can be said for the Honda Jazz, which is used less and hence would be rated as a better car.
|
|
|
|
|
I seem to remember that they same car has been entered in the past at two simultaneous places in the table, as noted by HJ in the car by car. Did the V70 and S70 not achieve significantly different scores? I cant beleive this an be right.
|
I'm going to sound a dissenting voice here.
Imperfect as it may be, the JD Power index is the best monitor consumers have got at the moment, and the manner in which car manufacturers and retailers responded to the arrival of the survey in the UK when Top Gear first brought it over spoke volumes about an industry that would be a lot happier if people didn't know about annoying little details like customer satisfaction.
It may not be perfect, but it isn't crude and we should encourage anything that puts pressure on manufacturers and their dealer network to provide a better service.
I had the great good fortune to meet a guy called Roger Penske late last year. If you don't know Penske through motorsport, he heads the United Auto Group, one of the USA's biggest car retailing companies. It's now very active in Europe, having bought the Sytner business a couple of years back.
He was above cheap shots about British business standards, but he did make it very clear that as far as he was concerned, the UK motor retailing network was very weak on 'mandating customer satisfaction' - i.e., making a serious, collective effort to find out what people really thought about the cars they bought and the service they got.
JD Power's index is one attempt to do something about that, and Penske for one was entirely comfortable with it. Question the methodology by all means, but don't pour cold water on the concept - that's exactly what the car makers want.
|
JD Power is an objective survey of consumers. The statistics appear sound to me - they are using large enough samples to produce statistically significant results. If the samples for a particular car are small then they exclude it. This reduces the risk of errors. The actual % values are less important that No Wheels and others have said. Just because everyone gets high scores doesn't mean that the lower scores in that range are are really not low. In postgrad medical exams most doctors score very high scores but only a certain number are allowed to pass e.g. 40%. The end result is that doctors scoring 80% in the exam can still fail becasue their cohort are scoring much higher. Similarly I recall in my Further Maths A level class at school people scoring 20% were still passing the exam. The reason was because the exam was so tough that 50% was enough for an A grade.
The JD Power survey is likely to be more representative of our views than the various awards that cars are given such as the What Car awards. You could argue that the reason the standard of cars have all improved is because of the JD Power survey. Poor manufacturers had to improve otherwise they would be shamed in the annual survey. I recall that it was because the survey was so influential in the USA that Top Gear asked them to survey UK cars as well in the early 90s. Its main fault appears to be the time lag of a year or two between survey year and report which means many of the reports are for cars that may have already been updated.
|
|
I've always found the Which? car buying guide to be pretty good. The certainly seem to accurately identify common failure points on most models.
|
|
|
Hull4000,
"JD Power is an objective survey of consumers."
Whilst I agree with most of the points you make I am not sure I would call such a survey 'objective'. It is after all asking owners to give their opinion by awarding a score. We could own identical cars, with identical history, and the grades we awarded could vary widely.
C
|
|
|
|
|
I have very little faith in any survey dependant on the subjective opinions of the owner. It is difficult for many to admit they have made a mistake and bought a poor car and to do so publicly only serves to drive down the resale value. There is also an extraordinary degree of loyalty toward a manufacturer; perusal of the Backroom will demonstrate how reliable Italian and French cars are; despite statistics that indicate they are the least reliable.
A second point is that this particular survey was carried out on cars between 1 and 2 years old. Thus the majority of faults would have been rectified FOC under warranty and doesn't hurt - as in the story Manatee relates about his Jaguar owning friend. It makes far more of an impact if you have to pay out a few hundred pounds.
Data from ADAC(Germany's AA/RAC) and fleet managers will give a better indication of reliability and costs but not driver appeal.
|
|
It's a pretty reliable survey of HOW OWNERS FEEL about their cars. That isn't a bad yardstick.
|
|
The survey is objective but contains the subjective views of car owners. I'm not sure how else you can carry out a satisfaction survey.
|
The survey is objective but contains the subjective views of car owners. I'm not sure how else you can carry out a satisfaction survey.
There was a newspaper 'quiz' some while ago called "how does your husband rate" (or something like that) that my wife completed. We both knew that it was only a sense of loyalty that gave me a reasonable score. I am under no illusion that there are far better, and younger, 'models' available!!
So perhaps it was objective after all!
|
There was a newspaper 'quiz' some while ago called "how does your husband rate" (or something like that) that my wife completed. We both knew that it was only a sense of loyalty that gave me a reasonable score.
If it was JDPower you'd have got a first-class degree, along with everyone else :)
|
|
|
|
I have absolutely zero faith in these surveys becuase as said before, it's all based on expectation.
Consider two owners, who are going to rate the same car - a Vauxhall Vectra 2.0 DTi LS.
The first owners previous car was an old G reg Cavalier. The Vectra is his first experience of a brand new car, infact, his first experience of a car less than 10 years old.
The second owner has recently had to buy one after opting out of the company car scheme due to budget cuts. He previously had a company supplied BMW 525i Sport.
Now, Owner 1 is going to think his Vectra is the best thing since sliced bread. Compared to his old car, it simply is absolutely fantastic. Yes, it's been back to the dealer 5 times, but thats nothing, his Cavalier got repaired once a week, and this time it doesnt cost him money. He also think it's fantastic to drive compared to his old banger. He rates the car very, very highly indeed. And as the dealer does everything free under warranty, he gives them a lot of praise.
Now, owner 2. He loathes this car. It's not a patch on his BMW, and doesn't he know it? He criticises the build quality, the performance, the refinement, he doesnt like the car at all, but has to put up with it becuase it's all his car allowance extended to. His car had only been back to the dealer twice, less than Owner 1, but both times he found the service not as good as that of BMW. Consequently, he rates the car quite poorly, as well as the dealer service.
Same car, same dealer, two wildly different expectations and consequently two wildly different sets of data.
This is perhaps best illustrated by the MG/Rover results in the last big survey I read. I read with amusement that owners of the Rover 45 thought their dealer was worth only 2 stars, yet MG ZS owners gave the dealers 4 stars. Rover 75 owners thought their dealers deserved 5 stars, but the MG ZT owners didn't agree..
Then we've got examples of a Porsche Boxster getting 4 stars for practicality, whereas a VW Golf got just 3 stars. Sorry, 2 seater rear engined sports car MORE practical than a 5 door hatchback?
So, liberal amounts of salt are needed.
Disclaimer:
Please note, the above examples are fictional and for illustrative purposes only. They bear no resemblance to persons, dealers or cars both living and dead and any likeness is purely co-incidental ;)
|
Michael,
Your points are all agreed. However there cannot be many owner 2 types(ex BMW driving, Vectra hating)in the JD Power Survey as all the cars scored so highly - 70-80% for those near the bottom.
Perhaps the fault with this type of survey is that only the owner 1 types respond to the survey and the owner 2 types can't be bothered - after all it is not compulsory to participate in the survey.
C
|
I don?t think anyone should take these surveys as absolute gospel, but as one or two people have said, they?re the best we?ve got until someone can come up with something much better.
Yes, there will be an element of expectation, but do people have low expectations when they buy a Lexus IS ? (year after year). Jaguar didn?t do all that well in this survey ? what expectations did their owners have? The Kia Sedona was near the bottom of the list, but has about the second-highest number of sales in its class, probably because it?s so cheap. Did all those purchasers have unduly high expectations, or have they simply found out that there?s a reason that it?s cheap?
And yes even the cars at the bottom of the list are scoring around 70%, but isn?t that a reflection of the fact that cars in general ARE more reliable than they used to be? And probably how manufacturers have pulled their socks up since the big song-and-dance when the first JD survey came out.
I try and look at all of JD Power, Auto Express, Which? surveys, and Parker?s and HJ?s ?known faults? lists, etc etc etc. Generally, one doesn?t see Lexus, Toyota, Mazda, Honda near the bottom of the ratings, nor Fiat, Alfa, Citroen, and certain Vauxhall models near the top.
That?s not to say that those cars are terribly bad these days, just that you may have more ?things?, with the car or the service, to have to put up with.
Based purely on the looks of the Alfa GT, I might be prepared to put up with a few ?things? to have one of those!
|
|
|
|
"..manufacturers achieving broadly similar outcomes"
Which is not altogether surprising, as they are mostly put together on similar production lines using machinery and robots from the same suppliers! There are some differences in quality control, particularly WRT outsourced components, and the Japanese are especially good at this, but most differences, I surmise, are to do with sample variation and owner treatment (not to mention treatment of owners by the franchises). In fact, looking at the variance in owner attitudes (we all know people who worship their cars and others who treat them as mere chattels), it's surprising that the results are as consistent as they are.
My chief reservation about JD Power is that they are looking at cars that are too new. Most cars should be able to withstand the first two or three years without too much difficulty, but how they behave 10 or 12 years on would be much more revealing, IMHO.
Excellent post, NW, I should add. You must be the most rational woman on the planet... :-)
|
Excellent post, NW, I should add. You must be the most rational woman on the planet... :-)
Thanks, JBW!
I can't resist rising to the bait about rationality, though.
When it comes to cars, we women are notoriously rational -- we want the things to be reliable, safe, comfortable, and to to do the job we actually use them for. We have a constant battle hauling men back down to earth from their fantasies :)
|
Fair enough, NW, although it was really meant to be a compliment. It's not often we get that thorough a dissection of a topic here from females, though - it was just difficult to make the point without sounding like an old git!
I have to admit that I do have a recurrent fantasy about a Porsche 928...
|
I have to admit that I do have a recurrent fantasy about a Porsche 928...
too much information......
|
|
|
|
