« Asymmetrical limits for the zone of equivalence (Chapter 21) | Main | Regression to the mean (Chapter 1 and 33) »

Dangers of overfitting (Chapter 38)

Many studies have purported to predict whether couples will divorce. But these are all done by fitting a complicated model with many independent variables, without validating the model with new data. Heyman and Selp demonstrate the need to validate models.

They used stepwise logistic regression to develop a model to predict divorce in one group of subjects. They had lots of variables (and interactions) so were able to find a model that worked quite well. When predicting a couple would divorce, the equation was correct 65% of the time (positive predictive value). When predicting that a couple would not divorce, the equation was correct 98% of the time (negative predicted value). Sounds useful.

They then applied that same model to a new group of subjects. Not surprisingly, the predictions don't work very well when used with new data.  When predicting a couple would divorce, the model was correct only 29% of the time. Not very impressive, considering that 33% of their subjects had divorced.

This makes an interesting data showing the dangers of overfitting, and the need to test models on new data sets.

 Heyman, R.E,  and Smith-Slep, A.M. The Hazards of Predicting Divorce Without Crossvalidation. J Marriage Fam (2001) vol. 63 (2) pp. 473-479

This same point is made, in plainer language, in an article in Slate.  

EmailEmail Article to Friend

References (7)

References allow you to track sources for this article, as well as articles that were written in response to this article.
  • Response
  • Response
    Response: Slimquick
  • Response
  • Response
    Response: Real Events
  • Response
    Response: Coach bags outlet
    Intuitive Biostatistics - Topics - Dangers of overfitting (Chapter 38)
  • Response
  • Response
    Intuitive Biostatistics - Topics - Dangers of overfitting (Chapter 38)

Reader Comments (7)

Tiffany and Co AccessoriesThe Samsung Galaxy S has an internal memory of one6 Gb out of which 2 Gb is kept for system storage.
Tiffany CuffLinks

September 9, 2011 | Unregistered CommenterTiffany and Co Charms Outlet

Couldnt be written any better. Reading this post reminds me of my old room mate! He always kept talking about this. I will forward this article to him. Pretty sure he will have a good read. Thanks for sharing!

September 20, 2011 | Unregistered Commentercar service in denver

It’s also ugg boots clearance ugg boots clearance uggs clearance sale utilized as a piece of boots ugg outlet appreciation of your ugg boots outlet employees. This way you’ll uggs clearance outlet boots ugg clearance be ugg ugg boots outlet sale boots outlet clearance able ugg boots clearance outlet to encourage ugg boots outlet kids ugg uggs outlet online clearance outlet them and produce ugg outlet online a conducive operating ugg outlet online atmosphere. This may further ugg boots outlet stores encourage his ugg boots outlet online fellow colleagues to perform better. ugg outlet stores Everybody loves to obtain recognition for their tough work and dedication. Corporate gifts might be considered to celebrate a vital event which includes a uggs outlet uggs clearance company’s silver jubilee, or major product launch. Offered online, is an extensive range of suppliers to suit any requirement you may have.

November 14, 2011 | Unregistered Commenterhemin

christmas decorationschristmas decorations
christmas ornamentschristmas ornaments
christmas lightschristmas lights
christmas treechristmas tree
ugly christmas sweaterugly christmas sweater
artificial christmas treeartificial christmas tree
business christmas cardsbusiness christmas cards
bulk christmas ornamentsbulk christmas ornaments
buble christmasbuble christmas
busch christmas stocking kitsbusch christmas stocking kits
busch gardedns christmasbusch gardedns christmas
bulk christmas lightsbulk christmas lights
burlap christmas stockingsburlap christmas stockings
bulk christmas giftsbulk christmas gifts
bulk christmas cardsbulk christmas cards
buble christmas albumbuble christmas album
vintage christmas tableclothvintage christmas tablecloth
vintage christmas treevintage christmas tree
vintage christmas tree toppervintage christmas tree topper
vintage christmas tree skirtvintage christmas tree skirt
vintage christmas tree pinvintage christmas tree pin
vintage christmas tree ornamentsvintage christmas tree ornaments
vintage christmas tagsvintage christmas tags
vintage christmas tree lightsvintage christmas tree lights
vintage christmas towelvintage christmas towel
outdoor christmas decorationsoutdoor christmas decorations
vintage christmas decorationsvintage christmas decorations
christmas dresschristmas dress
christmas disheschristmas dishes
girls christmas dressgirls christmas dress
christmas dvdchristmas dvd
christmas decorchristmas decor
christmas dinnerwarechristmas dinnerware
light outdoor christmas decorationslight outdoor christmas decorations
christmas movieschristmas movies
christmas movies in Movies & TVchristmas movies in Movies & TV
christmas movies in Musicchristmas movies in Music
christmas musicchristmas music
christmas movies on dvdchristmas movies on dvd
christmas music 2011christmas music 2011
vintage christmas tree ornamentsvintage christmas tree ornaments
vintage christmas tagsvintage christmas tags
vintage christmas tree lightsvintage christmas tree lights
vintage christmas towelvintage christmas towel
outdoor christmas decorationsoutdoor christmas decorations
vintage christmas decorationsvintage christmas decorations
christmas dresschristmas dress
christmas disheschristmas dishes
christmas crackerschristmas crackers
christmas cdchristmas cd
cheap christmas cardscheap christmas cards

December 21, 2011 | Unregistered Commenterdesigner handbags outlet

No problem. nip tuck on dvd
That was the first show I fell in love with.No Ordinary Family dvd
I was so sad to see it go off the air - it could have gone on forever. But yes, I do think some of the first eppy's female audience were persuaded just a tiny bit by that old Belthazor-lawyer hunk-of-McMahon!

January 7, 2012 | Unregistered CommenterThe Middle on dvd

Coach Outlet Stores. Why Coach Online Outlet Store are so attractive and Coach Factory Outlet Online so outstanding? Burberry Scarves Now let's start to see how many types Coach Online Store have. For the sort of Coach Store Online, it has shoulder bags, handbags, waist bags etc. Coach Outlet Store suitable for men and women, the style of Coach Store Online Outlet can embody the character of the men and women.
As for Coach Outlet Website designed for men and women, you can buy it in every exclusive store or some big malls. Burberry Especially for Coach Factory Outlet for women, we can say their products sit on the top luxury position in the world; Coach Outlet Online their design for women is so novelty and fantastic, Coach Store Online they always leading the fashion orientation. Coach Factory Store And they own the latest materials for Coach Factory Outlet Store . That is why they are so proud of their design and sales Coach Factory Outlet Online Store idea on their bags. Because they deserve this kind of honor. My Coach Outlet However, not only authentic Burberry Outlet Scarf popular but also replicas.
Designer Coach Factory Online Store are the exact copies of the genuine ones. Coach Store OnlineThey are exclusive as the original. Coach Outlets Online Every details like the stitching, logo, Burberry Handbags hardware and signature lining is cautiously mirrored those branded one. Coach Outlet Stores Online Different from the fake handbags which are made of interior materials; Coach Factory Outlet Store Online the replicas are crafted from high quality material instead.
There are various kinds of replica handbags of different qualities in the market. Coach Factory Outlet Online Store The good ones look more approximate to the genuine and cost more than those of poor qualities. Some of them also deserve high prices because they require much time and effort to finish.Coach Factory Online They are really ideal options for those people unaffordable for the real Coach Factory Outlet Store
You may find the most affordable Burberry Scarfs luxury handbags on our thecopybags.Burberry Outlet Scarves com while high quality can be guaranteed. Our Replica Coach Factory Outlet Store is made with special care to reach the level of an original one. Online Coach Outlet Here you will find luxury items are no longer something Coach Factory Outlet Store Online you ever hesitate going for. Coach Outlet Store Online To meet your expectation,Coach Online Factory Store we only provide Designer Coach Outlet Store Online that is perfectly imitated, featuring the slight details of originals.

January 12, 2012 | Unregistered CommenterCoach Outlet Online

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
All HTML will be escaped. Hyperlinks will be created for URLs automatically.