The Success Sequence Is Extremely Misleading And Impossible To Code

The "Success Sequence" is explained most recently as follows:

Graduate from high school;
Maintain a full-time job or have a partner who does; and
Have children while married and after age 21, should they choose to become parents.

Together, this is supposed to keep your risk of poverty very low. Last week I pointed out that rule (2) is doing basically all of the work in the 2007 dataset that Sawhill and Haskins used in the book that popularized the Success Sequence concept. In light of Brookings' new post about the Success Sequence and some information I have received about how rule (3) is identified, I want to share some additional points about this concept here. 2013 Figures Let's start with a calculation of the latest figures using the 2013 ASEC. In this calculation, I will follow as best I can the guidelines Sawhill/Haskins and Brookings has set out and be very explicit about how I am doing it, including providing parentheticals for the ASEC variables I am relying on. In 2013, there were 313.4 million people, 45.7 million of which were poor (FTOTVAL < FPOVCUT). I start with this sample. From there, following Sawhill/Haskins, I eliminate all families whose reference person (aka head) is below the age of 25 or above the age of 64.¹ I also then eliminate all families that received disability income (FDISVAL).² This initial screen rids the sample of 71.5 million people (22.8% of all people), and 11.4 million poor people (24.8% of all poor people). The plights of these very large numbers of families and people are simply ignored from the analysis altogether, which is a serious problem. This screen leaves us with 242 million people, 34.4 million of which are poor. The poverty rate of this sample is 14.2%. From there, I eliminate all families who do not have at least one person working full time according to the Sawhill/Haskins definition (WKSWORK >= 40, HRWRK >= 35). This elimination leaves us with 194.1 million people, 9.3 million of which are poor. The poverty rate of this group is 4.8%. From there, I eliminate all families who do not have at least one person with a high school degree or better (A-HGA >= 39). This elimination leaves us with 185.3 million people, 7.1 million of which are poor. The poverty rate of this group is 3.8%. Thus, under this very simple specification, I get pretty near whatever low-poverty conclusion the Success Sequence is aiming at with full-time work alone, and even nearer when I exclude families that contain nobody with a high school degree. I get this without ever bringing in the marriage and child-delaying stuff. I would tell you what you get when you bring in that stuff too, but it's actually not possible to do, which is my next point. The Marriage/Children Norm Cannot Be Identified As spelled out by a combination of the Sawhill/Haskins book and the latest Brookings post, a person counts as having satisfied the third norm by doing one of the following:

Being childless.
Being married before you have a child, maintaining that marriage, and having your first child after 21.

The identification problems mainly plague route (2). First, as Haskins himself notes in an explanation I received second-hand from Shawn Fremstad of the Center for American Progress, it is impossible in the ASEC to determine whether someone was married before they had a child. All you know is that they currently have a child in 2013 and that they are currently married in 2013, but you don't know which came first. Second, as Haskins also notes, the specification Brookings uses does not distinguish between step and biological parents. A married family head in a family with kids could be assuming responsibility for kids from a prior marriage. There is a way to distinguish those people using variables for biological relations, but Sawhill/Haskins apparently does not do that, and even if they did, it would be unclear what to do with step families. The problem that this specification presents is that it is impossible to know whether the head had kids before they were 21. Subtracting the eldest kid's age from the head's age doesn't tell you whether the head followed the delay-children norm when the eldest kid is not biologically related to the head. Additionally, the head could have had children prior to age 21 with another partner that they no longer live with, which would be completely undetectable. Route (1) does not have identification problems per se, but it does create other severe problems for tallying up Success Sequence followers. To see what I mean, consider the following "deadbeat dad" scenario. A man impregnates a woman (perhaps multiple times) outside of marriage. She has the kids and lives with them in one dwelling. He does not have the kids and lives elsewhere by himself. Under the ASEC, this scenario create two families: the single-mother family and the deadbeat-dad family. The single-mother family will score as not having followed the Success Sequence because she has kids but is not married. If she is in poverty, then her poverty is chalked up as evidence that not following the Success Sequence is bad. The deadbeat-dad family, provided he works full time and has a high school degree, shows up as having followed the Success Sequence because he lives in a childless family. If he is not in poverty (which he likely isn't if he works full-time and lives alone), then his lack of poverty is chalked up as evidence that the Success Sequence works. This might seem like a narrow example of this specification failing, but it isn't. For every single-parent family in the ASEC, there is another parent out there floating around that will be picked up in another ASEC family (unless they are dead or incarcerated). The single-parent family and all the people in it always get counted as not following the Success Sequence, but the absent parent floating around elsewhere often will get counted as actually following the Success Sequence. This is nuts and also unduly skews the results in a way that will overstate the performance of the Success Sequence. Families Are Complicated In addition to the above, it's worth noting that families are complicated and also joined by the Census in ways that overstate the Success Sequence point. A good number of households have multiple families living in them with complex relations between them, making it difficult to meaningfully group them into simplistic family types. I won't go too far into this because it gets quite technical, but consider the following illustration I pulled from the 2013 ASEC:

For poverty purposes all of these individuals are combined into one family and their income all summed together. This is why the second family is said to not be in poverty despite the fact that they only have a single earner making $5100. Consider how exactly you would classify this big group family? The primary family consists of an older married couple, one of who works full time and has a college degree (so a Success Sequence follower of sorts), and another relative they are living with (maybe a brother). Then the first related subfamily has a married couple, one of which has an advanced degree but doesn't work at all, and the other of which has less than a high school education and does work some. Then finally, there is a single mother on the bottom who worked full time with an Associate's degree. The whole thing defies simple classification, and though this is an extreme case, a lot of families are this way (and even spread across multiple households). Also, more importantly for the purposes of this post, the Census has a strange way of classifying many cohabiting couples. They count one of the partners as belonging to the primary family with all the children and then the other as belonging to an unrelated subfamily alone. Because the subfamily is "unrelated" under this taxonomy, that also means that their income is not joined with all the others for poverty calculation purposes, which generates households that look like this:

In both households, the top family ends up all in poverty because the income of the people in that family is not enough to pull them above poverty, while the cohabiting partner on the bottom is not in poverty because they make a decent income. But if you were to combine all these people so as to match the reality of what's going on here, none of them would be in poverty. For Success Sequence purposes, the Unmarried Partner at the bottom shows up as having followed the Success Sequence because they live in a childless family, work full time, and have a high school degree or better. Their lack of poverty is scored as evidence that the Success Sequence works. The top family then shows up as having not followed the Success Sequence because they are an unmarried parent, and their poverty gets scored as evidence of what happens when you stray. Needless to say, this is nonsense all around and, in fact, what we have here are cohabiting-couple families that did not follow the Success Sequence but are doing just fine. Conclusion Given the above, it's honestly hard to understand why Sawhill/Haskins and Brookings more generally have presented the Success Sequence like this, or indeed at all. Full-time work gets you the vast majority of the way to the low-poverty conclusion and then high-school education gets you basically right up to it. Bringing in the marriage and child-delay stuff is totally unnecesary and then can't even be properly identified in the data. Adding a condition that does basically no work for your conclusion that you can't even identify is utterly baffling. I hate to accuse others of bad faith, but it's very difficult to not wonder if there was an agenda for the marriage/child points that they crammed in no matter how irrelevant it was and how impossible it was to operationalize.

1. In households with related subfamilies, the primary family and related subfamily are combined together, with the age of the head of the primary family determining whether the families are dropped. This is necessary because related subfamilies are combined with primary families for poverty calculation purposes. 2. It's unclear to me what Sawhill/Haskins means by disability income. FDISVAL refers to private disability income, e.g. from an insurance plan. When I replicated their 2007 figures, this was the variable that got me the closest to their figures. However, if you were going to do this right, you should also include Social Security and Supplemental Security Income that is received for disability purposes. What's more, if the point is to screen out disability, there are much better ways to do that directly, including excluding those who say they did not work last year because of illness or disability (RSNNOTW = 2) and people who say they have one or more of the six disabilities that the ASEC tracks (PRDISFLG = 1).