Finland (Part 2)

Last week, I posted an article about the demographic causes of Finland’s preeminence on the 2009 Programme for International Student Assessments (PISA) exams. My purpose in that post was to show that a simple comparison between countries is impossible—but this is not to say that we can learn nothing from international comparisons. We can learn a lot, but only through careful and detailed examination of the data. I’ve spent much of the past two weeks sifting through that data, and I’ve come up with some interesting leads.

All of the issues addressed here bear further research; in general, it's dangrous to try to draw causal inferences from survey data—I can only hope that some better-funded researcher will look deeper into some of these issues.

1.       Rich kids are doing fine.

Breaking the population of each country down into quarters, according to economic, social, and cultural (ESC) status provides some interesting insight. First of all, the most ESC-advantaged quarter of the US population is actually quite competitive with the most advantaged quarters of other nations: our most advantaged quarter scores 8^th in the OECD in reading (compared to 14^th for all four quarters combined), and only one country (New Zealand, oddly enough) has a most advantaged quarter that scores statistically significantly higher than ours.^[1]

Data on math and science scores is much more limited, but what data I have suggests that our top ESC quarter isn’t doing quite as fine in those subjects; but it is definitely closer to fine than the rest of the country. My very rough estimates put the mathematics and science performance of our most advantaged quarter around 16^th and 10^th in the OECD, respectively, compared to 25^th and 17^th for all four quarters combined.^[2]

2.       Poor kids and lower-middle-class kids aren’t doing fine, but for very different reasons.

This is interesting—and completely unexpected. Simple demographics seem to account for the weak performance of the most ESC-disadvantaged quarter of the US population. Our most disadvantaged quarter is highly disadvantaged, considering how wealthy we are overall; it ranks 20^th in average ESC status within the OECD. In its mean score on the PISA test, however, it is tied for 18^th—so, in fact, our bottom ESC quarter is outperforming the less impoverished bottom ESC quarters of a couple other countries. Now, we shouldn’t be too proud of that, considering our high expenditure on education, but it’s a very different situation from that of our lower-middle-class.^[3]

The second quarter of the US population in terms of ESC status—the quarter second from the bottom, that is—is not so bad off, in terms of ESC status, by PISA’s measurements: it’s the 11^th most ESC-advantaged second quarter in the OECD—yet it ranks 21^st in terms of scores on the PISA exam. Lacking a demographic explanation, we might attribute this poor performance to a failure on the part of the public schools available to these students—although, it’s also possible that there are cultural explanations, e.g. attitudes towards education—that are not easily captured in demographic data.^[4] More>>

3.       Separate is not equal.

Schools in most nations are segregated by class, and US schools are more segregated than most.^[5] According to PISA’s data, schools serving disadvantaged kids are significantly worse places to learn, no matter what your individual ESC status; this too is true in nearly every country in the OECD, and it is less true in the US than in most, but it is still an important reality of education in today’s industrialized nations.^[6]

To illustrate the point—and I hesitate to illustrate the point, for it has some disturbing implications—suppose that two students of identical ESC status—say they are both in the 75^th percentile of ESC status in the US—attend different schools, one advantaged the other disadvantaged. On average, the student attending the advantaged school scores 62.6 points higher on the PISA reading exam than the one attending the disadvantaged school. That score difference is close to twice the US-Finland score gap in reading.^[7]

                The difference in education quality between advantaged and disadvantaged schools has at least two sources. One is the quality of the resources available to the school, but I suspect the more significant factor is the one that Justice Warren sites in Brown v. Board of ed.—that is, the inequality inherent in a system that is segregated by race or class. A school full of students from disadvantaged homes concentrates the academic and behavioral difficulties that arise from underprivilege, so that each student suffers not only his own disadvantages but the combined disadvantages of all his peers.

4.       Education begets education.

The US ranks third in the OECD in terms of per-capita GDP,^[8] making it one of the wealthiest nations in the world—yet, it ranks 10^th in terms of average ESC status.^[9] Here’s why: PISA measures ESC status by looking not merely at measures of family wealth, but at measures of social and cultural status as well—parental education levels, educational resources available at home, and cultural possessions, such as classical literature, poetry, and works of art. The result is a more useful measure, because educational outcomes are more closely correlated to parental education levels and home culture than to income.

Thus, over the long term, a lackluster school system leads to lower average ESC status within the society, which leads in turn to worse educational outcomes in the next generation. Similarly, improvements in a school system, if they are sustained, create a snowball effect of rising outcomes, with each generation of parents providing better at-home support than the last.

5.       If you have kids and live in the USA, don’t get divorced.

The gloomy joke in that headline is that we don’t know, of course, whether the relationship between single parenthood and low-achievement is causal. The data seems quite clear, however, that being from a single-parent family is bad luck for a kid, more so in the US than almost anywhere else in the world.

The negative relationship between single-parent homes and poor test performance in the US holds even after you account for a host of other family variables. Two students in the US, both speaking English as a first language, with no immigrant status, identical parental educational attainment and occupational status, the same number of books and other cultural possessions at home, and the same amount of family wealth, but differing crucially in the number of parents at home will score, on average, 18 points apart on the PISA exam. Only three nations that took the PISA exam—Jordan, Qatar, and Dubai—have a stronger correlation between single parent family status and educational performance.^[10]

6.       …And don’t buy TVs, either.

Again, this isn’t necessarily causal—I haven’t even run any regressions on it—but the negative correlation between a US student’s performance on the PISA exam and the number of TVs that her family owns is large—and consistent across all three subject tests. 78.86% of US students report having three or more televisions at home; that’s the highest percentage of three-TV students in the OECD. In the US, those three-TV kids score lower than students who report owning only one TV by an average of 41 points on the reading exam, 34 points on the math exam, and 40 points on the science exam, making US students one of the four or five most television-addled in the OECD.^[11] (Again, no causal relationship between TV and test-scores has been demonstrated, only a correlation, so "television-addled" isn't quite the right phrase here. It’s possible that families that own more televisions tend to be less intellectual and thus produce weaker students without any help from the TVs… My money’s on the TVs though.) More>>

More About Rich and Poor Kids

It’s informative, I think, to look at the data for all four quarters of the US and Finnish population by ESC status.

Bottom Quarter

Second Quarter

Third Quarter

Top Quarter

Reading Performance

ESC Status

Reading Performance

ESC Status

Reading Performance

ESC Status

Reading Performance

ESC Status

Mean Score

Rank In the OECD

Rank In the OECD*

Mean Score

Rank in the OECD

Rank in the OECD*

Mean Score

Rank in the OECD

Rank in the OECD*

Mean Score

Rank in the OECD

Rank in the OECD*

Finland

504

1st

5th

257

2nd

4th

548

2nd

4th

565

4th

10th

US

451

18th

21st

481

21st

11th

512

14th

11th

558

8th

10th

US-Finn score gap

53

46

36

7**

* In descending order of ESC status: thus, for example, the 5^th in this column for Finland in the Bottom Quarter section means that the most ESC-disadvantaged quarter of Finland has the 5^th highest mean ESC status of any such quarter in the OECD.

**not statistically significant, even at very low confidence levels.

It's worth mentioning a hidden caveat in this data, though the significance of that caveat is difficult to assess: the PISA exams are administered to a randomized sample of 15-year-old students in each country—not represented, it seems, are those who drop out of school or switch to homeschooling before the age of 15. Dropouts presumably tend to be those members of the bottom ESC quartile with the worst educational outcomes; by selecting them out of the data, we therefore may be inflating the score of the bottom quarter. Most students who stop attending school do so between the ages of 16 and 18—in many states its illegal to do so earlier, which doesn't mean it doesn't happen—so it's difficult to find data on dropout rates among younger students. For the same reasons, though, we can probably expect these dropout rates to be relatively small. Lacking data on either pre-16 dropout rates or the demographics of such dropouts, its impossible to speculate further on the impact of excluding these kids from the data.

More about TVs

American kids with only one TV at home score an average of 33 points above the national average, across all three subject exams, and place only 11^th in the OECD, when compared to one-TV kids from other countries; overall, we placed 20^th, as a nation. Other big winners in the one-TV comparison are Australia and Luxembourg, which jump to 1^st and 18^th from 6^th and 29^th, respectively.

In general, correlation between TV ownership and test-scores varies interestingly across nations, and I encourage the reader to look at Table 2 below, which provides a summary of the available data. In a few countries—Turkey, Mexico, and Chile, for example—television ownership is presumably a sign of wealth, and is therefore associated with higher test scores. It’s hard to know the extent of this wealth-indicator effect, but it seems likely that many OECD countries are sufficiently wealthy to render it irrelevant. In the table below, I’ve included the percentage of each country’s student population that has three or more TVs at home and the per-capita GDP for each country; both of which provide some hint of the extent to which television ownership may be a predictor of wealth in a given country—i.e. in countries with higher GDP and more TVs, TV-ownership is less likely to be an indicator of wealth.

The exception would appear to be Israel (see the bottom of the table below) where television-ownership and high performance on the PISA are strongly and positively correlated, despite a high per-capita GDP. I asked an Israeli friend about it and he offered the following explanation, which I find compelling and kind of fascinating. People in most countries turn on the tube to watch either domestically produced television or foreign—usually American—television programming dubbed into the national language. In Israel, most television is American or British programming, in English, with Hebrew subtitles. What Hebrew programming there is, my friend tells me, is of unusually high quality. Thus, watching TV in Israel usually means practicing reading and learning a foreign language simultaneously: a vastly richer activity, in terms of cognitive engagement, than watching TV in most countries. I’d like to do a randomized study in which you show students subtitled foreign-language films, say, twice a week. I wonder what the impact on educational outcomes would be.

Table 2: Summary of Correlations Between Performance on PISA Exams and Television Ownership

Organized in descending order by mean difference in score between students owning one television and those owning three or more. A cautionary note: standard errors on the performance of 1-TV students is quite high in many countries, especially the US, so we cannot be as confident in these numbers as I would like.

Country	GDP/Capita	Percentage of students who report owning three or more TVs	Performance of 1-TV kids		Performance of all students		Mean Difference in Score Between 1-TV Kids and 3-or-More-TV Kids
			Mean Score†	Rank in the OECD	Mean Score†	Rank in the OECD
Luxembourg	82456.00	58.04	514.67	18th	481.67	29th	43.33
France	32495.00	49.63	525.33	14th	497.00	18th *	41.67
Australia	37615.00	67.2	553.33	1st	518.67	6th *	40.67
Belgium	34662.00	43.22	534.67	8th	509.33	11th	39.33
US	46434.00	78.86	529.67	11th	496.33	20th	38.33
Slovenia	26557.00	34.22	525.00	15th	498.67	17th	37.00
Iceland	36325.00	65.06	532.67	10th	501.00	12th *	35.67
Switzerland	41800.00	29.1	534.33	9th	517.33	8th	33.33
New Zealand	27020.00	58.29	543.67	3rd *	524.00	5th	29.33
Austria	36839.00	51.85	507.33	22nd	486.67	26th	28.00
Denmark	36326.00	78.75	526.33	13th	499.00	16th	27.33
Netherlands	39594.00	66.8	536.67	7th	518.67	6th *	20.67
Germany	34683.00	78.5	529.00	12th	510.00	10th	20.67
UK	34957.00	53.54	518.33	17th	500.00	15th	20.67
Japan	33635.00	49.28	543.67	3rd *	529.33	3rd	20.00
Canada	36397.00	69.27	542.33	5th	526.67	4th	19.67
Italy	31016.00	65.72	502.33	25th	486.00	27th	19.67
Sweden	36785.00	66.97	511.67	19th	495.33	22nd	19.00
Czech Rep.	23995.00	37.34	504.00	23rd	490.33	23rd	16.33
Finland	35322.00	58.96	553.00	2nd	543.67	1st	15.67
Norway	53672.00	51.6	511.33	20th	500.33	14th	15.67
Hungary	18763.00	46.96	510.00	21st	495.67	21st	13.33
Slovak Rep.	20270.00	28.55	497.00	26th	488.00	25th	11.33
Greece	27793.00	47.57	480.00	29th	473.00	30th	10.67
Korea	26574.00	7.92	542.00	6th	541.00	2nd	10.33
Estonia	20620.00	34.42	521.67	16th	513.67	9th	10.00
Spain	31469.00	56.78	490.67	28th	484.00	28th	7.33
Ireland	44381.00	77.86	497.00	26th	497.00	18th *	2.00
Poland	16312.00	29.39	502.67	24th	501.00	12th *	-2.33
Portugal	22638.00	70.47	478.00	30th	489.67	24th	-14.67
Chile	14106.00	52.53	426.00	33rd	439.00	33rd	-25.67
Israel	26444.00	50.3	440.00	31st	458.67	31st	-32.67
Mexico	14128.00	36.33	402.00	34th	420.00	34th	-38.00
Turkey	13362.00	10.08	437.33	32nd	454.33	32nd	-54.00
OECD Ave.	32218.97	51.8047059	508.93	17th	496.62	17th	14.11

* Ranks marked with an asterix (*) are ties.

† Mean scores in this table are averages of scores in all three subject tests. Such an average is a bit of a Frankenstein Monster, but they make for a very concise table. Anyone who wants more detail can email me at deweytodelpit@gmail.com, and I’ll send you an excel spreadsheet with all of the data on television ownership and test scores.

---

[1] ^ PISA 2009 Results, Volume II, Table II.3.1

[2] ^ I estimated the math and science performance of students in the top ESC quartile by barbaric means. Lacking access to multi-level data, I simply looked at the PISA scores of students whose families owned three or more cars. Those students, who make up a stunning 46.43% of the population, placed 7th, 16th, and 10th in the OECD, in terms of reading, math, and science, respectively. Anyone interested in examining this question further should play around with the data querying tool on the PISA website. I warn you, though, it took a lot of Excel wrangling to turn those PHP tables into something that would reveal international rankings.

[3] ^ PISA 2009 Results, Volume II, Table II.3.1

[4] ^ Ibid

[5] ^ PISA 2009 Results, Volume II, Table II.5.10

[6] ^ Ibid

[7] ^ Ibid

[8] ^ PISA 2009 Results, Volume I, Table I.2.20

[9] ^ PISA 2009 Results, Volume II, Table II.3.1

[10] ^ PISA 2009 Results, Volume II, Table II.2.4

[11] ^ All of these data are drawn from the data-query tool on PISA’s website. If anyone wants to see all the data I pulled in spreadsheet form, just email me at deweytodelpit@gmail.com.