Thursday, December 30, 2010

Finland (part 1)

Every time I get into a conversation about education lately, someone drops a side-comment about Finland, like it's the Valhalla of public schooling. In the past few months, the success of Finnish students on international exams has been widely touted and press-ganged into the service of various arguments about what we do and don't need more of in America. Most notably, perhaps, in the documentary Waiting for "Superman", Davis Guggenheim cites Finland's success as proof that you can have a good public school system without spending any more money per pupil than we do currently.

I don't know much about the Finnish school system—I'm sure it's a good one—but to give teachers, schools, and school-systems the sole blame and credit for differences in test scores is one of the most common misuses of data in the study of education. One should look first not at the schools but at the society that surrounds them: differences in the cultural and economic backgrounds of students and parents typically account for much more of the variation in test scores than do differences in pedagogy, school design, or teacher training. I spent the past week looking through the data on demographics, international test scores, and their interaction. The available data was copious, and it had some interesting stories to tell.


 

Background

Last year, 15-year-old students at randomly selected schools in 65 countries around the world sat for a series of internationally normed exams in reading, mathematics, and science, created and administered by the Programme for International Student Assessment (PISA). The results, published this year, showed the USA, as usual, doing worse than our status as the most powerful country in the world would seem to warrant. Leaving Hong-Kong and Shanghai out of the data—since they're not nations and probably aren't comparable[1]—we placed 15th in reading (tied with Poland and Iceland), 21st in science, and 29th in mathematics (tied with Portugal and Ireland); the latter represents a five-place drop from 2003, when we placed 24th in mathematics. Finland had the highest over-all scores of any nation. (See Appendix A for a full list of results.)

America's poor performance is especially stinging because we spend a hell of a lot on education—far more than Finland. From age 6 to age 15, the average student costs the US government and its citizens (public and private expenditure combined) $105,752 in cumulative schooling costs; for Finland that number is only $71,385. (These numbers are adjusted using purchasing power parities to account for differences in the cost of goods and services in each country.) By this measure, the US spends more on education than any other nation in the Organisation for Economic Coopration and Development (OECD)—i.e. probably any nation in the world—except Luxembourg (poor Luxembourg spends a stunning $155,624 per student from age 6 to 15 and still scores abysmally); of the 34 nations in the OECD, Finland is the 14th smallest spender. [2]


 

Mitigating Factors

A week or so ago, a commenter on this blog wrote, "I don't know what the Finns are doing right…." Well, part of what they're doing right is having very few disadvantaged students, low immigration rates, and a homogeneous national culture.

In the CIA World Factbook's list of nations by income inequality, Finland comes in 114th; the US, with greater income inequality than the Ivory Coast, Nicuragua, Cambodia, Thailand, Kenya, and every nation in Europe, comes in 42nd. PISA estimates that 10.4% of US students suffer from social, cultural, and economic (ESC) disadvantage, as opposed to only 3.9% of Finnish students.[3]

Perhaps even more important than its level economic and socio-cultural playing field is Finland's cultural homogeneity. 91.2% of Finnish citizens speak Finnish as a first language. 93.5% of them are ethnic Finns. 82.5% belong to the Lutheran church. Only 82.1% of Americans speak English as their native language. Just under 80% are "white," a broad, ill-defined, and permeable ethnic category. The largest religion in America, at 51.3% of the population, is Protestantism, really more a category of religions than a single religion. Almost 20% of American students are either first or second generation immigrants, as opposed to 2.6% of Finnish students.[4]

Thus Finnish schools face few of the challenges that American public schools must overcome. With fewer disadvantaged students, they likely have a lower incidence of serious learning deficits and behavioral issues; they have fewer students whose native language differs from that of the school; and most importantly, they have a shared sense of culture, history, and values, which must make the school—both as an extension of the Finnish state and as a collection of teachers and administrators—appear familiar and comprehensible to the parents. We can expect to find in Finnish schools none of the mistrust, miscommunication, and tribalism that plague inner-city schools here in America.

There are some interesting details relating to educational expenditure as well. First of all, America has an immense student population—the largest of any country that took the exam and more than two and a half times larger than the runner up.[5] A big student population means a lot more bureaucracy, and thus greater expenses for the same quality of education. Secondly, although the Finns spend less per pupil in absolute resources, they actually spend more
per pupil in terms of percentage of their GDP. Thus, though the purchasing power devoted to each student is less, the commitment of the available resources within the society is greater.[6]


 

Demographics and Test Scores

The people at PISA have been conscientious enough to analyze their data in terms of many of the demographic variables discussed above, allowing us to see the actual impact of those variables on test scores in each country. Unfortunately for our purposes, their analysis focuses on reading scores (new assessments with analysis focused on science and math will be carried out in the next five years), but presumably demographic impacts on test scores do not differ too much from subject to subject. (I will take a closer look at that assumption in my next post.)

According to PISA's analysis, adjusting scores to account for ESC (economic, social, and cultural) disadvantage[7] eliminates about 22% of the difference between Finnish and US scores. Using PISA's data, I calculated Finland's national average if they had as many immigrant students as the US and educated them as effectively as they educate their immigrant students currently; the result eliminated about 32% of the Finland-US score gap.[8] (NB: very likely, these are not cumulative effects; low-scoring immigrants in the US are mostly ESC disadvantaged as well, and accounting for both variables simultaneously would probably not eliminate more than about 32% of the score gap.)

Nationwide variables also account for some of the gap. According to PISA's analysis, adjusting scores to account for the number of students in the school system eliminates just under 20% of the score gap.[9] PISA provides no measure of cultural homogeneity and it is impossible to estimate the impact of that quality on national test scores, using their data. It's always hard to pin down culture in quantitative terms, but I suspect that if we could, it would account for a significant share of the US-Finnish test gap.

Even if we could explain away the entire gap, though, we still would have to account for the fact that we're spending more than twice as much per pupil as the Finns. My point is not that the differences in scores and spending can be entirely explained away, but rather that a direct simple comparison is impossible. Ultimately, the search for excuses is less productive than the search for explanations. As I looked deeper into this data, I grew less interested in whether or not the fault was in our school system and more interested in just what the data could teach us about the educational challenges that face our country. It can tell us a lot, I think, but this post is getting long, so I'll save that discussion for next week.


 


[1] ^ Presumably, Shanghai draws many of the best students from around China, through that nation's highly meritocratic system of public boarding schools, making it hard to compare their data to that of other countries. Hong Kong I know less about, but I'm hesitant to compare its results to those of entire nations.

[2] ^ PISA 2009 Results, Volume I, What Students Know and Can Do: Student Performance in Reading, Mathematics and Science. See table I.2.20. The PDF is viewable here, but you have to download the whole 272-page pdf, so you'll want a decent internet connection.

[3] ^ PISA uses a complex measure of socioeconomic advantage and disadvantage, called the PISA index of economic, social, and cultural status (ESCS). The ESCS index is calculated using a combination of parental job status and education, family possessions like cars and dishwashers that are indicative of wealth, and the availability of cultural and educational resources at home.

[4] ^ All these data are from the CIA World Factbook

[5] ^ See again PISA 2009 Results, Volume I, Table I.2.20

[6] ^ In the US, we spend 5.5% of our GDP on education; the Finns spend 5.4%. The 0-14 age-group, however, makes up only 16.4% of the Finnish population, as compared to 20.2% of the US population. If we imagine that educational expenditures are devoted entirely to 0-14 year olds, then the US and Finland spend approximately 27% and 33% respectively of their per-capita GDP per student. Of course, educational expenditures go to older kids as well, so those estimates are very, very rough. The website Trading Economics provides some good piecemeal data, summarized in the table below.

 

Per-Pupil Expenditure on Education as a Percentage of Per-Capita GDP at Various Levels of Education (2003-2007 average)


Primary

Secondary

Tertiary

US

21.5

24.57

24.92

Finland

18.10

30.07

35.432


[7] ^ this adjustment is made based on the number of economically, socially, and culturally disadvantaged students, not based on the average economic, social, and cultural status of the nations. The number of disadvantaged students is estimated as the number of students who fall more than one standard deviation below the international mean on PISA's index of economic, social, and cultural status (see footnote 3).
     The full analysis can be found in PISA 2009 Results, Volume II, Overcoming Social Background: Equity in Learning Opportunities and Outcomes, table II. Like volume I, it's a big file.

[8] ^ I did this calculation myself, because PISA's estimation appears to be either erroneous or based on a conglomeration of data from nations with very different immigrant situations. (In several nations outside the OECD—e.g. Dubai and Qatar—immigrants outscore native students by a wide margin; obviously, these are situations quite different from that abiding in the US.) The PISA analysis actually adjusts scores down to account for high immigrant populations, clearly an inappropriate adjustment for Finland and the US, where immigrant students consistently score below native students. You can find the PISA analysis in the second half of table I.2.20 of the PISA 2009 Results, Volume I, referenced above. My own analysis is based on the more detailed data found in table II.4.1 of The PISA 2009 Results, Volume II.

[9] ^ PISA 2009 Results, Volume I, Table II.2.20, again.



Appendix A: Average scores on the PISA exam by subject and country or city

For partial data on PISA exams from previous years, see the PISA Wikipedia page.

Country or City
Mean Reading Score
Shanghai-China
556
Korea*
539
Finland*
536
Hong Kong-China
533
Singapore
526
Canada*
524
New Zealand*
521
Japan*
520
Australia*
515
Netherlands*
508
Belgium*
506
Norway*
503
Estonia*
501
Switzerland*
501
Poland*
500
United States*
500
Iceland*
500
Liechtenstein
499
Germany*
497
Sweden*
497
Ireland*
496
France*
496
Chinese Taipei
495
Denmark*
495
United Kingdom*
494
Hungary*
494
Portugal*
489
Macao-China
487
Italy*
486
Latvia
484
Slovenia*
483
Greece*
483
Spain*
481
Czech Republic*
478
Slovak Republic*
477
Croatia
476
Israel*
474
Luxembourg*
472
Austria*
470
Lithuania
468
Turkey*
464
Russian Federation
459
Dubai (UAE)
459
Chile*
449
Serbia
442
Bulgaria
429
Uruguay
426
Mexico*
425
Romania
424
Thailand
421
Trinidad & Tobago
416
Colombia
413
Brazil
412
Montenegro
408
Jordan
405
Tunisia
404
Indonesia
402
Argentina
398
Kazakhstan
390
Albania
385
Qatar
372
Panama
371
Peru
370
Azerbaijan
362
Kyrgyzstan
314
Country or City
Mean Math Score
Shanghai-China
600
Singapore
562
Hong Kong-China
555
Korea*
546
Chinese Taipei
543
Finland*
541
Liechtenstein
536
Switzerland*
534
Japan*
529
Canada*
527
Netherlands*
526
Macao-China
525
New Zealand*
519
Belgium*
515
Australia*
514
Germany*
513
Estonia*
512
Iceland*
507
Denmark*
503
Slovenia*
501
Norway*
498
France*
497
Slovak Republic*
497
Austria*
496
Poland*
495
Sweden*
494
Czech Republic*
493
United Kingdom*
492
Hungary*
490
Luxembourg*
489
United States*
487
Ireland*
487
Portugal*
487
Italy*
483
Spain*
483
Latvia
482
Lithuania
477
Russian Federation
468
Greece*
466
Croatia
460
Dubai (UAE)
453
Israel*
447
Turkey*
445
Serbia
442
Azerbaijan
431
Bulgaria
428
Uruguay
427
Romania
427
Chile*
421
Mexico*
419
Thailand
419
Trinidad & Tobago
414
Kazakhstan
405
Montenegro
403
Argentina
388
Jordan
387
Brazil
386
Colombia
381
Albania
377
Tunisia
371
Indonesia
371
Qatar
368
Peru
365
Panama
360
Kyrgyzstan
331
Country or City
Mean Science Score
Shanghai-China
575
Finland*
554
Hong Kong-China
549
Singapore
542
Japan*
539
Korea*
538
New Zealand*
532
Canada*
529
Estonia*
528
Australia*
527
Netherlands*
522
Chinese Taipei
520
Liechtenstein
520
Germany*
520
Switzerland*
517
United Kingdom*
514
Slovenia*
512
Macao-China
511
Poland*
508
Ireland*
508
Belgium*
507
Hungary*
503
United States*
502
Norway*
500
Czech Republic*
500
Denmark*
499
France*
498
Iceland*
496
Sweden*
495
Austria*
494
Latvia
494
Portugal*
493
Lithuania
491
Slovak Republic*
490
Italy*
489
Spain*
488
Croatia
486
Luxembourg*
484
Russian Federation
478
Greece*
470
Dubai (UAE)
466
Israel*
455
Turkey*
454
Chile*
447
Serbia
443
Bulgaria
439
Romania
428
Uruguay
427
Thailand
425
Mexico*
416
Jordan
415
Trinidad & Tobago
410
Brazil
405
Colombia
402
Montenegro
401
Argentina
401
Tunisia
401
Kazakhstan
400
Albania
391
Indonesia
383
Qatar
379
Panama
376
Azerbaijan
373
Peru
369
Kyrgyzstan
330

  • Countries marked with an asterix (*) are in the OECD; those without are not.
  • Scores colored green are above the OECD average in that subject; scores marked in pink are below. Note that the OECD average is higher than the average for all 65 nations, because most non-OECD nations score well below most OECD nations.

2 comments:

  1. So how does the US score in comparison with the Finns (or anyone else) once you factor out the ESC disadvantaged? Wouldn't that give us a "common denominator" and be a reasonable place to start thinking about our educational system? Without it, how do we even know what we're talking about?

    ReplyDelete