Hour-by-hour Comparison of Beijing's PM2.5 Pollution

Public outcry over the air quality in Beijing has impelled the government to start releasing PM2.5 pollution data, something the U.S. Embassy has been doing via Twitter since 2009. For some background on "PM2.5", see this post on the air pollution situation in Beijing. In short, PM2.5 is a dangerous form of pollution, measured in micrograms per cubic meter. The WHO's standard for a maximum acceptable yearly average is 10 μ/m³; Beijing regularly sees days over 200 or 300 μ/m³.

Now that we have an official source of hourly PM2.5 measurements in Beijing, it seems appropriate to do an hour-by-hour comparison with the embassy's, duly noting that different values from each station do not necessarily indicate inaccurate data from either source. The U.S. Embassy takes their measurements in eastern Beijing near Liangmaqiao and the Beijing Municipal Environmental Monitoring Center (BJMEMC) takes theirs in the west near Chegongzhuang, a six-mile distance. The different environments and equipment undoubtedly add to variability. In the past, local officials have stated that the embassy's location near construction sites and a major road inflate their measurements. (As a side note, if that is the case, the embassy's inflated data would more accurately represent a person's daily experience given the prevalence of both features.)

PM2.5 sensor locations in western and eastern Beijing

The Data
The numbers compared here cover February 2 to 4 for a total of 45 measurements. BJMEMC currently provides access to only the last 24 hours of data. Note that the embassy did not have data for 21:00 and 23:00 on February 2, so those times were omitted.

	BJMEMC	U.S. Embassy	Difference
Average PM2.5 (μ/m3)	33.51	49.07	31.70%
Average Hourly Change	12.80	10.39	18.83%
AQI (U.S.) Equivalent	95	123	1 category

Beijing PM2.5 levels, Feb 2 - 4, 2012

BJMEMC's measurements were at times higher than those of the embassy, but on average the embassy recorded poorer air quality. The different was enough to bump the embassy's average into a more serious AQI category. Also, the BJMEMC data's hourly fluctuation was higher. On two occasions the BJMEMC readings dropped to 3 μ/m³ and remained there for two hours. This could be an error, especially at 18:00 February 3, when the reading jumps back to 111 μ/m³ an hour later.


Impressions
Overall, the results are consistent, but different enough to rank as "Moderate" at BJMEMC's sensor and "Unhealthy for Sensitive Groups" at the embassy. The important difference, though, is not between the BJMEMC and embassy data, but between BJMEMC's own PM10 and PM2.5 measurements. Up to now, official assessments of the air pollution were based on PM10 pollution, which does not have the same health implications as the finer PM2.5 pollution and was often measured in lower concentrations. For example, BJMEMC's assessment of the last 24 hours, based on PM10 at a nearby location, is "Excellent", which is not consistent with the visible pollution outside. With these new measurements, an informed public can now make health-conscious decisions.

But what about an uninformed public? Unfortunately, the web page does not translate its measurements into an AQI category and qualitative description (or "API" in the Chinese standard). Further increasing the potential for misinterpretation, the measurements are given in milligrams instead of the usual micrograms. And finally, the descriptions of the pollution levels don't seem consistent with the given health effects. For example, the description:

Significantly aggravated symptoms in heart disease and lung disease patients, reduced exercise tolerance, and general symptoms in healthy people. Recommended that the elderly and patients with heart disease or lung disease stay indoors and reduce physical activity.

is matched with "Moderately polluted", a term that itself makes no implications about health.


The Website
Less satisfying than the data is the platform through which they are distributed. Unlike the embassy feed, which can take advantage of all of the distribution methods offered by Twitter, the BJMEMC data are trapped on a web page that is not even directly accessible from a URL. Instead, users must click a PM2.5 tab that loads the data in an iframe element using Javascript. The target page in that iframe cannot be accessed directly due to some validation techniques with Javascript and some hidden form fields. When we consider that the page only shows 24 hours at a time and does not provide an archive, it will be very difficult to extract these measurements in bulk for research purposes. Furthermore, the measurements are not updated consistently; the 21:00 measurement was the last added on February 3.

What is really disappointing—and telling—is this bit of HTML:

<body ondragstart="return false" onselectstart="return false" oncopy="document.selection.empty()" onbeforecopy="return false">

This was added to prevent users from selecting or copying the text on the page. Fortunately, it seems the site was developed on and for Microsoft applications and the controls do not work in Firefox, leaving users of that browser free to copy the text. These methods are supposed to be deprecated in WebKit as well, but I could not select any text in Chrome.

Regardless, I'll be exploring ways to capture the data in a more usable format.


Conclusion
The data quality is not perfect, but neither is the embassy's. The data access as it is currently provided can hardly be described as open, but it is definitely ajar, allowing citizens to peek in and maybe get a foot in the door to negotiate on some of the government's other data. This is a big step for such a secretive, paranoid authority, and the government deserves a pat on the back for it. Still, Beijingers need to push for:

• Setting up more sensors in representative locations
• Improving the data distribution platform and reducing the delay in publication
• Expanding this practice into other datasets, especially in public health

Once BJMEMC gets a sensor near the embassy, I'd like to see another comparison.

Namaste Dagoba imagery update

In August 2009 I visited Namaste Dagoba in the "Famen Temple Cultural Scenic Area" in Shaanxi Province, China. Afterwards, I looked for it on Google Earth, but the images available then only showed farm land. Since then I've waited for Google to release updates.

Namaste Dagoba

Today, thanks to Follow Your World, I received an email saying that new imagery is available. Unfortunately, the resolution is not very high, and the complex appears to be under construction, which would make the image at least two and a half years old. Nonetheless, it's clear and complete enough to show the grand scale of the place. 

Famen Temple Cultural Scenic Area

From end to end it's nearly 2.5 km, with a wide avenue lined with large, golden statues leading to the Namaste Dagoba. The Namaste Dagoba holds a number of treasures discovered in the Famen Temple, including what is supposedly a relic of the Buddha (his finger bone). The historic Famen Temple and monastery area is just east of the avenue's northern end. 

Grand entrance to the avenue

One of ten large statues along the avenue, with Famen Temple in the background

Looking down the avenue, from the Namaste Dagoba to the main entrance.

Beijing Air Comparison: Good vs. "Hazardous"

On June 22, 2011, Beijing had a particularly miserable day as far as air pollution goes. It wasn’t as serious as the “crazy bad” day in November 2010, but it was still remarkably polluted. Visibility was low, it stunk, and I could actually taste the pollution in the air.

Air pollution in Beijing, June 2011

According to the sensors used at the U.S. Embassy in Beijing, the pollution level was "Hazardous", meaning it had an air quality index value greater than 300. At its worst it was 473 that day (the scale only goes up to 500), with a PM2.5 concentration of 459 micrograms per cubic meter. For reference, in 2009, the average PM2.5 concentration in the Los Angeles area was "only" 12.68, and the World Health Organization recommends an average annual exposure of no more than 10. Keep in mind that the 10 μ/m³ guideline is only an "acceptable and achievable objective"—not even necessarily a safe level.

Progression of Beijing's air pollution on June 22, 2011

Readings from the sensors at the U.S. Embassy, as posted to Twitter, @beijingair

Then, just a couple days later, Beijing saw some of the best air it's had in a while.

Great air in Beijing, June 2011

For two whole hours the sensors reported air pollution levels within acceptable limits.

In Beijing, barely acceptable = outstanding

To be fair, Beijing is somewhat making progress in cleaning up the air. Earlier this year, we had a record number of consecutive blue-sky days—the best since 1998.

To compare the good and bad days, I made a page to easily switch back and forth between photos of Beijing's pollution.

Twitter at Night

I found a dataset of Twitter user locations and wanted to see what it looked like as a map. In particular, I wanted to make it look something like an “Earth at night” image and compare global light pollution with global Twitter activity. The idea is to separate industrialization from “informatization”. If a luminous place lacks proportional web activity, the characteristics of that city or country could tell us something about the nature of the digital divide, including non-technical aspects like politics.

A dataset like this could also be a useful addition to Digital Earth, as a location's web activity would be an interesting and increasingly important characteristic.

The dataset, found on infochimps.org, contains 3.8 million locations for tens of millions of Twitter users. It was gathered by mining data from user profiles between 2006 and 2010. For the uninitiated, Twitter profiles have an open text field where users can enter their location. This means it may contain any number of things, including country names, street addresses, coordinates, or “woudnt u like 2 no ;D”. While addresses and text like “Nap Town All Day Babyyyy” could have been geocoded, for this project I decided to only keep coordinates. Furthermore, I only kept coordinates that appeared to have been automatically entered by third-party applications or location-aware devices. These were prefixed by something like “iPhone:”. To clean the data, I used regular expressions in Notepad++, then did some manual cleansing in Excel.

When all was said and done, the dataset was significantly reduced, but still contained over 700,000 unique locations. QGIS and Photoshop helped produce an interesting visualization:

Twitter at Night: The globe illuminated with Twitter user locations
10000 x 3900 1.2MB jpeg*

You can see some imperfections in the data on the political map, e.g., points in the water, but we get the idea:

716,412 points plotted from coordinates taken
from the location field of Twitter user profiles
15000 x 6630 1.3MB gif*

Here is an “Earth at Night” composite image from NASA and the Defense Meteorological Satellite Program with the Twitter data overlaid. Not surprisingly, places like North Korea have few lights and also few Twitter points. But Eastern Europe, Russia, China, North Africa, and India are fairly well-lit with very sparse Twitter coverage. And take a look at Cuba – there isn’t a single point.

Twitter user locations plotted over a map of the world's lights
10000 x 3900 2.8MB jpeg*

Some things to keep in mind:

• These are mostly locations of mobile users, and the data are not necessarily a representative sample of Twitter users (or Internet users).
• In some countries, other microblogging services may be popular
• In mainland China, Twitter has been blocked since 2009. The lack of Twitter activity in China isn’t a good indicator of Internet access, but it can say something about a different kind of digital divide, maybe an information divide.

Here is a higher quality PNG of the Twitter at Night image (3.9 MB, 10000 x 3900).

*Note that all of these images have really high resolutions. If your browser has trouble displaying them, try right-clicking on the links and saving to your computer.

Putting North Korea on the map

If you've happened to pan over northeastern Asia in Google Maps, you may have noticed something odd. The roads, water bodies, markers, and labels that cover the extent of South Korea, Japan, Russia, and China are curiously absent over North Korea.

North Korea blank on Google Maps

In searching for an explanation I've found a lot of speculation, mostly about North Korea's authoritarian government, the country being cut-off from the West, the poverty, and so on. But these factors shouldn't necessarily prevent Google from mapping features in the DPRK, especially since high-resolution satellite imagery is available on Google Maps. After all, Bing has labeled the most basic features. Instead, the [partial?] answer seems to be that Google simply hasn't published the data. From a topic on the Google Maps Help Forum: 

"We never launched coverage in some countries because we simply weren't satisfied with the map data we had available. We're constantly searching for the best map data we can find, and sometimes will delay launching coverage in a country if we think we can get more comprehensive data." 

There's some good news, though, for those who want to browse detailed maps of North Korea. Google Map Maker has opened up the DPRK for contribution (along with 187 other countries as of this writing) and it looks like there is already a lot of detail.

In contrast to Google Maps, North Korea has more features in Map Maker

Features added to Google Map Maker in Pyongyang

Content from Map Maker may eventually lead to a satisfactory map of North Korea and publication on Google Maps. Until then, Map Maker, satellite view on Google Maps, and Panoramio photos on Google Earth are pretty good resources. Also have a look at the massive amount of data available from the North Korea Uncovered project. There you can download a KML file with layers for infrastructure, palaces, artillery, and many other interesting sites.There was a WSJ article about the project.

Together, all of these resources should help take some of the mystery out of the blank spot on the map.

New satellite images show the demolition of Tangjialing

New satellite images in the recent round of updates for Google Earth included extensive updates in Beijing. This is particularly interesting because it shows the transformation of an area known as Tangjialing in northwestern Beijing (Haidian District).

If you had visited the Tangjialing community a year ago, you would have seen a neighborhood typical of the ones on the outskirts of Beijing. If you visit it today, you see this:

The ruins of Tangjialing, March 2011

Currently, Google Earth has images from October, 2010, that show the demolition of Tangjialing in only a matter of months. The village goes from being completely intact in May to mostly demolished in October, and most of the buildings visible in the October images are gone now. Panoramio images suggest that even going into summer, Tangjialing was still a functioning community.

If the former residents weren't satisfied with their living conditions there, and they were compensated fairly for their relocation, then this development would seem like a win-win. Otherwise, that's a few thousand more people resenting the guys at the top whose policies are rapidly reshaping Beijing's urban landscape.

A KML file outlining the most recent image updates around the world can be downloaded at:
mw1.google.com/mw-earth-vectordb/Imagery_Updates/imagery_updates.kml

You can track image updates for a specific location at:
followyourworld.appspot.com