Using GIS to Evaluate a
New Source of Transportation Census Data:
The American Community Survey

Wende A. O’Neill^, Ph. D. & Daniel Baldwin Hess

Introduction

In a rapidly changing society, many people have come to view the decennial (ten year) census as slow and antiquated. Data taken from a census captured only once in a ten-year period is not released until a few years after the decennial year and, for many applications, the data is stale before it is time for a recount. To improve the situation, the U.S. Bureau of the Census is implementing plans as the 21st century approaches for the American Community Survey (ACS). Using refreshed multi-year accumulations, the ACS will provide timely data by sampling households nationwide every month of the year to collect housing, social, and economic data.

Transportation planners typically spend a great amount of effort adjusting decennial Census data to fit their needs. The ACS, however, provides data users with timely, comparative housing, social, and economic data throughout the decade about communities and population groups. Since continuous measurement is an untested process in the United States, transportation planners have expressed concern about the different data structure under which information will be released. This study explores the problems of geographical and data-structuring differences by comparing ACS estimates to locally-derived transportation data that planners are accustomed to using.

Importance of Census Data to Transportation Planning

State and metropolitan planning organizations rely on data from the decennial census for a broad array of applications. Data from the long-form questionnaire, which includes questions covering place of work; mode of transportation to work; carpooling; travel time and time of departure to work;vehicles available; and mobility limitations, are used for planning and modeling travel behavior. These data have been summarized for some state and local areas in the Census Transportation Planning Package (CTTP). Travel demand forecasting models and other transportation planning activities apply these data to estimate the impact of transportation and land use policy on the transportation system. The Transportation Efficiency Act for the 21st Century (TEA-21), the Clean Air Act Amendments of 1990 (CAA 1990), and the Americans with Disabilities Act all increase transportation planning requirements and related data requirements of states and metropolitan planning organizations (MPOs) (Implications of Continuous Measurement 1996).

Census transportation data can be used alone to provide descriptive analysis of work trip patterns and, when compared with previous census-year surveys, of how those work-trip patterns are changing over time. In combination with other data, journey-to-work data supports a region’s travel demand forecasting package providing input on workplace attractions, work-trip origin-destination distribution patterns, work-trip departure times, work-trip length distribution, and travel-time distribution. To smaller metropolitan areas with limited resources, the survey becomes a foundation on which to build a model system, and in larger metropolitan areas the survey is a useful data base on observed travel behavior with which to calibrate or validate the model system.

The Census Bureau anticipates greater accuracy with continuous measurement in coding and interviewing due to permanent staffing, instead of the temporary staffing associated with taking the census every ten years (Implications of Continuous Measurement 1996). Because of the nature of data collection for the American Community Survey, there is high sampling error, so the data is released with caveats. The high sampling error is countered by moving averages, in which multi-year samples are averaged. Because census data would be collected over five years, instead of on the traditional "census day," the bureau will report "rolling averages" instead of numbers fixed in time.

This study extends the previous BTS study by focusing on the impact of the ACS on data for transportation planning and policy analysis. An analytical approach is taken to document the validity of estimates from the first year of the ACS demonstration project. This is accomplished by comparing 1996 ACS estimates of household characteristics and transportation variables for Multnomah County, Oregon to data developed locally by the Portland Metropolitan Service District (Portland Metro).

Because geographical analysis is a framework for much of the transportation planning activity of Portland Metro and is used to link various data sets, GIS has been selected as the primary methodology for this study. Zoning systems are the principal way in which spatial structure is incorporated into transportation planning models. A zone system is a set of contiguous polygons (areas defined by at least three sides). As this study is focused on evaluating estimates from the ACS, census block groups are considered to be fundamental reporting zones. Because census geography is defined independently of its attribute data, census zones are considered " artificial statistical areas," as the geography is unrelated to zone characteristics (Goodchild et al 1993). In this sense, census data models are different from those commonly used for transportation data, where the spatial basis is dependent on unique land use and transportation characteristics.

In contrast to census geography, the design of transportation zoning systems is viewed as an important geographical component of spatial analysis. Transportation Analysis Zone (TAZ) structures endeavor to create homogeneous areas of socioeconomic characteristics and tripmaking tendencies of the population. Portland Metro maintains a set of TAZs for use in planning studies. The zones in the TAZ system constitute the objects, or geographical individuals, that are the basic units for observation and measurement of spatial phenomena.. The current Portland Metro travel demand model forecasting system uses 1260 zones for the four-county area, 481 of which are Multnomah County TAZs and examined in detail for this study.

Figure 1 shows the census block group and TAZ coverages for Multnomah County. Since TAZs and census block groups have noncoterminous boundaries, this study develops spatial basis change methods to transfer socioeconomic data (developed locally at the MPO level) from its reporting zones (TAZs) to an independent set of reporting zones (census block groups). The decision to convert TAZ data to block group data was based on the ACS providing confidence intervals for its estimates. Unfortunately, the Portland Metro data set does not include this information. Only when the spatial basis of household and transportation data has been changed can a one-to-one direct comparative analysis be performed between locally derived data and census data.

Selection of Variables for Comparison

Residential trip generation rates are usually classified by household structure, household size, and auto ownership. These characteristics are important determinants of travel tendencies and are typically considered in trip rate analyses, trip generation studies, and transportation demand models. Unlike household activity surveys, which record information about trips for all travel purposes, the census only reports information about the journey to work. But the journey to work is significant because it represents about 25% of all daily trips. More work trips occur during peak travel periods and the average trip length is greater than that for other trip purposes. Destinations for the work trip are less diverse, but origin/destination data are more readily available than the origins and destinations for other trip types (Racca 1998).

This study compares the following variables that are commonly used in travel demand forecasting (Ortúzar and Willumsen 1990):

The selected variables’ definition and structure are different for ACS and Portland Metro data. Table 1 lists the eight variables chosen for comparison [in Column (1)], each variable’s strata in Portland Metro data [in Column (2)], and each variable’s strata in ACS data [in Column (3)].

The chief source of Portland Metro data is the 1994/1995 Household Activity Survey. Both temporal and categorical adjustments are necessary to compare the two data sets. First, all Portland Metro data is adjusted to its 1996 equivalent using linear interpolation based on growth factors from other sources. Next, the data are manipulated so that the strata for each variable are identical. Converting some of the variables, such as age of householder and household size requires fairly straightforward aggregation of categories. However, other variables, such as income and means of travel to work (which is the number of work trips by mode) require a great deal of manipulation to achieve similar categories. Some categories cannot be readily converted so there is also information loss in the conversion process. For example, there is no direct conversion of transit trips walk access and park and ride trips to public transportation because we do not know how many of the park and ride trips result in a carpool, vanpool, or public transportation. In this model, we assumed that anyone who carpooled identified their mode as shared ride trips and park and ride trips belong to public transportation. We could not use the variable, time leaving home to go to work because of the aggregate nature of the Portland Metro data. For the travel time to work variable, Census reports the number of households in each time interval. An average value was calculated for each block group by multiplying the mid range value by the number of households, summing then dividing by the number of households. The minimum range value was used for the last interval and the worked at home households were assigned a travel time of 0 minutes. After the adjustments have been made, spatial methods are used to interpolate Portland Metro block group level between the two data sets.

Spatial Methods

A literature review performed for this study found regional and geographic science literature dealing with the theory of spatial basis change (see Batty and Sammons 1978, Batty and Sikdar 1982, Bracken and Martin 1995, Fisher and Langford 1995, Flowerdew and Openshaw 1987, Flowerdew et al 1991, Goodchild et al 1993, Openshaw and Taylor 1981) and planning and engineering literature dealing with applications of spatial basis change (see Flood and Siaurusaitis 1997, Gan 1994, Hsiao et al 1997, Keck 1997, O’Neill et al 1992, Peng and Dueker 1993, Zhao 1999). Each allocation method has different data needs and results in errors and different levels of accuracy. The literature review and previous studies point the way toward a methodology for this study.

First, an area ratio basis change is performed as a baseline areal interpolation scenario. The results are a point of departure for comparing other methods. Second, an areal interpolation with an auxiliary variable (street length) is performed. Third, a set of control zones is designated that capture known residential settlement patterns (from land use information). Because of the rich data source from the local planning agency, two roadway networks representing different street types are used. In the third approach, the spatial basis change is enhanced by adding more information about the source zones. This approach represents an improvement over past studies. The three spatial methods used in this study are as follows.

ACS estimates are released in the form of a direct estimate as well as a lower and upper bound representing the 90 percent confidence interval. Portland Metro interpolated estimates at the block group level are compared to the ACS 90 percent confidence interval to see whether they lie below, within, or above the confidence interval. If the interpolated estimate falls within the ACS 90 percent confidence interval, the interpolated estimate is considered to be a good fit. The results of a block-group level analysis are summed for the entire county and shown in Table 2. The columns containing the percentage of interpolated estimates that lie within the ACS 90% confidence interval are highlighted.

Some differences in the ACS estimates and Portland Metro interpolated estimates can be explained by differences in source data. Table 3 gives the aggregate counts for Multnomah County variables from the ACS (for block groups) and Portland Metro (for TAZs) prior to any spatial modeling. Both data sets encompass all of Multnomah County, so the aggregate county sums for household characteristics and transportation variables can be compared to each other. Income ranges 1 and 4, and carpool, transit, and walk/bike means of transportation to work have large aggregate differences. This is partly attributable to temporal and structural changes made to the Portland Metro data to derive comparable data sets.

Both the area ratio and roadway coverage spatial basis change methods give different estimates for household characteristics and transportation variables. Error analysis (RMSE and MAE) of the results indicate that the control zone method outperforms the roadway coverage method and the roadway coverage method outperforms the area ratio method. The control zone method produces more block group-level estimates that fall within the ACS 90 percent confidence interval, and the roadway coverage estimates deviate less from the ACS direct estimate.

With regard to spatial analysis of the findings, in general there appears to be greater differences between the estimates of the two data sets on the periphery of Multnomah County, particularly the far northwestern and far eastern sections of the county. This is consistent with general findings of spatial interpolation, which performs poorest when the street pattern is irregular and/or the two zone systems are not of similarly sized zones.

The three methods used to perform a spatial basis change from TAZ to Block Group and the ACS direct estimates are compared using Analysis of Variance. ANOVA tests were performed for eighteen variables to compare the means of the data generated from the three spatial change methods and the ACS generated data to each other. The null hypothesis tested was:

H₀: m_acs = m₁ = m₂ = m₃

Where m_acs = mean of data from the 1996 ACS

m₁ = mean of data determined from the area ratio spatial basis change method

m₃ = mean of data determined from the control zone spatial basis change method.

Results of the ANOVA are shown in Table 4. Generally, the spatial basis change methods have the most impact on the transportation related variables. The null hypothesis is rejected for all transportation variables except vehicles available per household and two of the household income variables. For these variables, we conclude that the data resulting from the spatial conversion algorithms and the ACS data are not the same when the level of significance is 0.05.

For each variable for which the previous null hypothesis was rejected, a separate ANOVA was run comparing only the spatial basis change methods to each other. In other words, the null hypothesis is:

H₀: m₁ = m₂ = m₃

Results from this analysis are in Table 5. The spatial basis change methods are not significantly different for the two income ranges. However, there is a significant difference among the methods for the transportation variables.

Paired T-tests between the ACS data and the data generated using method 1, areal interpolation, on household income ranges 1 and 4 result in rejecting the null hypothesis that the means are equal at a level of significance = 0.05. The same holds true comparing the ACS income data to the results from the other two spatial basis change methods. Consequently, none of the spatial basis change methods are capable of generating data similar to that collected in the ACS for household income ranges 1 and 4. However, part of this conclusion may be a function of the data conversion methods used to achieve comparable categories in each database prior to applying spatial basis change. Particularly, temporal adjustment and linear interpolation between categories may have an effect on the data.

Paired T-tests were also performed for the transportation variables to determine if any of the methods resulted in values that were similar to the ACS. For total number of work trips, the hypothesis, that the difference in the means equals zero, is rejected when comparing each method individually to the ACS. However, when comparing data from the areal interpolation method (1) and the control zone method (3) we fail to reject the hypothesis that the means are equal. Similar conclusions are reached for the drive work trips, carpool work trips, transit work trips, and walk/bike work trips. The exception is average travel time to work. In this case, none of the methods compare favorably with the ACS and the null hypothesis for the test comparing Method 1 and Method 3 is rejected.

From a research perspective, the big question at hand is how reliable are the household and transportation estimates of the ACS? For most household characteristics, local data falls within the ACS 90 percent confidence interval 80 to 90 percent of the time. For most transportation variables, local data falls within the ACS 90 percent confidence interval 75 to 90 percent of the time.

Additional analysis reveals that the three spatial basis change methods generate comparable data to the ACS estimates for the socio-demographic data except certain income ranges. The three methods do generate data in these income ranges that are comparable with each other. None of the methods produces data comparable to each other or the ACS data for the transportation variables except average number of vehicles available to a household. However, for the remaining transportation variables, with the exception of average travel time to work, method 1 and method 3 produce comparable results by block group. Consequently areal interpolation yields similar results to the control zone method for all but one variable.

None of the methods are sufficient for reproducing transportation data similar to the ACS estimates. There are a few reasons this may occur. First, the Portland Metro data may be dissimilar to the ACS estimates given the different survey methods used. Additionally, no information on the sampling error associated with the Portland Metro survey is provided. According to Keith Lawton at Portland Metro, the activity-based survey in Portland undercounts the people who are too busy with activities to answer the CATI but there is no published information on the sources and sizes of sampling errors. Second, the transportation variables all required some type of transformation to generate similar categories for comparing ACS data to Portland Metro data. These transformations, including temporal, introduce error into the data that may be significant enough to impact the results from the spatial basis change methods. Third, none of the spatial basis change methods may capture the underlying geographical influences to adequately convert the data. For example, the number of work trips by transit is influenced more by accessibility (proximity) to transit services than the ubiquity of the street network. Also, carpool trips to work are greatly influenced by the proximity of those living near you or in your travel corridor who also work near you or in your travel corridor. None of these spatial relationships are considered by the spatial basis change methods described here.

Further consideration of the impacts on transforming variable types is certainly recommended. Unlike the geography problem, which will disappear when the journey to work coding of the ACS is performed on TAZs, this problem on characterization of data by Census and transportation planners will not change until planning models change. Consequently, planners will have to transform ACS data into the structure with which they are used to working.

Zhao, Fang. 1999. GIS Analysis of the Impact of Community Design on Transit Accessibility. (Working paper.) Miami, Florida: Florida International University.