My Account: Log In | Join | Renew

Natural Sciences Education - Article

 

 

This article in NSE

  1. Vol. 42 No. 1, p. 98-103
    OPEN ACCESS
     
    Received: Nov 27, 2012
    Published: September 23, 2013


    * Corresponding author(s): bmahler@uidaho.edu
 View
 Download
 Alerts
 Permissions
 Share

doi:10.4195/nse.2012.0025

The National Water Survey Needs Assessment Program

  1. Robert L. Mahler *a,
  2. Michael D. Smolenb,
  3. Tatiana Borisovac,
  4. Diane E. Boellstorffd,
  5. Damian C. Adamse and
  6. Nicola W. Sochackad
  1. a Soil Science Division, PSES 442339, Univ. of Idaho, Moscow, ID 83844-2339
    b Biosystems and Agricultural Engineering Dep., Oklahoma State Univ., 111 Agricultural Hall, Stillwater, OK 74078-6016
    c Food and Resource Economics Dep., 1097 McCarty Hall B, P.O. Box 110240 IFAS, Univ. of Florida, Gainesville, FL 32611-0240
    d Dep. of Soil and Crop Sciences, Texas A&M AgriLife Extension Service, Texas A&M Univ., College Station, TX 77843-2474
    e School of Forest Resources and Conservation, and Food and Resource Economics Dep., P.O. Box 110410 IFAS, Univ. of Florida, Gainesville, FL 32611-0410
    d College of Engineering, Driftmier Engineering Center, Univ. of Georgia, 597 DW Brooks Drive, Athens, GA 30602

Abstract

Water quality and availability are identified as priority areas for outreach and educational programs in many U.S. regions. This project offers a needs assessment tool (survey) that could be used on a state-by-state, regional, or national basis to analyze public attitudes, opinions, and behaviors as related to water resource issues. The information collected through the needs assessment can be used to identify objectives, strategies, and delivery modes for extension programs, and to evaluate the programs. The goal was to implement the needs assessment survey at 5-year intervals to: (1) measure changes in public priorities; (2) evaluate the effectiveness of educational programs; and (3) reprioritize, as necessary, the educational goals. A needs assessment survey template was developed and piloted in the Pacific Northwest. During 2001–2010, surveys were conducted in 41 states and six Pacific Island entities. Almost 11,000 surveys were returned completed for a response rate of 44.5%. Five-year follow-up surveys have been completed in five states to date. The results of these surveys have been used to prioritize extension water programs and compare issues and needs across and within regions. It is anticipated that some form of this survey will be conducted at 5-year time intervals until water is no longer of national concern.


Impact Statement

The surveys conducted through this national project provide 485,000 base line data points from 41 states and six entities associated with the United States. This base line data can be used in conjunction with future surveys to measure public change in attitudes and actions taken to protect water resources. This is the largest database of its kind and is a valuable resource for extension educators to measure the impacts of water educational programs over time.

About the author…

Robert L. Mahler is a professor of soil and environmental sciences at the University of Idaho. He teaches courses in Soil Fertility and Plant Nutrition and in Drinking Water and Human Health. He also teaches the largest science class on campus: Introduction to Environmental Science. Since 1980, more than 8,000 students have taken his classes. In addition to resident instruction, he is the University of Idaho’s Extension Water Resource Coordinator. His teaching (both resident and extension) interests center on voluntary changes made by the public to address water resource issues.

Water is widely regarded as the most important natural resource in the United States. Consequently, the USDA, land-grant universities, and the Extension Service have treated water resources as a priority issue since the 1980s (Shepard, 2002; Huter et al., 1999). In 2000, the USDA/CSREES (Cooperative State Research, Education, and Extension Service) National Water Program was revised as mandated by Section 406 of the Agricultural Research, Extension, and Education Reform Act of 1998 (AREERA). This resulted in the land-grant institutions (LGIs) developing regional and multistate, integrated research and extension projects to address the water resource educational needs of their clientele on a larger scale than state-by-state.

The development of large-scale AREERA-funded programs called for a new paradigm for identifying urgent water resource issues and designing and managing multi-state water resource programs. Traditionally, many scientists, policy makers, and educators at LGIs have used a standard approach of identifying critical water resource issues through their day-to-day contact with state agencies and clientele. However, some programs developed in this way have been found not to be effective, producing little to show in the way of behavioral change (Mahler et al., 2005). Large-scale AREERA-funded programs required new methods for examining similarities and differences in public concerns and informational needs among the states and regions.

Consequently, early in the national AREERA-funded water program (section 406), state extension water program coordinators recognized the need to develop a needs assessment tool to identify the public’s understanding, attitudes, and behaviors regarding various water resource issues on a state-by-state and regional basis. The needs assessment project originally focused on the states in the Pacific Northwest Region, then extended it, region by region, to states across the United States.

The objectives of this national needs assessment survey project were (1) to develop a survey tool that could be used to collect baseline information about public attitudes, aptitudes, and actions taken toward water resource issues on a state-by-state or regional basis; (2) to use the survey to collect the baseline information and to make this information available to extension educators, environmental agencies, and other stakeholder groups; and (3) to use this baseline information to develop programming priority areas and evaluate the impacts of water resources educational programs. It was envisioned that the project would be implemented periodically at 5-year intervals to (1) measure changes in public priorities over time, (2) evaluate the effectiveness of educational programs, and (3) reprioritize water educational goals.

Below we discuss the development of the needs assessment tool and summarize the key applications of the tool by members of the Water Policy and Economics Team of the USDA and LGI Southern Region Water Program.


MATERIALS AND METHODS

The needs assessment project was initiated in the Pacific Northwest Region (PNW, including AK, ID, OR, and WA) in 2001 to assess public aptitudes, attitudes, and actions relative to water issues. The project team consisted of the Extension Water Resource Coordinators from the University of Alaska, the University of Idaho, Oregon State University, Washington State University, and the Northwest Indian College. The team identified a mail public survey as the primary tool to be used to collect comprehensive information about aptitudes, attitudes, and actions from diverse socio-demographic and interest groups within the states and the region. The survey questionnaire was designed using the following 10-step process:

Step 1. Ground Rules and Focus: Coordinators agreed that survey questions had to address one or more of the eight USDA/CSREES national water priorities: (1) animal waste management; (2) drinking water and human health; (3) environmental restoration; (4) nutrient and pesticide management; (5) watershed management; (6) pollution assessment and prevention; (7) water conservation and management; and/or (8) water policy and economics.

Step 2. Submission of Questions: Each coordinator contributed a set of appropriate questions.

Step 3. Draft Questionnaire: The coordinators met to review the proposed questions and identify gaps where additional questions were needed. The outcome of this meeting was a list of 60 potential survey questions.

Step 4. Local Review: The coordinators from each state met with their state USDA/NRCS (Natural Resource Conservation Service) technical group for input and recommended questions.

Step 5. Survey Instrument Revision: The coordinators collated the inputs from USDA/NRCS and produced a revised set of survey questions.

Step 6. Review by the USEPA: The coordinators took the draft survey to the EPA Region 10 office in Seattle, WA, for additional input. Suggestions from this meeting were incorporated into the survey draft.

Step 7. Survey Instrument Modification: Selected LGI rural sociologists reviewed the revised survey draft. Based on this input the next draft was prepared.

Step 8. Approval for Human Subjects Research: The revised survey was submitted to the University of Idaho Human Assurance Committee for IRB approval. Human assurance approval was obtained from the University of Idaho as the USDA/NIFA 406 grant awardee for no. 2004-51330-02245, Coordination of Water Quality Programs in the Pacific Northwest.

Step 9. Pre-Test the Survey Instrument: A group of 20 individuals was selected as representatives of the general public to pre-test the survey instrument and help improve the wording of questions.

Step 10. Final Survey Instrument: Comments and suggestions received during the pre-test were incorporated into the final version of the survey that was then prepared for mailing.

The final 50-question survey instrument included five main sections: (1) public attitudes related to the importance of various water quality issues and water use categories (consisting of 5-level Likert scale questions with answer choices ranging from “not important” to “very important”); (2) public opinions about water quality and availability (multiple-choice questions focused on the overall water quality and availability, as well as the specific pollutants and pollution sources); (3) public evaluation of government performance in protecting water resources (multiple-choice questions asking about the performance of and trust in the different levels of government); (4) water resource information sources, topics of interest, and the preferred information delivery modes (close-ended questions asking respondents to check applicable informational sources, topics, and learning opportunities from the response choices offered); and (5) behavioral changes related to the protection of water resources (multiple-choice questions about respondents’ willingness to pay for environmental problems and to change those practices that could affect water resources). The final section of the survey focused on demographics (including gender, age, education level, community size, and duration of residence in respondents’ states).

This survey was mailed to a randomly selected sample of PNW residents following the procedures recommended in Dillman (2000) and Salant and Dillman (1994). Specifically, the sample size of approximately 900 completed surveys was selected to achieve the target margin of error in responses of 5% (Dillman, 2000; Salant and Dillman, 1994). Assuming a 50% survey response rate, the required mail-out sample was set at approximately 1800 residential addresses. To define the sample size for the individual states in the region, the most recent U.S. Census population estimates were used for every state (rounded to the nearest 10,000). Each state was allocated 200 surveys for a base population of 500,000 people. An additional 25 surveys were added per any additional 250,000 people. A random sample of residential addresses was generated using state telephone books.

A four-stage mailing procedure was used to obtain a high response rate (Dillman, 2000). The first mailing to residents in the PNW states took place in January 2002, and included the survey, a self-addressed business reply envelope, and a cover letter that: (1) identified the survey’s authors; (2) explained the purpose of the survey; (3) assured the respondents of anonymity; and (4) asked the respondents to fill out and return the survey via the self-addressed pre-paid business reply envelope. The first survey mailing was followed by a reminder postcard. Approximately 20 days following the reminder postcard, a letter with a second survey was sent to those who had not yet responded. About 3 weeks later, a second reminder postcard was mailed to non-responders. A similar process was followed in each state as the survey was administered in the PNW (and then across the country), always using a local contact in the cover letter.

Survey answers were coded and entered into Microsoft Excel files. Overall, a 46.3% response rate was achieved. The data were then copied to SPSS, a statistical software package (Norusis, 1986). Using SPSS, a basic data summary was developed. This summary indicated both the total number and percentage of respondents to each question. Next, cross-tabulation, or contingency tables, were developed to isolate how subgroups of survey takers (e.g., those defined by demographic characteristics, such as gender and education levels) responded to specific questions. Statistical significance of the differences in responses observed among the subgroups of respondents was tested using a chi-square distribution (Babbie, 1983, p. 359, 424). The significance level deemed valid in this study was p = 0.05. The survey results were shared with extension educators and policy makers in each state.

Needs Assessment Public Survey in Western States. Following completion of the first round of surveys in the PNW, several state water extension leaders in the western United States expressed interest in conducting similar needs assessment surveys in their own states. Based on this interest, PNW extension water educators partnered with educators in the Pacific Southwest (AZ, NV, CA, HI) and the Rocky Mountain/Great Plains (UT, WY, CO, MT, ND, SD) regions to develop similar public surveys. After extensive discussions, a survey questionnaire consisting of many questions identical to those asked in the PNW (50%) as well as questions pertinent to specific states and regions (50%) was developed. In 2004, the survey was administered by mail following a four-stage procedure similar that used in PNW. A random sample of residential addresses was purchased from Survey Sampling International (Shelton, CT).

Needs Assessment Public Survey in Other Regions. Following the success of the National Water Needs survey project implemented in 2001 to 2004 in the three regions, public surveys were subsequently co-developed with extension water leaders for the Northeast (MA, ME, RI, CT, VT, NH); North Atlantic (NY, NJ); Mid-Atlantic (WV, VA, MD, DE, PA); Southeast (SC, GA, FL, AL, TN); South-Central (TX, OK, AK, LA); and Midwestern (IA, NE, KS, MO) states (Table 1). Similar to the surveys developed for the Rocky Mountains/Great Plains and Pacific Southwest, these surveys drew on the PNW survey template, with a mix of questions used across the country (50%) and regional interest (50%). The survey logistics, data collection, and data analysis methods used in these regions were identical to the methods used in the Rocky Mountains/Great Plains and Pacific Southwest.


View Full Table | Close Full ViewTable 1.

The number of surveys mailed out, returned completed, and percent returned from the states participating in the water resources needs assessment survey.

 
State Survey date Surveys mailed Surveys returned Percent returned
Alabama 2009 610 291 48
Alaska 2002 215 112 52
Arkansas 2008 425 257 60
Arizona 2004 700 370 53
California 2004 2,400 988 41
Colorado 2004 600 309 52
Connecticut 2005 498 226 45
Delaware 2006 230 94 41
Florida 2008 1,150 523 45
Georgia 2010 1,100 520 47
Hawaii 2004 420 163 39
Idaho 2002 281 160 57
Iowa 2006 450 273 61
Kansas 2006 425 218 51
Louisiana 2008 600 251 42
Maine 2005 282 142 50
Maryland 2006 600 257 43
Massachusetts 2005 794 294 37
Mississippi 2009 440 284 64
Missouri 2006 725 359 50
Montana 2004 250 134 54
Nebraska 2006 325 209 64
Nevada 2004 616 243 39
New Hampshire 2005 280 119 43
New Jersey 2006 750 278 37
New Mexico 2005 350 177 51
New York 2006 1,075 413 38
North Dakota 2004 225 141 63
Oklahoma 2008 512 264 52
Oregon 2002 476 243 51
Pennsylvania 2006 875 394 45
Rhode Island 2005 258 128 50
South Dakota 2004 250 130 52
Tennessee 2008 700 354 51
Texas 2008 1,275 419 33
Utah 2004 400 235 59
Vermont 2005 212 109 51
Virginia 2006 700 269 38
Washington 2002 715 372 52
West Virginia 2006 325 143 44
Wyoming 2004 200 138 69
41 states 23,714 10,984 46.3

Needs Assessment Public Survey in U.S. Territories. The survey questionnaire used in Hawaii was also used in the six island entities associated with the United States—Republic of Palau (Palau), Federated States of Micronesia (FSM), Republic of the Marshall Islands (RMI), American Samoa (AS), Guam, and the Commonwealth of the Northern Mariana Islands (CNMI). Because several languages were spoken in these entities, it was decided that the survey would be administered in-person by trained individuals to 50 randomly chosen people on each entity.

Five-Year Follow-Up Survey. Follow-up needs assessment surveys have been completed in five states (AK, HI, ID, OR, WA) (Table 2). These surveys contained a combination of questions asked in the initial needs assessment survey and new questions (developed through discussions with state water quality coordinators) to determine what action(s), if any, the public had taken to address water issues in the last 5 years, and how public opinions and concerns related to water resources had changed.


View Full Table | Close Full ViewTable 2.

The number of surveys mailed out, returned completed, and percent returned from the states participating in the 5-year follow-up water resources needs assessment survey.

 
State Survey date Surveys mailed Surveys returned Percent returned
Alaska 2007 220 111 50
Hawaii 2010 480 258 54
Idaho 2007 330 198 60
Oregon 2007 547 276 50
Washington 2007 844 427 51

RESULTS AND DISCUSSION

A total of 23,714 needs assessment surveys have been mailed to residents of 41 states between 2001 and 2010 (Table 1). Almost 11,000 surveys were returned completed for a response rate of 46.3%. The response rates in states ranged from 33 to 67% (Table 1). States with response rates less than 40% included Hawaii, Massachusetts, Nevada, New Jersey, New York, Texas, and Virginia. States with response rates above 58% included Arkansas, Mississippi, Nebraska, North Dakota, Utah, and Wyoming. In general, states with the higher populations had the lowest response rates, whereas less populated states had higher response rates. In addition, response rates were much higher from Western and Midwestern states than from states in the South, East, and Northeast. A total of approximately 485,000 data points (average of 77 answers per survey multiplied by 10,984 surveys) have been collected to date.

We attribute the relatively high response rate (46.3%) in this mail-based survey to the following three factors. First, most of the mailing lists were obtained from a professional social survey company; second, the Dillman survey methodology (Dillman, 2000) was diligently followed; and third, American adults appeared to be genuinely interested in water issues, thus resulting in a high degree of “buy-in” to complete and return the survey.

Demographics. Relevant demographic information, including age, gender, education, and spatial information down to zip code, was collected from each completed survey (Table 3). In general, the population surveyed was somewhat older, better educated, and more urban than average, but all sectors of the over 18-year-old population were represented in statistically significant numbers. Even though the sampled population was designed to be split evenly between genders, males were more likely to complete this needs assessment survey. This gender difference was observed in all 41 participating states.


View Full Table | Close Full ViewTable 3.

Demographic characteristics of the 10,984 people who completed water needs assessment surveys in 41 states between 2002 and 2010.

 
Demographic Category Percent of survey takers
Gender
Male 65.0
Female 35.0
Age
<30 5.0
30 to 39 12.1
40 to 49 18.2
50 to 59 23.3
60 to 69 19.1
70 to 79 15.2
80+ 7.1
Education
Less than high school 4.7
High school graduate 16.2
Some college 30.0
College graduate 27.8
Post graduate course work 21.3
Community size >100,000 32.1
25,000 to 99,999 28.6
7,000 to 24,999 18.0
3,500 to 6,999 9.2
<3,500 12.1

Almost 50% of survey respondents had received a 4-year college degree. This is much higher than the U.S. population in general (Table 3); however, all levels of education were statistically represented in this survey. More than 60% of the 10,984 survey respondents resided in towns of 25,000 or more people. All community sizes were well-represented in this national survey.

Past Uses of the Responses Collected through the Needs Assessment Project. Many of the results from individual state and regional needs assessment surveys have already been published (Mahler et al., 2004, 2005, 2008a, 2008b, 2010; Clay et al., 2007; Wawrzynek et al., 2007; Hu et al., 2011; Borisova et al., 2012; Brauer et al., 2012). Specifically, these publications have evaluated water needs on a state (e.g., Brauer et al., 2012), regional (e.g., Mahler et al. 2004, 2005), or national basis (e.g., Hu et al. 2011), and examined a variety of water themes, including drinking water (Brauer et al., 2012; Hu et al. 2011), water information sources (Clay et al., 2007, Mahler, 2009), water priority issues (Mahler et al. 2004, 2005), and actions taken by the public to address priority issues (Wawrzynek et al., 2007; Borisova et al., 2012). Two papers have been published integrating the results from the 5-year follow-up surveys, focusing on the change in public attitudes, opinions, and behavior over time (Brauer et al., 2012; Mahler et al., 2010).

Very few or no water needs assessments had been previously implemented in the six Pacific Island entities (CNMI, RMI, FSM, Palau, AS, and Guam). Hence, the needs assessment survey resulted in a unique dataset that allowed extension to establish initial water educational programs on these islands (Table 4). Each island had different water needs. For instance, on RMI, the need was water quantity because all drinking water is collected from rainfall. Salt-water intrusion was the primary issue on CMNI, whereas groundwater protection was the priority on Guam. Even though the number of responses from each island was too small to conduct statistical analysis, the results are informative and they have been used by governmental officials to proactively deal with citizen-identified issues of concern (Mahler et al., 2008).


View Full Table | Close Full ViewTable 4.

Specific water educational programs recommended for development and implementation on the six U.S. Pacific Island entities based on the orally administered water needs assessment survey.

 
Island entity† Program recommendation
AS Address beach, shoreline, and coastal water pollution
CNMI Address salt water intrusion and water quality
FSM Address wetlands, drinking water, salt water intrusion, beaches, and shoreline protection
Guam Address groundwater protection
Palau Address the importance of water for tourism
RMI Address water quantity and water conservation
AS, American Samoa; CNMI, Commonwealth of the Northern Mariana Islands; FSM, Federated States of Micronesia; RMI, Republic of the Marshall Islands.

Current Series of Articles. This series includes four articles that use the needs assessment survey responses to evaluate education programs and to examine the determinants of attitudes, perceptions, and behaviors related to water resources. The series considers a wider range of states in different regions and delves deeper into the implication of survey results than previous studies that have used the needs assessment survey.

The four articles presented in this issue address three basic themes: (1) outreach program evaluation (i.e., the reach of outreach programs and the association between outreach programs and behavioral changes); (2) identification of the issues of public concern (specifically, water availability, water quality, and global climate change); and (3) outreach and policy program targeting (using audience demographic characteristics, urban or rural character of the communities, and timing around the periods of higher public concern, e.g., times of drought).

Two articles in this series are devoted to the first theme (outreach program evaluation). Specifically, Boellstorff et al. (2013) used the Needs Assessment Survey to examine the reach of water resource extension programs into rural and urban populations, assessing interest in water resource topics and preferences for various modes of information delivery. They found that both traditional (rural) and new (urban) audiences have high interest in water resource–related topics and that extension water resource programs reached 15% or more of the adult population in the Southern, Mid-Atlantic, and Northeastern states. In turn, Adams et al. (2013) evaluated the impact of knowledge-based water conservation programs (including traditional education programs) vs. non-knowledge-based factors on peoples’ water conserving behaviors. The article shows that receiving water resource information from extension or environmental groups through active learning increases the likelihood of adopting outdoor water conservation, whereas passive learning such as from newspapers and TV is associated with lower outdoor water conservation. Non-knowledge-based factors (such as views of government and the perceived importance of water resources as well as uses of water resources) also have a significant influence on water conserving behavior.

For the second theme (i.e., identification of the issues of public concern), Borisova et al. (2013) focus on issues related to surface water quality issues and examine public opinions and concerns about water availability. This article shows that many needs assessment survey respondents perceive surface water to be of good or excellent quality while suspecting or knowing that nutrients are negatively affecting their surface or groundwater quality. This article also show a high level of concern about current and future water availability and the increasing likelihood of droughts.

The articles in this series also demonstrate the importance of water resource extension programs to be tailored to the specific audiences. Specifically, urban or rural characteristics of the community of residence (along with demographics) are shown to significantly correlate with the preferences for information delivery modes (Boellstorff et al., 2013), water resource topics of interest (Boellstorff et al., 2013), and the level of concerns about water quality (Borisova et al., 2013) and availability (Adams et al., 2013).

CONCLUSIONS AND FUTURE WORK

The national water survey program has provided a tremendous amount of data showing how the public feels about its water resources and the effectiveness of educational tools for water resource education. These data have already had a significant impact on extension programming in more than 40 states, and offer the opportunity for further impact. Overall, 485,000 data points across the 41 sampled states allowed us to learn much about the public attitudes, opinions, and behavior as related to water resources in the United States. This database is large enough to make comparisons between states and regions. More importantly, it can allow us to make comparisons over time. Consequently, water resource survey data collected in the future will be much more meaningful if it is compared to this initial database to examine how public opinion has changed over time, and to develop sound public policy for water resources—both water quality and water quantity. Future follow-up surveys will further enrich this dataset and allow educators to have a major impact on U.S. water resource issues.

The fact that this program has so many extension educators talking to each other about common educational targets for water resource issues and the apparent similarities between individual state, regional, and national needs suggest a strong and successful contribution of the Needs Assessment Survey project to LGI extension, research, and educational missions. Many educators and researchers have joined forces across state boundaries to compile, analyze, and interpret these water resource data to make them useful for enlightened decision making.

The 20-year follow-up water needs assessment survey (a compilation of four 5-year follow-up surveys) in 2022 is expected to be conducted in the Pacific Northwest, where the program first began. Programs developed for the public in 2023 will be based, in part, on historical baseline information that was first collected in 2002. It is anticipated that many of the other 36 states will conduct a 5-year follow-up survey using the template designed in this effort. The Pacific Northwest states are currently analyzing the data from their 10-year follow-up survey that was conducted in early 2012.

About the author…

Robert L. Mahler is a professor of soil and environmental sciences at the University of Idaho. He teaches courses in Soil Fertility and Plant Nutrition and in Drinking Water and Human Health. He also teaches the largest science class on campus: Introduction to Environmental Science. Since 1980, more than 8,000 students have taken his classes. In addition to resident instruction, he is the University of Idaho’s Extension Water Resource Coordinator. His teaching (both resident and extension) interests center on voluntary changes made by the public to address water resource issues.

 

References

Footnotes


Comments
Be the first to comment.



Please log in to post a comment.
*Society members, certified professionals, and authors are permitted to comment.