http://aje.sagepub.comAmerican Journal of Evaluation

American Journal of Evaluation,Vol. 29 No. 1, March 2008 73-85
DOI:10.1177/1098214007313023
© 2008 American Evaluation Association
Ethical Dilemmas in Evaluations Using
Indigenous Research Workers
Leslie B. Alexander
Bryn Mawr College
Kenneth A. Richman
Massachusetts College of Pharmacy and Health Sciences
Abstract:This article addresses ethical dilemmas experienced by street-level research and evaluation
workers recruiting and gathering data in community-based research projects. The authors focus on a
subgroup of street-level research workers, whom they call research extenders (REs), employed
because they share important characteristics with the target population. Like other street-level
research workers, some REs are single role (only do research work) and others are dual role (do
research work and provide services). They discuss relevant literature and findings from pilot focus
groups exploring how REs understand responsible ethical conduct in their research work in the
community. REs face ethical issues somewhat different from those of more traditional, non-
indigenous street-level research workers. These issues increase social risk for study participants and
could compromise the validity of data. Researchers need to be aware of and address the distinctive
set of ethical issues raised by research and evaluation projects employing indigenous, street-level
research workers.
Keywords: ethical dilemmas; indigenous research workers; research extenders

To make research and evaluation results more relevant for decision making and to increase
the external validity of findings (Green & Glasgow, 2006), recent federal initiatives have
expanded research on health disparities as well as community participatory and translational
research. Translational research refers to “the systematic translation of evidence-based
research findings, tools, and information into practice”(Sussman,Valente, Rohrbach, Skara,
& Pentz, 2006, p. 7). The target research populations are often hard to reach and are under-
studied, which makes them highly relevant to health disparities and to community participa-
tory and translational research. The populations may include those with HIV/AIDS and IV
drug problems; those who have experienced street or domestic violence; those in maternal and
child care programs who are low income and from minority backgrounds; and individuals
with a range of physical health problems, including diabetes, cancer, asthma, and sexually
transmitted diseases. Research teams in community-based evaluation projects may include
street-level, frontline research workers, whose primary tasks are to recruit participants and
gather data (via questionnaires or in-person interviews). Some of the frontline workers provide
social services or interventions in addition to recruiting and collecting data from the target
population. This article focuses on the ethical dilemmas experienced by these research workers.
© 2008 American Evaluation Association.

The article has two goals. First, we review literature about ethical dilemmas in evaluation,
for which a subgroup of street-level research workers, whom we call research extenders
(REs),is at risk. This subgroup is made up of research workers who are indigenous to the pop-
ulation being studied. By indigenous, we mean that REs share important characteristics with
research participants, including living in the same neighborhood, having similar cultural
understandings about life, being of the same social class and educational background (typi-
cally community college or less), and/or having personal experience with the focal problem
addressed by an evaluation project. Ethical issues that REs face are embedded within a social
and experiential context somewhat different from that experienced by the traditional street-
level research workers, who work with similar target research populations but are not indige-
nous to the target study population. REs are the ultimate insiders who, because of their
familiarity with the social and physical environments of the target research population, pos-
sess contextual knowledge that provides “improved access to an otherwise highly restricted
group, enhanced rapport-building capabilities, and greater understanding of the language
unique to the study population”(Berg et al., 2004, p. 87). At the same time, these common-
alities have the potential to increase social risk for study participants and could compromise
the validity of data.
Second, we present findings from four pilot focus groups that explore how REs understand
responsible ethical conduct in their research work in the community. These pilot data suggest that
employing REs may raise a distinctive set of ethical issues that evaluators need to be aware of
and address, for which there currently is minimal formal guidance or evaluation. These pilot
research data were collected in 2003 and 2004 to develop content for Web training for REs,under
a contract with the Office of Research Integrity (ORI), U.S. Department of Health & Human
Services (see http://www.ori.dhhs.gov/education/products/mass_cphs/training_staff/index.htm
for the training material).
Throughout the article, we use the term research instead of or in conjunction with evalu-
ation because our focus groups and much of the literature address the ethical issues of REs in
a research context. This is driven to some extent by the definition given in the Code of Federal
Regulations (n.d.): “Research means a systematic investigation, including research develop-
ment, testing and evaluation, designed to develop or contribute to generalizable knowledge.”
If evaluation is done for internal purposes, that is, not intended to contribute to generalizable
knowledge, then it does not meet this definition and would not be subject to the regulations.
Nonetheless, as is reflected in the overlap between the American Evaluation Association’s
Guiding Principles for Evaluators and the principles that undergird the federal regulations
(see below), the ethical issues remain the same.
Literature Relevant to REs:
Boundaries and Ethical Issues
Although there is growing literature on methodological (Hildebrandt, 1996; Minkler &
Wallerstein, 2003; Silka, 2003) and ethical issues (Fadem et al., 2003; Harper et al., 2004;
Minkler,2005) in community participatory action research,this type of research should not be
confused with research carried out by REs. Israel et al.’s (2001) clarification of the distinction
is instructive:Whereas “community action research involves conducting research that recog-
nizes the community as a social and cultural entity with the active engagement and influence
of community members in all aspects of the research process,”researchina community con-
siders “community as a place or setting and allows community members limited, if any,
involvement in what is primarily a researcher-driven endeavor. Such an approach might be
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
more appropriately referred to as ‘community-placed research’”(p. 3). Although indigenous
researchers in community participatory research might face similar ethical dilemmas as
REs in community-placed research,this article focuses on indigenous workers in community-
placed research.
Dual-Role and Single-Role REs
As is true for street-level research workers more generally, REs can be divided into two
groups:single role (who only do research work) and dual role (who do research work and pro-
vide services). Community health workers have a variety of different roles and titles:At least
30 different synonyms are currently used to describe them (Andrews, Felton, Wewers, &
Heath, 2004). By performing service and outreach functions, they provide critical links
between health providers and the community precisely because of their indigenousness with
individuals in targeted communities. For example,promotorasare women from the Hispanic/
Latina community who are trained to provide accurate, culturally appropriate health infor-
mation. Although a developing literature describing the functions and effectiveness of com-
munity health workers exists (see Andrews et al.,2004,for a review),it is unknown how many
of these workers recruit and collect research data either as a primary job function, similar to
our single-role extenders, or in conjunction with other service and outreach functions, simi-
lar to our dual-role extenders. More than 10 years ago,Hill,Bone,and Butz (1996) described
ways that community health workers could and should be used to perform research as well as
service functions, but more recent literature directly addressing ethical conundrums that
might arise in combining the two functions is sparse. At the same time, the likelihood that
many community health workers also recruit and gather research data is high,given that most
are employed in projects funded by grants or contracts that require an evaluation component
for continued funding of the project. The literature on single-role REs is similarly meager.
Articles addressing the matching of research participants with interviewers having the same
presenting conditions are the primary source of information. The next section summarizes the
findings from the limited material on dual- and single-role REs, focusing on the advantages
and disadvantages of their use in research and evaluation.
Overall Advantages and Disadvantages of the Use of REs
The literature on the use of REs is mixed. On the positive side, research participants may
be more likely to disclose more of their real feelings to client interviewers than to staff inter-
viewers. For example, Clark, Scott, Boydell, and Goering (1999) developed a client satisfac-
tion interview for use with clients with major mental illness from two different outpatient
clinics. At each clinic, 60 clients were randomly assigned to be interviewed by either a client
or a staff member. Data collected by both types of interviewers revealed similarly high levels
of client satisfaction. However, client interviewers were also more likely to obtain extremely
negative responses on some of the items in the satisfaction survey from clients, suggesting
that research participants disclosed more of their real feelings to client than to staff inter-
viewers, thereby increasing the validity of the results. Perry and Felce’s (2004) study, which
matched research participants with intellectual disabilities to research interviewers who also
had intellectual difficulties, was equally positive, concluding that validity of data was not
compromised but rather enhanced by such practices.
Looking beyond the literature on matching interviewer and participant characteristics to that
more directly germane to REs, a number of positives have been noted about using REs.
Facilitating recruitment of research and evaluation participants is one advantage of using REs.
Results from an examination of several sites involved in the NIDA Cooperative Agreement for
Alexander, Richman / Ethical Dilemmas 75
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
AIDS Community-Based Outreach/Intervention Research Program (Cunningham-Williams
et al., 1999) revealed that, overall, the street outreach method, using single-role REs who
were recovering from drug abuse, was effective in recruiting and enrolling male, female, and
non-White drug users. Despite their overall positive finding,Cunningham-Williams et al. also
found cross-site and within-site differences in “levels of accuracy and in recruitment and
enrollment yields”(p. 1), suggesting the need for additional research to explore factors that
may have contributed to these differences. Simoni,Weinberg,and Nero (1999),writing about
training women who were seropositive for HIV to conduct survey interviews in their own
communities, provide additional support for the advantages of using single-role indigenous
REs. They pointed out that the women could recruit participants quickly, including partici-
pants who were not connected to clinics or the other usual services. They also found that the
REs gained valuable work experience and training through their participation in the research.
Using indigenous interviewers can “lessen the skepticism of community residents and feel-
ings of being ‘used’by academics”(Holbrook, Farrar, & Popkin, 2006, p. 781). These inter-
viewers may be viewed as more trustworthy, and participants may be more cooperative and
honest in their answers with them.
Although there is evidence that using REs has benefits for both the study and the REs
themselves, little work has been done that compares the effectiveness of indigenous and non-
indigenous research workers. In addition to the Clark et al. (1999) study of staff and client
interviewers (see above), we could locate only one other empirical study (Holbrook et al.,
2006). This exploratory study compared the quality of interviews conducted by indigenous
and nonindigenous interviewers in two public housing developments in Chicago and found
no difference between the two types of interviewers in the cooperation obtained or the hon-
esty of answers to sensitive questions. However, the number of interviewers was small and
confined to one particular context (Holbrook et al., 2006).
At the same time, there may be drawbacks to using indigenous recruiters and data gather-
ers. Geter (1995) pointed out that a potential downside in using “ex-addicts”as addict locators,
data collectors, and data validators was the potential for putting them at risk for relapse.
Furthermore,Simoni et al. (1999) reported interviewer attrition,often without informing study
supervisors in advance, and incompleteness and inaccuracy of some of the data collected.
Others have reported more missing and inaccurate data, including data recorded in a “format
that could not be coded (e.g. one wrote in ‘a bunch of times’instead of an actual number as
was requested)”(Fox, 1991, p. 87). Drawing on their experience using indigenous interview-
ers in a study of Latino men who have sex with men, Berg et al. (2004) reported challenges
with the interviewers’lack of previous experience in performing in-depth interviews and the
refusal of some participants to be interviewed by a member of their own community.
A disadvantage of using indigenous REs that is specific to those with dual roles is the dif-
ficulty they face maintaining a balance between following the research protocol and main-
taining confidentiality while continuing to function as outreach workers (Berg et al., 2004;
Fox, 1991). Reflecting on her experience working with dual-role REs, Fox noted that “hiring
mostly ‘insiders’or indigenous members to do outreach work can contribute to identity prob-
lems for workers embroiled in all-too-familiar street scenes day after day”(p. 234). She also
remarked on some of the conflicts (such as conflicts between helping individuals and col-
lecting valid data) particular to dual-role REs. As described below,some of these same issues
were also found in our pilot study.
The Specific Issue of Curbstoning
Curbstoning,a term used to characterize “fabrication or falsification of data ‘on the street’”
(ORI, 2006e, p. 1), has been declared an act of scientific misconduct under the statutory
76 American Journal of Evaluation / March 2008
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
authority of ORI since 2004. The Public Health Service (PHS) regulations “apply to anyone,
regardless of their position,who is involved in proposing,conducting or reporting of research
supported by PHS funds or proposed in applications for PHS funds”(ORI, 2004c, p. 4).The
ORI (2006e) Web site indicates that the PHS found research misconduct in several ORI cases
involving lower level staff (this includes anyone who administers questionnaires, conducts
surveys, or interviews research participants in person, over the phone, or through the use of a
computer interface). This Web site urges institutional officials not to purge tainted data
obtained through regular “quality control”measures, because some of these purged tainted
data might provide evidence of research misconduct at a later date.
Research misconduct clearly occurs in research and evaluation projects; however,unfunded
studies and those that are funded but not by PHS or other federal agencies are not subject to
investigation and oversight by ORI. The prevalence of curbstoning, independent of the par-
ticular research design or whether REs or nonindigenous research workers are used to recruit
participants or gather data, is simply unknown. What is known is that data falsification at any
level of the evaluation process raises not only quality control issues but also ethical ones.
A perusal of the Annual Reportsand quarterly Office of Research Integrity Newsletters is
instructive in terms of the types of curbstoning reported and who was responsible for it. For
example, a review of the 2006 issues of the Office of Research Integrity Newsletteridentifies
instances of curbstoning committed by a wide variety of research staff. These include two dif-
ferent cases in which former graduate students falsified or fabricated data,including pilot data
and questionnaires, and one in which a “research project coordinator”was found to have fal-
sified patient data,including interview data (ORI,2006a,p. 11). The June 2006 issue presents
the case of a site coordinator at a medical center who falsified test results to enroll “study sub-
jects who did not qualify under the particular study protocol,”resulting in the death of a
research participant (ORI, 2006b, p. 7). Falsification of human subjects’data by a PhD with
the title “professor of radiation oncology”is described in the September 2006 issue (ORI,
2006c, p. 9), and fabrication and falsification by a research assistant is described in the
December issue (ORI,2006d). This is just one year’s issues; case summaries like these appear
in every issue of the newsletter (see http://ori.dhhs.gov/publications/newsletters.shtml for all
the issues of the newsletter).
More specifically related to REs, in its March 2004 Newsletter, ORI reported that three
REs, hired as part-time workers by the Department of Pediatrics, University of Maryland,
Baltimore, engaged in different types of curbstoning, including claiming to collect data from
addresses where they never actually went, making up responses for individuals they never
actually interviewed,and making tape recordings of their own families and friends,who were
not enrolled in the study (ORI, 2004a, 2004b).
Ways to detect data falsification are beyond the scope of this article (see Survey Research
Laboratory,2004,for information on data falsification detection). Ways to minimize curbstoning
by indigenous and nonindigenous research workers alike include interviewer training,including
particular attention to falsification threats; careful,proactive attention and support from supervi-
sors; attention to interviewer motivation; and adequate payment and payment by the hour rather
than by completed interview. Design factors that may encourage falsification include interviewer
production bonuses, long data collection routines, difficult-to-locate research participants, short
field periods, and interviewers working off-site from the parent organization (see American
Association of Public Opinion Research,2003,for a set of useful guidelines).
Roth’s (1966) discussion more than 40 years ago of the dangers of using “hired hands”in
research is instructive in understanding why data falsification may occur for both indigenous
and nonindigenous research personnel recruiting and gathering data from hard-to-reach
populations in the community. Roth pointed out how the structural situations of such research
Alexander, Richman / Ethical Dilemmas 77
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
workers might unwittingly promote cutting corners,leading to inaccurate or suspect data. The
integral relationship between academic careers and research production, characteristic of PIs
and other key research personnel, does not extend to research workers, such as REs, student
assistants, or recent college graduates (Staggenborg, 1988). At the same time, these lay
research workers gain valuable work experience, which can translate into a stronger interest
in additional education and career planning.
Pilot Study:Focus Groups to Develop a Web-Based
Curriculum for Indigenous Workers
Responsible Conduct of Research:The Standard Framework
The ethical framework provided by the Belmont Report and the Common Rule (Code of
Federal Regulations, n.d., Subpart A) is the standard and accepted framework for under-
standing responsible conduct of research involving human subjects. These documents address
human subjects’protections explicitly. In the requirement that research be “consistent with
sound research design”(45 C.F.R. 46.111), they also carry implications for other areas of
responsible conduct, such as falsification of data (cf. Richman, 2002). The standard frame-
work is conventionally summarized in three principles:
1. The Principle of Respect for Persons holds that researchers have a duty not to interfere with each
participant’s right to choose what happens to him or her. Voluntariness of participation,informed
consent, and special protections for vulnerable populations fall under this principle.
2. The Principle of Beneficence holds that researchers have a duty to create good, generally in the
form of knowledge.
3. The Principle of Justice requires that burdens and benefits be distributed fairly. Unlike with pro-
vision of care, which assumes a direct benefit to the participant, research does not necessarily
benefit participants directly.
Training on the regulations and ethical guidelines for research is mandatory for all key per-
sonnel (i.e.,those involved in the design or conduct of the research; see http://cme.cancer.gov/
clinicaltrials/learning/humanparticipant-protections.asp for an example of acceptable, free
training). Such training emphasizes the principles of the Belmont Report as a way to help
researchers and evaluators prepare research protocols that are ethically responsible and appro-
priate for institutional review board (IRB) approval. Our pilot study suggests that some REs
are not asked to complete such formal training, which in any case is geared more to PIs and
senior research personnel. REs are expected to implement protocols designed by others. In
some cases where training had taken place, REs were somewhat vague about the content of
the training or what specific guidance they had taken from it.
Beginning Explorations:Initial Focus Groups
As mentioned earlier,we conducted a total of nine focus groups in 2003 and 2004 to explore
how REs understand responsible ethical conduct in their research work in the community. The
focus groups were part of a pilot study generated in the execution of a 12-month contract
in 2003 and 2004 with ORI as part of their Responsible Conduct of Research Resource
Development Program.1 The purpose of the contract was to develop Web-based training in
human subjects protections for REs who do research work in the community.
To develop appropriate and effective training materials, we held focus groups with research
staff at community agencies around Philadelphia. Four focus groups with research field workers
78 American Journal of Evaluation / March 2008
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
(including REs and some nonindigenous research workers, or NIRs) and one focus group with
research supervisors were held in January and February 2004. We also met with several super-
visors individually. Some of the field workers were similar to the research subjects they were
recruiting (e.g., recovering IV drug users working with current IV drug users), and some were
entry-level workers with college degrees, for whom this work was their first job. Some worked
full-time, some part-time. African Americans, Latinas, and Caucasians were represented. The
majority was female. Ages ranged from early 20s to mid-50s. The supervisor group was less
demographically diverse. Most of the participating supervisors had advanced degrees.
For the first round of focus groups,informed consent was obtained from participants to have
the group sessions audiotaped for review by the investigators and for use in developing training
materials. The project was not initially expected to yield important generalizable knowledge.
Therefore,participants in the first round of focus groups were not asked to consent to the use of
findings for research purposes. Using a series of open-ended questions, we obtained informa-
tion about the priorities, vocabulary, and experiences of REs in participating agencies. Among
other issues, the focus groups probed beliefs about the most important ethical issues in their
work with human subjects,ways to recruit research participants without coercing them,methods
for obtaining informed consent, and issues around the confidentiality of research data.
The investigators had planned to develop training materials organized along the lines of the
Belmont principles. However, following informal evaluation of the tapes of the first set of
focus groups, it became clear that this approach would be inappropriate for REs. Indeed, this
realization was striking enough that the authors decided to add a research component to the
second round of focus groups. The primary purpose of the round 2 focus groups was to review
a draft of the training Web site for feedback about its content and format. Approval was
granted by the Bryn Mawr College IRB to use data from the second round of focus groups
for research purposes. Participants in the second round of focus groups were asked to com-
plete a research consent form prior to the start of their group. To minimize coercion for par-
ticipation in research while still moving ahead with the work of developing the training
materials, we adopted a special consent procedure. We asked all participants to use a consent
form to indicate whether they agreed to have the data used for research. Each participant
placed a form in an envelope before the focus group began. We explained that the envelope
would not be opened until after the focus group and that if any one participant from a given
group declined to participate in this research,no data from that group’s session would be used
for research purposes. (The group would still have been useful for developing the training.)
As it happened, all participants consented to participate in the research.
Research Focus Groups
Sample
In the second round, three focus groups were held with REs and NIRs, and one with
research supervisors. Many of the participants had also been part of the first round focus
groups.2At the time of the second round,the authors had not come to appreciate possible dif-
ferences in response between REs and NIRs, so indigenous and nonindigenous workers were
mixed together. Demographic characteristics of the 12 REs and NIRS and the nine supervi-
sors who participated are provided in Tables 1 and 2. Because the total Ns were small and to
preserve the confidentiality of our participants, we did not nest the information about race
under the ethnic categories (e.g., the number of Black Hispanic females who participated).
Findings From RE/NIR Focus Groups
First, we identified themes related to the ethical conduct of research by REs and NIRs.
The major themes were rejection of strict rules, suspicions about the intentions behind and
Alexander, Richman / Ethical Dilemmas 79
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
appropriateness of research protocols, concern for the RE’s personal safety on the job, desire
to maintain existing relationships with community members, and worry about coercing vul-
nerable individuals. After a brief discussion of these themes and a comparison of the per-
spectives of the research supervisors and the research workers on the draft training materials,
this section concludes with some general considerations.
Rejection of strict rules. One reason that the Belmont principles seemed inappropriate as
an explicit framework for REs was a general disinclination to think in terms of rules. For
instance, a Latino male RE stated,
We think we can reduce everything to rules. And when we don’t, we make more rules. And then
we [all of us] have problems.
This same RE later elaborated,
You should include a section or chapter [in the training] called “cheating.”. . . The truth is,some-
times we have to cheat. . . . The kid is lost, you know, we don’t know where he is. . . . It could
be a life and death matter. Well, you know the confidentiality. . . . Bull! . . . Cheat. Find a way to
cheat that you do the least amount of harm. And cheat and get it done.
These statements indicate a rich set of obligations that do not fit neatly into the categories of
beneficence,respect for persons,and justice as they are described in the Belmont Report. The
underlying values were not seen as falling under general laws. Where a bioethicist might think
of an action not as cheating but as applying a rule in a less obvious way, this RE challenged
rules in general.
80 American Journal of Evaluation / March 2008
Table 1
Characteristics of Research Extender and Nonindigenous
Research Worker Focus Group Participants (n ==12)
Female Male
Ethnic category
Hispanic or Latino 1 1
Not Hispanic or Latino 8 2
Racial category
Asian 0 1
Black or African American 7 0
White 2 2
Total by gender 9 3
Table 2
Characteristics of Research Supervisor Focus Group Participants (n ==9)
Female Male
Ethnic category
Hispanic or Latino 1 1
Not Hispanic or Latino 5 2
Racial category
Asian 0 0
Black or African American 2 1
White 4 2
Total by gender 6 3
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
Suspicions about research protocols. Concern about the protocols for recruiting partici-
pants and collecting data was central in the remarks of both NIRs and REs. REs expressed
doubts about the importance of particular features of the protocol, considering some of them
unnecessary or manipulative. One White female RE was suspicious of protocols that required
REs to collect a lot of personal information such as addresses, telephone numbers, and con-
tact information for participants’parole officers. She remarked,
I still don’t care what they tell you. If I don’t like it,I ain’t doing it. . . . I’m not a compliant employee.
Some issues that experts in research integrity might view as falling under respect for per-
sons were described by REs as difficulties with following the protocol. For instance, when
asked about whether compensating participants could be unfairly coercive, one African
American female RE said,
Sometimes the money could be a good and a bad thing. . . . Sometimes there might be people who
might bombard you and they might not even be eligible for the study,but they want to get paid. . . .
They could be vulnerable,but it also can be dangerous for us sometimes.
She continued,
I’ve been in situations where people have become hostile . . . because they want the ten dollars . . .
[for participating]. A lady got upset [at an RE], spit on her, and threw a rock.
Concern for safety. As indicated by the previous quotation, REs also expressed concerns
for their personal safety. They described individuals who were aggressive toward them when
asked to wait their turn and others who became aggressive when they were turned away
because they wanted to get paid multiple times for participating in the same study. These dan-
gers affect RE safety and can also affect an RE’s ability to implement a protocol to collect
valid data. However,at least from the perspective of one RE,these dangers were not foremost
in the minds of those who supervise REs. Contrary to what he reported as instructions from
his supervisor,the RE said,“You don’t have to be nice to someone if someone is being nasty.”
Maintaining existing relationships. Some REs expressed strongly held views that surprised
the investigators. For instance,in discussing challenges to confidentiality that may arise when
collecting data from acquaintances, a White female RE remarked,
If she [an interviewer] knows me, she can’t ask me if I want to [participate]? Is that what you’re
saying? That’s, that’s kind of wrong. I think it should be left up to the person if they know some-
body. . . . I think I’d be pissed off if she said, because I know you I can’t go on with this.
Unlike the issues of safety or problems implementing the research protocol, the theme of
maintaining existing relationships indicates a nexus of values that may not arise for NIRs,
because they do not share similar geographic, cultural, and linguistic characteristics with the
research participants they recruit and gather data from. Their relationships with research par-
ticipants are clearly secondary relationships, with clearer boundaries.
Issues of coercion. Other issues raised by REs were directly related to standard issues such
as coercion and data integrity. A Latino male RE who worked in the gay community explained
that he avoids participants who express sexual attraction toward him:
The Yo, babykind of thing. . . . I don’t want this person telling me stories about how great they
are because they like me and want my phone number.
Alexander, Richman / Ethical Dilemmas 81
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
Dual-role REs expressed concerns about coercion, such as,
People may be afraid of not participating in the survey because somehow they think it might
affect the services they receive.
Issues of vocabulary arose, as well. For example, a White female RE asked for clarification
of a point about “vulnerable populations,”remarking, “When you hear the word vulnerable,
you think about being hurt emotionally.”
Reactions to draft training materials. To develop possible differences in perception between
supervisors and other professional researchers, prior to the second round of focus groups, pro-
fessional content experts with extensive experience with research in urban communities
responded to a draft of the training materials. In addition, the main activity in the second round
of focus groups was to view a draft of the training. Responses to the draft Web site provided
evidence of differences in thinking between supervisors and REs. For instance, an injunction
against REs giving rides to participants was greeted with warm recognition by supervisors but
with rejection by some REs. One RE who works with IV drug users reacted incredulously:
It’s pouring rain and you’re gonna tell somebody, “No, walk!”?
The same RE reacted similarly to the claim that REs should not give out personal information
such as their telephone numbers:
They need to eat, they want to get in a program. . . . That has nothing to do with what I’m doing
here [i.e., helping this person is unrelated to my research work and thus should be permitted].
This RE put concern for others’well-being ahead of maintaining professional distance.
Summary
The following lessons represent our overall impressions,based on data from the focus groups.
1. REs tended not to distinguish between ethical issues and other practical issues. When asked
about what ethical issues they face in research work, REs tended to respond with information
about a variety of practical challenges (e.g., contacting potential participants with no telephone
service), most without an obvious ethical component. It is not surprising that REs face practical
challenges in recruiting participants and collecting data; these present problems for research
staff at all levels, especially when working with hard-to-reach populations. However, we found
that the REs we spoke with tended to treat the ethically charged challenges and the practical
challenges as the same. This finding is directly relevant to our goal of learning about how REs
understand the ethical dimensions of their work and how their understandings relate to the stan-
dard framework. It reflects a difference between how the REs think about their work and how
higher level research personnel (PIs, co-PIs, and consultants) think about the same thing. It also
reflects the REs’relationship to their work. Unlike higher level research personnel, REs see
themselves as employees who are paid to do specific tasks, so that they view themselves as
ethically bound to recruit and/or collect data to meet the expectations and research goals of their
supervisors. This aspect of REs’experience is intensified by the fact that they are working with
vulnerable target populations that often are not easily accessible. In addition, they may be feeling
even more pressure because they were chosen due to their supposed familiarity (and hence facility)
with the target populations.
2. Many street-level research workers do not fall neatly into categories of indigenous/nonindigenous.
These focus groups brought us into contact with individuals serving the role of REs whose
education (a bachelor’s or master’s degree) or training gave them a perspective closer to that of
82 American Journal of Evaluation / March 2008
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
an NIR. In fact, this distinction of status and perspective was a finding from the focus groups
and necessitated careful reassessment of inclusion criteria for screening potential participants in
our subsequent research. It also suggests that evaluators should avoid assumptions about the
education level or prior research experience of indigenous workers.
3. Nonindigenous research workers were more likely to identify issues familiar to higher level research
personnel,such as data integrity and participant autonomy,than the REs in the same focus groups
were. NIRs were also more likely to report having had more formal training in research ethics.
Strengths and Limitations
For eliciting views on ethics of research-related behaviors, in this pilot work, focus group
questions about personal experiences and understandings of ethical issues in research elicited
more relevant and enlightening responses than the scenario-based interview questions.
Scenarios carefully constructed to portray behavior that could be defended or condemned on
ethical grounds prompted mostly direct condemnation of questionable behavior along with
reports that other REs (i.e., not the participant) do engage in the behavior condemned.
It would be useful to know more about the employment relationship between the REs and
NIRs and the actual agencies they worked for in terms of whether these workers were full-
time, permanent employees with benefits or were temporary (contract) employees, who
worked on an hourly basis. The nature of the employment relationship (permanent or tempo-
rary) is clearly an important consideration but not one for which we gathered data in this pilot
work. In what direction these different power relationships between workers and agencies
might or might not influence the level of discretion that these street-level workers use in their
work is simply unknown but should be explored in the future.
Having NIRs and REs in the same focus group was a limitation, given the general guide-
line of having homogeneous focus groups (Morgan & Krueger, 1998). Although we believe
that this did not greatly affect the data or conclusions, we now hypothesize that there are sig-
nificant differences between the views and experiences of the two groups. We also hypothe-
size that there are differences between the views and experiences of REs who perform only
research and evaluation duties and those who also provide services to the same population.
As described below,we are testing these hypotheses with additional intensive interviews with
relevant research workers.
Conclusion
Although REs make significant contributions to research and evaluation, the literature and
our preliminary findings raise issues that require attention from the research and evaluation
community. Most immediately, the focus group data support the need for more targeted and
nuanced training approaches for this category of research staff to protect research participants,
enhance research integrity, and promote collection of valid data in vulnerable populations. As
intended,the results of the second round of the pilot study were used to develop the draft into
training materials that address issues raised in the standard framework,but they do so in a way
that accords more with the concepts and experiences of REs.
The focus groups also served as pilot work for a grant to explore in greater depth the way
that single- and dual-role REs make sense of and respond to ethical dilemmas experienced when
performing community-based research. We include a contrast group of NIRs to determine if
they respond similarly or dissimilarly to the same kinds of ethical dilemmas. We will also be
gathering intensive interview data from some frontline supervisors who work with REs and
NIRS about issues they face when supervising REs and NIRs, what kinds of supports would
assist them as supervisors, and ways that training for REs and NIRs could be improved.3
Alexander, Richman / Ethical Dilemmas 83
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
REs have been recognized by federal authorities as “members of the ‘scientific commu-
nity’subject to PHS regulations on scientific misconduct”(ORI, 2004c, p. 4). However,
standard ethics training and guidance do not speak directly to the realities of the community
research work of REs. The experiences of REs are both similar to and different from those of
more traditional research workers (NIRs) working with the same targeted research popula-
tions. REs and NIRs often have a lot of discretion and have to make decisions on the spot,fre-
quently without easy access to supervisors. Even though the easy rapport and ease of access
of REs to research participants is a real advantage, the indigenousness of REs may also
increase social risk for research participants, raise difficulties for maintaining confidentiality,
and challenge appropriate boundaries in research relationships. These in turn can compromise
the validity of the data. Indeed, the factors identified by the American Association of Public
Opinion Research (2003) as increasing the likelihood of data falsification,including (as noted
above) difficult-to-locate research participants, short field periods, and interviewers working
off-site from the parent organization,are common conditions for REs. We must keep in mind,
as well, that if REs are indigenous to a vulnerable population, they are themselves vulnera-
ble. This vulnerability may intensify any concern they may have about disappointing their
supervisors by failing to meet recruitment or data collection goals.
Understanding the perspective of REs and the pressures they face can promote respect and
protection for individual research participants, respect for communities, and valid science.
The pilot results presented above and the research currently under way represent a small step
in that direction.
Notes
1. The contract was number 03T00302701D, Educating Staff in Community Agencies about Human Subjects
Protection in Research, awarded by the Office of Research Integrity, U.S. Department of Health & Human Services,
to Leslie B. Alexander, project director, and Kenneth A. Richman, project codirector.
2. There were 12 RE and NIR participants in the second round. One of these had not participated in a first-round focus
group. Five from the first round did not participate in the second round. Of the nine supervisors in the second round,one
had not participated in a first-round focus group. Four from the first round did not return for the second round.
3. The grant is number 5RO1 NR9879–2, Research Extenders and Research Integrity:A New Frontier, awarded
to Leslie B. Alexander, principal investigator, and Kenneth A. Richman, co-principal investigator, funded by the
Office of Research Integrity and the National Institute of Nursing Research.
References
American Association of Public Opinion Research. (2003). Interviewer falsification in survey research: Current best
methods for prevention, detection, and repair of its effects. Retrieved May 7, 2007, from http://www.aapor.org/pdfs/
falsification.pdf
Andrews, J. O., Felton, G., Wewers, M. E., & Heath, J. (2004). Use of community health workers in research with
ethnic minority women. Journal of Nursing Scholarship, 36, 358-365.
Berg, B. L., Sanudo, F., Hovell, M., Sipan, C., Kelley, N., & Blumberg, E. (2004). The use of indigenous interviewers
in a study of Latino men who have sex with men:A research note. Sexuality & Culture, 8, 87-103.
Clark, C., Scott, E., Boydell, K. M., & Goering, P. N. (1999). Effects of client interviewers on client reported satis-
faction with mental health services. Psychiatric Services, 50(7), 961-963.
Code of Federal Regulations, Title 45, Part 46.102(d).
Code of Federal Regulations, Title 45, Part 46 (45 C.F.R. 46). Subpart A: Federal Policy for Protection of Human
Subjects. Retrieved from http://ohrp.osophs.dhhs.gov/humansubjects/guidance/45cfr46.htm
Cunningham-Williams, R. M., Cottler, L. B., Compton, W. M., Desmond, D. P., Wechsberg, W., Zule, W. A., et al.
(1999). Reaching and enrolling drug users for HIV prevention: A multi-site analysis. Drug and Alcohol
Dependence, 54, 1-10.
Fadem,P.,Minkler,M.,Perry,M.,Blum,K.,Moore,L.,& Rogers,J. (2003). Ethical challenges in community-based
participatory research. In M. Minkler & N. Wallerstein (Eds.), Community-based participatory research for
health(pp. 242-262). San Francisco:Jossey-Bass.
84 American Journal of Evaluation / March 2008
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from
Fox, K. J. (1991). The politics of prevention: Ethnographers combat AIDS among drug users. In M. Burawoy,
A. Burton, A. A. Ferguson, K. J. Fox, J. Gamson, N. Gartrell, et al. (Eds.), Ethnography unbound:Power and
resistance in the modern metropolis(pp. 227-249). Berkeley:University of California Press.
Geter,R. S. (1995). Drug research:An institutional methodology. International Journal of the Addictions,30,617-627.
Green,L. W.,& Glasgow,R. E. (2006). Evaluating the relevance,generalization,and applicability of research:Issues
in translation methodology and external validity. Evaluation and the Health Professions, 29, 126-153.
Harper, G. W., Bangi,A. K., Contreras, R., Pedraza,A., Tolliver, M., & Vess, L. (2004). Diverse phases of collabo-
ration:Working together to improve community-based HIV interventions for adolescents. American Journal of
Community Psychology, 33, 193-204.
Hildebrandt, E. (1996). Survey data collection: Operationalizing the research design. Public Health Nursing, 13(2),
135-140.
Hill, M. N., Bone, L. R., & Butz, A. M. (1996). Enhancing the role of community health workers in research.
Image—The Journal of Nursing Scholarship, 28, 221-227.
Holbrook, A. L., Farrar, I. C., & Popkin, S. J. (2006). Surveying a Chicago public housing development:
Methodological challenges and lessons learned. Evaluation Review, 30, 779-802.
Israel, B. A., Lichtenstein, R., Lantz, P., McGranaghan, R., Allen, A., Guzman, J. R. et. al. (2001). The Detroit
Community-Academic Urban Research Center: Development, implementation & evaluation. Journal of Public
Health Management Practice, 7, 1-19.
Minkler, M. (2005). Community-based research partnerships: Challenges and opportunities. Journal of Urban
Health:Bulletin of the New York Academy of Medicine, 82, 3-12.
Minkler, M., & Wallerstein, N. (Eds.). (2003). Community based participatory research for health. San Francisco:
Jossey-Bass.
Morgan, D. L., & Krueger, R. A. (1998). The focus group kit. Thousand Oaks, CA:Sage.
Office of Research Integrity. (2004a). Case summaries. Office of Research Integrity Newsletter, 12(2). Retrieved
June 18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2004b). Misconduct findings made against curbstoners. Office of Research Integrity
Newsletter, 12(2). Retrieved June 18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2004c). Survey researchers address curbstoning. Office of Research Integrity
Newsletter, 12(4). Retrieved June 18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2006a). Case summaries. Office of Research Integrity Newsletter,14(2). Retrieved June
18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2006b). Case summaries. Office of Research Integrity Newsletter,14(3). Retrieved June
18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2006c). Case summaries. Office of Research Integrity Newsletter,14(4). Retrieved June
18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2006d). Case summaries. Office of Research Integrity Newsletter,15(1). Retrieved June
18, 2007, from http://ori.dhhs.gov/publications/newsletters.shtml
Office of Research Integrity. (2006e). Handling misconduct—preliminary assessment:Can survey research staff commit
research misconduct? Retrieved June 14,2007,from http://ori.hhs.gov/misconduct/survey_research.shtml
Perry, J., & Felce, D. (2004). Initial findings on the involvement of people with an intellectual disability in inter-
viewing their peers about quality of life. Journal of Intellectual & Developmental Disability, 29, 164-171.
Richman, K. A. (2002, Fall). Responsible conduct of research is all well and good. American Journal of Bioethics,
2, 62-63.
Roth, J. A. (1966). Hired hand research. The American Sociologist, 1, 190-196.
Silka, L. (2003). Community repositories of knowledge:A tool to make sure research pays for university partners.
Connection:The Journal of the New England Board of Higher Education, 71, 61-64. Retrieved June 1, 2007,
from http://www.nebhe.org/info/pdf/nejhe/Connection_Spring03.pdf
Simoni, J. M., Weinberg, B. A., & Nero, D. K. (1999). Notes from the field: Training community members to
conduct survey interviews:Notes from a study of seropositive women. AIDS Education and Prevention,11,87-88.
Staggenborg, S. (1988). “Hired hand”research revisited. The American Sociologist, 19(3), 260-269.
Survey Research Laboratory, College of Urban Planning and Public Affairs, University of Illinois. (2004). Publications.
Survey Research,35,15-16. Retrieved May 7,2007,from http://www.srl.uic.edu/Publist/Newsletter/2004/04v35n1.pdf
Sussman,S.,Valente,T. W.,Rohrbach,L. A.,Skara,S.,& Pentz,M. A. (2006). Translation in the health professions:
Converting science into action. Evaluation & the Health Professions, 29, 7-32.
Alexander, Richman / Ethical Dilemmas 85
© 2008 American Evaluation Association. All rights reserved. Not for commercial use or unauthorized distribution. at UNIV CALIFORNIA DAVIS on April 19, 2008 http://aje.sagepub.comDownloaded from