|
| |
 |
|
| ARTICLE |
|
| Year : 2020 | Volume
: 18
| Issue : 1 | Page : 3-17 |
|
What works in psychosocial programming in humanitarian contexts in low- and middle-income countries: a systematic review of the evidence
Emily E Haroz1, Amanda J Nguyen2, Catherine I Lee1, Wietse A Tol3, Shoshanna L Fine3, Paul Bolton1
1 Department of International Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
2 Curry School of Education, University of Virginia, Charlottesville, Virginia, USA
3 Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, Baltimore, MD, USA
| Date of Submission | 01-Apr-2019 |
| Date of Decision | 09-Sep-2019 |
| Date of Acceptance | 27-Nov-2019 |
| Date of Web Publication | 29-May-2020 |
Correspondence Address:
PhD, MA, MHS Emily E Haroz
Center for American Indian Health, 415 N. Washington St., Baltimore, MD 21205
USA
 Source of Support: None, Conflict of Interest: None
DOI: 10.4103/INTV.INTV_6_19
While there is growing evidence for the effectiveness of mental health interventions in low- and middle-income countries and humanitarian contexts, this is lacking for psychosocial programming. We aimed to summarise the evidence for psychosocial programming in these contexts through a systematic review (PROSPERO: CRD42017069066) of peer-reviewed and grey literature of programme evaluations. A total of n = 42,435 unique records were initially identified, with n = 211 records meeting full inclusion criteria. We identified 51 randomised controlled trials of 47 different interventions. The remaining studies used different evaluation methodology. Only three interventions had more than one experimental/quasi-experimental evaluation: Brief Intervention and Contact, Problem Management Plus and Child Friendly Spaces. While there are many studies of interventions, it was challenging to identify the same intervention across studies, leaving almost no interventions with more than one rigorous study supporting their use and many interventions that are poorly described. This makes it difficult to choose between them or even to implement them. Future research should focus on replication of well-described interventions in multiple different sites, to place future intervention selection on a more scientific basis. There is also a need to better understand the impact of psychosocial programmes in sectors other than health and protection, such as nutrition. These sectors may provide critical delivery mechanisms for psychosocial programming to broaden the reach of such interventions.
Key implications for practice
- Efforts to build the evidence base for psychosocial programming in humanitarian settings are challenged by the breadth of programming delivered, populations served and outcomes targeted in this field, resulting in predominantly one-off studies of a wide range of interventions.
- To inform future intervention selection, research initiatives should focus on replication of well-described interventions in multiple contexts or stages of humanitarian response.
- Rigorous evaluation of community-focused psychosocial programmes is also needed.
Keywords: evaluation, humanitarian, LMIC, MHPSS, psychosocial
How to cite this article:
Haroz EE, Nguyen AJ, Lee CI, Tol WA, Fine SL, Bolton P. What works in psychosocial programming in humanitarian contexts in low- and middle-income countries: a systematic review of the evidence. Intervention 2020;18:3-17 |
How to cite this URL:
Haroz EE, Nguyen AJ, Lee CI, Tol WA, Fine SL, Bolton P. What works in psychosocial programming in humanitarian contexts in low- and middle-income countries: a systematic review of the evidence. Intervention [serial online] 2020 [cited 2023 Jun 9];18:3-17. Available from: http://www.interventionjournal.org//text.asp?2020/18/1/3/285314 |
| Introduction | |  |
Globally, an estimated 135.3 million people are affected by humanitarian crises, such as armed conflicts and disasters (United Nations Office for the Coordination of Humanitarian Affairs (UNOCHA), 2018). Humanitarian crises have a profound impact on individuals, families and communities including on people’s mental health (MH) and wellbeing. In 2018, UNOCHA documented over $15 billion in humanitarian funding provided by governments and other non-profit organisations (UNOCHA, 2018). Only a proportion of this is allocated to interventions and activities focused on mental health and psychosocial support (MHPSS); in a 2011 review, $226.1 million was provided between 2007 and 2009 for MHPSS programming (Tol et al., 2011a). In order to achieve maximum impact of the limited funds invested in MHPSS, it is important to understand the evidence base for MHPSS interventions.
The Interagency Standing Committee Reference (IASC) Group on Mental Health and Psychosocial Support in Emergency Settings defines MHPSS as ‘any type of local or outside support that aims to protect or promote psychosocial wellbeing and/or prevent or treat mental disorder’ (IASC, 2007). MHPSS interventions range from incorporating social and psychological considerations in the delivery of basic services to activities focused on clinical treatment for those affected by mental disorders (Figure 1). International consensus-based guidelines for MHPSS emphasise the importance of providing multi-layered supports to address a range of needs (IASC, 2007; Sphere Association, 2018).
Humanitarian agencies have tended to focus either on MH activities (i.e., management of mental disorders) or on ‘psychosocial’ activities. The term psychosocial underscores the connection between psychological functioning and social factors (e.g., social support, education and livelihoods). Psychosocial responses represent a diverse set of activities focused, for example, on: reducing social risks to wellbeing (e.g., establishing community-based protection mechanisms); strengthening protective factors (e.g., facilitating positive family interactions); and improving social aspects of humanitarian service delivery (e.g., strengthening participation of affected populations in humanitarian response) (IASC, 2007, (2010); UNICEF, 2018). However, despite a potential distinction between MH and psychosocial activities, major coordinating bodies, such as IASC, group these approaches together recognising the lack of a perfectly clear distinction between the two and the potential need to address these together.
There is mounting expectation that humanitarian interventions be based on strong evidence (Ager et al., 2014; Blanchet et al., 2017; Samarasekera & Horton, 2017). A growing body of empirical evidence demonstrates the effectiveness of MHPSS interventions (e.g., Purgato, van Ommeren, Tol, & Barbui, 2018a; Tol et al., 2011a,b) and for specific sub-populations (e.g. Brown, de Graaff, Annan, & Betancourt, 2017; Jordans, Pigott, & Tol, 2016; Purgato et al., 2018b). However, many of the interventions that have been evaluated most rigorously are manualised, MH treatment interventions, rather than the less targeted, but more commonly implemented psychosocial support (PSS) interventions (Tol et al., 2011a). Many PSS programmes aim to reduce the likelihood of developing MH problems by strengthening resilience and protective factors and reducing distress for people affected by humanitarian crises. Using the IASC intervention pyramid for MHPSS as a framework, we conceptualised PSS programmes as being programmes within the bottom three layers of the pyramid (i.e. focused non-specialised supports, community and family supports and basic services and security) that focus on prevention or promotion rather than treatment of mental disorders. Existing systematic reviews (Brown et al., 2017; Jordans et al., 2016; Purgato et al., 2018a,b; Tol et al., 2011a), have relied primarily on results from randomised controlled trials (RCTs) and while incredibly important to the field fail to provide a full picture of the evidence for a vast number of interventions that are commonly used in practice, target other subgroups and rely more on observational data, pre-post assessment and/or qualitative feedback for evaluation.
We aimed to address this gap by systematically reviewing the literature on evaluations of PSS interventions that are delivered to a range of individuals in a range of contexts. While we originally aimed to examine what is currently known about the effectiveness of specific PSS interventions in low-resource humanitarian settings, the large number of interventions without common names and/or detailed descriptions resulted in an inability to draw conclusions about the level of evidence for most specific interventions. Thus, instead, we sought to examine: 1) What types of PSS interventions are implemented and evaluated in humanitarian settings? 2) Who do they target and in which context are they implemented? 3) What types of evaluations exist for these interventions? and 4) What do these evaluations tell us about the level of evidence for PSS programmes in the field and future research goals? The goal of this review was to inform a research strategy to improve knowledge on what works in PSS programming in low-resource humanitarian settings globally.
| Methods | | |
The protocol for this review was registered in the International Prospective Register of Systematic Reviews (PROSPERO), number CRD42017069066 and follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines (Moher et al., 2015). We aimed to bridge the research-practitioner gap by conducting the review in close coordination with humanitarian practitioners. A six-member Steering Committee (see Acknowledgements) with expertise in implementation and evaluation of PSS interventions in humanitarian contexts oversaw the review, providing feedback throughout the process on search terms, inclusion criteria, data extraction procedures and interpretation of results. The scope of the review was defined through extensive individual stakeholder consultation with 109 psychosocial programme implementers. The goal of this approach was to complete a stakeholder-driven review, with the study team mainly serving as technical advisors.
Identification and selection of studies
We searched peer-reviewed and grey literature in five academic databases (PubMed, PsychINFO, Embase, PILOTS and Global Health) and twelve grey literature sources (GODORTS, UNHCR, WHO, Save the Children, USAID, IMC, ICRC, IOM, MSF, UNICEF, Mercy Corps, MHIN). Peer-reviewed search terms varied by database (see S1 in the Supplemental Material) but involved searching Medical Subject Heading (MeSH) terms, titles and abstracts using the strategy: Crisis/Disaster/Destructive Occurrences terms AND Psychosocial Programming terms AND Outcome terms. Humanitarian settings are defined as ‘a range of situations including natural disasters, conflict, slow- and rapid-onset events, rural and urban environments and complex political emergencies in all countries’ (Sphere Association, 2018, p. 9). However, we also included interventions implemented in the context of the HIV epidemic and with populations suffering from chronic poverty in low- and middle-income countries (LMIC), as lessons learned from these contexts may be applicable to humanitarian settings. We also hand-searched the journal Intervention for relevant articles. Grey literature search terms included ‘Psychosocial AND Evaluation’ and ‘Psychosocial AND Impact’ due to restrictions with search engines. The grey literature search included a manual review of organisations’ websites; for each website, reviewers screened title and abstract and included relevant reports for full-text review.
Inclusion and exclusion criteria
All articles were screened based on the following inclusion and exclusion criteria. To be included, the article must include primary data (i.e. represent empirical research) to evaluate a programme or intervention that was designed to address a psychosocial problem or outcome and be written in English. Within these criteria, we aimed to be as inclusive as possible: data could be at the individual, family or community level; the evaluation could employ any of a range of quantitative (e.g. RCTs or non-RCTs, pre-post evaluations, interrupted time series, cross-sectional, pilot studies) or qualitative (e.g. in-depth interviews, focus groups, participant feedback) study designs; the interventions could include social and psychological activities or more general humanitarian programming that was designed to impact psychosocial outcomes; and the outcomes could range from community-focused to sub-diagnostic symptoms of disorder.
Articles were excluded if they 1) were a review or other non-empirical paper; 2) reported a study that did not evaluate a programme or intervention, such as a needs assessment, prevalence study, or study of risk factors, or reported a small sample (i.e. n < 6) case study or case series design; 3) did not specify a main psychosocial outcome a priori, but instead focused on, for example, only physical health outcomes such as diabetes, obesity or HIV; 4) focused on reducing sexual risk behaviours or HIV/STI-specific outcomes; 5) evaluated a structured psychotherapy such as cognitive behavioural therapy, interpersonal psychotherapy or narrative exposure therapy; 6) evaluated an intervention that treated a diagnosable mental illness or disorder; and 7) reported evaluations in a World Bank-defined high-income country.
Operationalisation of our definition of a PSS programme (e.g. programmes that aim to reduce the likelihood of developing MH problems by strengthening resilience and protective factors and reducing distress for people affected by humanitarian crises) was done through our exclusion criteria. Our initial search was not limited to interventions implemented in low- and middle-income countries, but this criterion was added later based on suggestions from the advisory board on what is most relevant. While the original search also included studies focused on reduction of sexual risk behaviours and HIV/STI-specific outcomes, these were excluded during abstract review as being too broad in scope. Secondary data analyses were also excluded, but we used these to help in identifying the original data analysis for inclusion.
Screening
All references were screened based on titles and abstracts, with eligible references undergoing subsequent full-text review. At each stage, subsets of materials were screened by two reviewers to establish inter-rater reliability. Twelve raters were involved in the screening. At both the title/abstract and full-text review stage, records were double-reviewed until we reached above 90% agreement. Disagreements were reviewed by either EH or AN, who made a final, blinded decision (i.e., they were not aware of the prior reviewers’ decisions) about whether to include or exclude the abstract. Cause of conflicts were discussed to help improve agreement going forward. Both title/abstract and full-text screening were conducted using Covidence Systematic Review Software (n.d.).
Data extraction and quality review
After full-text review, data were extracted from each included reference by a team of nine reviewers. Reviewers received initial training and double-extracted a subset of records with biweekly team review until reliability was reached. Data extraction included: 1) basic demographic information; 2) description of study design; 3) description of programmatic activities; and 4) up to three target outcomes. Given that our goal was to examine the breadth of evidence for PSS interventions and the known limitations in the field regarding quality of research, we refrained from rating the quality of different types of studies in this review.
Data synthesis
We initially established criteria for classifying interventions by their amount of evidence for effectiveness following the approach of Weiss et al. (2016). This approach heavily relies on RCTs to determine level of evidence. However, stakeholders did not favour this approach because of the emphasis on RCTs. Additionally, this approach required us to identify repeated studies of the same intervention. This proved challenging due to inadequate intervention descriptions, extensive local adaptations or slight differences between similar interventions that made it difficult to determine whether interventions should be treated as the same or different. To address these challenges, we developed an alternative data synthesis approach in which we more broadly summarised the number and types of evaluations conducted for groups of similar interventions to show the degree and extent to which they had been evaluated. Our rationale was that interventions that had undergone multiple evaluations demonstrating positive results, particularly with a degree of causal evidence (e.g. RCTs; quasi-experimental designs), would be considered promising. We were careful not to make conclusions about likely effectiveness, as in an effort to provide an inclusive and comprehensive review of the state of evaluations of PSS programmes, we did not rate quality of each evaluation.
To create meaningful groupings of interventions, we used two alternative coding frameworks: 1) an emergent coding approach based on extracted intervention descriptions (i.e., ‘bottom up’); and 2) application of intervention category labels developed and used internationally to map MHPSS activities (i.e., ‘top down’): so-called ‘Who is Where When doing What (4Ws) codes’ (IASC, 2012). Use of the emergent coding approach aligned with terms used by stakeholders and provided a summary description of the activities of the interventions, while the ‘top-down’ codes aligned with the common 4W framework. We used the two approaches based on input from the advisory board to align our results summary with commonly used frameworks such as the 4Ws. However, we found that using only the 4Ws did not provide enough insight into the actual activities being implemented. Examples of codes are included in the supplemental material (S2 and S3).
For the emergent coding approach, a single coder (SF) reviewed all references and generated an initial codebook, which was revised by EH and AN. All subsequent references were coded by one of two coders (SF, AN). For final 4W codes: AN coded all records based on intervention descriptions; EH coded 10% of records to check for consistency and achieved 90% agreement. The inconsistent ratings were resolved through consensus rating. Finally, due to the vast number of interventions evaluated, we included a table of all interventions that were evaluated using RCT methods only so as to provide a more detailed picture of programmes that were evaluated in this way and their findings.
| Results | | |
Our initial search yielded a total of n = 42,435 unique records. After title and abstract screening, n = 39,713 were excluded, leaving n = 2523 records assessed for eligibility based on full texts. [Figure 2] lists the reasons for exclusion based on full-text review. The grey literature search subsequently added n = 62 records after full-text review and another n = 4 were identified from reference lists, leaving a total of n = 210 records representing n = 216 interventions for full data extraction and analysis (Figure 2).
Characteristics of identified evaluations
Examining evaluations by World Bank region, most evaluations took place in Sub-Saharan Africa (51.0%), followed by the Middle East/North Africa (15.2%) and South Asia and Europe/Central Asia (9.0% each). War/post-conflict and HIV were the most common contexts in which interventions were evaluated (35.0% and 31.0%, respectively), followed by refugee/displacement (7.6%) and natural disaster (7.2%) contexts.
Almost all programmes were implemented with mixed gender participant groups (82.4%), with 13.3% implemented among women only and 1.9% (n = 4 interventions) implemented among men only. Approximately a quarter of interventions targeted children and youth (26.7%), while the rest either served adults (34.8%) or adults and children/youth together (38.5%). We identified no evaluations of interventions targeting older adults (65+ years).
Most evaluations concerned interventions implemented in the health or protection sectors (49.1% and 29.5%, respectively). Eighteen per cent of interventions evaluated were implemented in education, while almost no evaluations were implemented in food security and nutrition or shelter and site planning. Aligned with the IASC intervention pyramid, 61.9% were classified as community and family support interventions, 34.3% as focused non-specialist services and 3.8% as basic service interventions.
Quantity and type of evaluations
We identified two interventions with multiple independent RCTs and significant positive findings on one or more outcomes. These include Brief Intervention and Contact for suicide prevention (Fleischmann et al., 2008; Riblet, Shiner, Young-Xu, & Watts, 2017; Vijayakumar et al., 2017Vijayakumar, Mohanraj, Kumar, Jeyaseelan, Sriram, & Shanmugam, 2017) and Problem Management Plus (PM+) to reduce symptoms of depression and anxiety (Bryant et al., 2017; Rahman et al., 2016). One of the PM+ studies did not meet inclusion criteria because the programme was provided only to people with a likely mental disorder, so it is unclear whether this should be considered a MH or PSS programme. Two other programmes had repeated evaluations including Child-Friendly Spaces (CFS), based on a set of studies coordinated by the same research team (Metzler, Savage, Yamano, & Ager, 2015) and the Classroom-Based Intervention (Tol et al., 2008, (2012), (2014); Jordans et al., 2010). Aside from these few exceptions, other interventions were either locally developed, implemented in adapted versions or not described with enough detail to determine whether they were the same programme.
The most frequently evaluated intervention approaches by 4Ws codes were person-focused psychosocial work (n = 66; 30.4%), strengthening community and family support interventions (n = 45; 20.7%) and psychological interventions (n = 36; 17.0%). Person-focused psychosocial work had proportionately more randomised evaluations (31.8%) with at least one positive outcome. All other MHPSS activity codes had fewer evaluations overall and proportionally more observational or quasi-experimental evaluations ([Table 1]). A table of the number of evaluations and frequency of designs by psychosocial outcome can be found in the supplemental material (S4).
Using emergent coding categories, the most frequently evaluated approaches were multi-component counselling (n = 33; 15.7%), family strengthening interventions (n = 15; 7.1%) and peer support groups (n = 13; 6.2%). Family strengthening interventions and multi-component counselling also had proportionally more randomised evaluations (47.5% and 45.5%, respectively). Due to the body of evidence using strong study designs, these types of interventions could be considered likely effective, but more work is needed to support replication of these findings, evaluation of study quality and examination of intervention components used across these interventions. Other approaches with multiple RCTs included peer support groups, financial assistance interventions, violence prevention interventions and community mobilisation interventions. These approaches may be considered in need of more research, as it is unclear whether the studies focused on the same or different interventions. Finally, approaches such as case management, recreational activities, health education and reintegration were dominated by observational evaluations, suggesting a need for stronger evaluations of these types of interventions ([Table 2]).
[Table 3] lists the interventions, contexts, World Bank regions, sample sizes and characteristics, the name of the intervention, the sector where it is implemented and observed outcomes for each intervention evaluated using a randomised design. Most interventions focused on reducing symptoms of distress and/or promoting coping skills and wellbeing. Four of the five interventions that aimed to reduce violence showed non-significant results with the exception of Chaudhury et al. (2016) who found through qualitative feedback reports of violence reduction. Overall this table shows the wide variety of interventions that are targeting similar outcomes and the challenges with building an evidence base for any particular one given diversity in approaches.
Seven interventions were identified with null results for all outcomes of the study (Nyangara, Thurman, Hutchinson, & Obiero, 2009; Pairojkul, Siripul, Prateepchaikul, Kusol, & Puytrakul, 2010; Richards, Foster, Townsend, & Bauman, 2014; Segatto, Andreoni, de Souza e Silva, Diehl, & Pinsky, 2011; Thurman, Kidman, & Taylor, 2015; Train & Butler, 2013). The relative rarity of these null findings suggests potential publication bias, limited utility for these interventions, problems with implementation or a combination of these factors. As these results have not been replicated, no conclusions should be drawn about the effectiveness of these types of interventions beyond these particular studies.
| Discussion | | |
Our review aimed to better understand the range of psychosocial interventions implemented in humanitarian contexts and the body of evidence that exists to support them. We found that most interventions with evaluations were implemented in sub-Saharan Africa in the context of war, conflict and the HIV epidemic and through the health and protection sectors. Overwhelmingly these programmes served men and women together, with very few gender-specific programmes and none focusing on older adults. A central finding was an almost complete lack of interventions that have been studied multiple times. Without replicated findings across multiple studies, we are unsure whether the effectiveness of an intervention found in a single study would be generalisable to other contexts.
We also explored the literature by type of intervention. Person-focused PSS interventions are the most frequently and rigorously studied. However, these interventions often consist of multiple components (e.g., psychoeducation, mindfulness) that vary in their use with this variation not recorded, making it difficult to determine what components individuals actually receive (Brown et al., 2017). Community-focused PSS interventions have been less rigorously evaluated. A possible explanation for this is that person-focused interventions lend themselves more readily to the medical model-driven, ‘gold standard’ RCT study design than the often much broader reaching PSS interventions. However, these interventions reach fewer persons and, thus, are of less interest to service organisations and funders in terms of numbers served. Interventions that serve larger numbers, such as media campaigns, are more challenging to empirically evaluate using standard study designs, resulting in a lack of comparative research on these widely implemented interventions and strategies. This finding suggests a need to develop and disseminate study tools to facilitate rigorous empirical evaluation of complicated (e.g. multi-component, tiered, group or community-level) PSS interventions.
We found a large proportion of more rigorous evaluations for family strengthening interventions (n = 7; 47% RCTs) and CFS (n = 8; 73% were quasi-experimental designs). These are two intervention approaches that have been prioritised in prior research, resulting in multi-site replication and therefore progress in advancing our knowledge of whether these interventions are appropriate, feasible and effective (Tol et al., 2011a,b). Most of the CFS evaluations were coordinated by one organisation, World Vision, to use similar methodology in different settings (Metzler et al., 2015). This approach of selecting one intervention and studying it in similar ways in diverse contexts is necessary to identify the most appropriate and effective interventions for future programming.
Such replication is one of the key challenges across all PSS interventions. Our review identified 51 RCTs of 47 different interventions. We believe this reflects a trend in which implementing organisations conduct a single evaluation of their own programme. These interventions may be similar across organisations, but there is limited reporting or ability to evaluate their ‘sameness’ across studies. However, this large number of different studies also highlights the wide range of interventions, approaches and outcomes that are considered ‘psychosocial’. By comparison, evaluations of MH interventions have predominantly focused on only a few psychotherapy models and their relatively few targeted outcomes, which are restricted to delivery among a smaller target group of highly symptomatic individuals; this narrower focus has facilitated more rapid development of empirical support for the impact of these interventions across multiple LMIC (Singla, Kohrt, Murray, Anand, Chorpita, & Patel, 2017). Building an evidence base on which to compare PSS interventions will require collaboration by service providers and researchers to standardise, deliver and evaluate similar interventions across contexts and organisations. The lack of replicated studies presented a challenge for our original goal of identifying promising interventions to prioritise for future programming and research. As a result, we are unable to give clear recommendations about use of specific interventions at this time.
Most interventions were implemented in the health or protection sector; however, both guiding frameworks (e.g., IASC, 2007) and stakeholder perspectives (Lee, Nguyen, Haroz, Tol, & Bolton, 2019) recommend that PSS considerations are integrated in other sectors (e.g., nutrition, WASH). These interventions are missing in this review as they are not designed primarily to address PSS outcomes (an inclusion criteria). In the future, impacts on PSS outcomes should be included in evaluations of these programmes.
Many interventions were evaluated using multiple outcomes with little theoretical rationale for their inclusion. For example, the most commonly assessed outcomes were symptoms of psychological distress, which are not necessarily the appropriate choice to evaluate preventive and promotive programmes. There is a need for greater guidance and coordination around an appropriate set of outcomes associated with both individual and community wellbeing. As this review was being conducted, there was parallel work in this area of measurement that can provide such guidance including by the International Federation of Red Cross and Red Crescent Societies (IFRC) and the IASC (Augustinavicius, Greene, Lakin, & Tol, 2018; Inter-Agency Standing Committee, 2017).
Limitations
Our search was limited to English language only, potentially missing evaluations published in other languages. Given that humanitarian disasters disproportionately affect populations in non-English-speaking countries, this is an important limitation in our findings. Second, contrary to most systematic literature reviews that have stringent inclusion criteria to support causal inference with minimal risk of bias, we employed a more liberal set of inclusion criteria to better represent the breadth and diversity of research done in this field. Moreover, humanitarian settings present numerous, well-recognised challenges that may either deter organisations from attempting to conduct research at all or may contribute to a wide range in study quality (Ager et al., 2014). While we believe that providing this complete picture is a major contribution to the literature, given the great diversity of types of evaluations we did not evaluate quality of each evaluation for inclusion. The resulting wide variation in study quality presents its own limitation for interpretation. Aside from the noted exceptions, we recommend that our findings be used to support efforts to establish a better research agenda rather than drawing inferences about the relative effectiveness of particular interventions. Third, there remains a grey area related to which interventions are considered MH and which are considered psychosocial (Tol, Purgato, Bass, Galappatti, & Eaton, 2015). This probably resulted in some studies being included in this review (e.g. PM+) that many in the field may classify as a MH intervention. Where there was uncertainty about inclusion, we consulted with our steering committee and generally erred on the side of inclusivity. This also relates to our need to draw a clear distinction between MH and PSS, which was done in an effort to review a body of interventions that is often missed but does not necessarily reflect how these types of interventions are implemented in practice. Finally, codes using the emergent coding framework were applied using single-rater coding rather than a more rigorous double-coding approach. Because these classifications are not always clear-cut, it is likely that there is some level of unreliability in the codes applied.
| Conclusions | | |
This review summarises the body of evidence for psychosocial interventions in humanitarian contexts in LMIC. Based on over 200 evaluations, our recommendations are primarily focused on next steps for setting a PSS research agenda as follows: 1) The field needs to focus on replication efforts of promising interventions, including better specification of what is done and when to build more conclusive evidence of whether these interventions work and how; 2) the over representation of PSS interventions in health and protection sectors indicates a need to integrate and evaluate PSS interventions in other sectors of a humanitarian response; 3) as the majority of evaluations reported on person-focused interventions, there is a need for evaluation of community-based PSS programming with theory-driven outcomes; 4) evaluation of community-based PSS will require development and dissemination of field-friendly study tools and approaches that are a better fit to these types of interventions; and finally, 5) almost no interventions were designed for and evaluated for outcomes specific to certain sub-groups, particularly men and older adults − there should be an effort to include these in future studies.
Acknowledgements
The authors would like to thank the student reviewers who reviewed these articles. We would also like to thank the Steering Committee members (Nancy Baron, Carmel Gaillard, Ananda Galappatti, Sarah Harrison, Mark Jordans, and Patrick Onyango Mangen) and Advisory Board members (Lauren Bienkowski, Martha Bragin, Marie Bray, Shahla Eltayeb, Craig Higson-Smith, Rebecca Horn, Ashley Nemiro, Miryam Rivera-Holguin, Teresa Wallace, Guglielmo Schinina, Leslie Snider, Pieter Ventevogel, Marie Waaade, Inka Weissbecker, and Mike Wessells) for their contributions to the process described in this manuscript.
Financial support and sponsorship
This work was supported by the USAID Office of Foreign Disaster Assistance (grant number AID-OFDA-G-16-00212). Dr. Emily E. Haroz was partially supported by a grant from the National Institute of Mental Health (K01MH116335).
Conflicts of interest
There are no conflicts of interest.
[81]
| References | | |
| 1. | Aboud F. E., Singla D. R., Nahil M. I., Borisova I. (2013). Effectiveness of a parenting program in Bangladesh to address early childhood health, growth and development. Social Science & Medicine, 97, 250–258. doi:10.1016/j.socscimed.2013.06.020  |
| 2. | Abramsky T., Devries K., Kiss L., Nakuti J., Kyegombe N., Starmann E., Watts C. (2014). Findings from the SASA! Study: A cluster randomized controlled trial to assess the impact of a community mobilization intervention to prevent violence against women and reduce HIV risk in Kampala, Uganda. BMC Medicine, 12(1), 122. doi:10.1186/s12916-014-0122-5 |
| 3. | Ager A., Akesson B., Stark L., Flouri E., Okot B., McCollister F., Boothby N. (2011). The impact of the school-based Psychosocial Structured Activities (PSSA) program on conflict-affected children in Northern Uganda. Journal of Child Psychology and Psychiatry, 52(11), 1124–1133. doi:10.1111/j.1469-7610.2011.02407.x |
| 4. | Ager A., Burnham G., Checchi F., Gayer M., Grais R. F., Henkens M., Spiegel P. (2014). Strengthening the evidence base for health programming in humanitarian crises. Science, 345(6202), 1290–1292. doi:10.1126/science.1254164 |
| 5. | |
| 6. | Annan J., Sim A., Puffer E. S., Salhi C., Betancourt T. S. (2016). Improving mental health outcomes of Burmese migrant and displaced children in Thailand: A community-based randomized controlled trial of a parenting and family skills intervention. Prevention Science, doi:10.1007/s11121-016-0728-2 |
| 7. | Augustinavicius J. L., Greene M. C., Lakin D. P., Tol W. A. (2018). Monitoring and evaluation of mental health and psychosocial support programs in humanitarian settings: A scoping review of terminology and focus. Conflict and Health, 12(1), 9. doi:10.1186/s13031-018-0146-0 |
| 8. | Azad Marzabadi E., Hashemi Zadeh S. M. (2014). The effectiveness of mindfulness training in improving the quality of life of the war victims with post traumatic stress disorder (PTSD). Iran J Psychiatry, 9(4), 228–236. |
| 9. | Bass J., Murray S. M., Mohammed T. A., Bunn M., Gorman W., Ahmed A. M., Bolton P. (2016). A randomized controlled trial of a trauma-informed support, skills, and psychoeducation intervention for survivors of torture and related trauma in Kurdistan, Northern Iraq. Global Health: Science and Practice, 4(3), 452–466. doi:10.9745/ghsp-d-16-00017 |
| 10. | Berger R., Gelkopf M. (2009). School-based intervention for the treatment of tsunami-related distress in children: A quasi-randomized controlled trial. Psychotherapy and Psychosomatics, 78(6), 364–371. doi:10.1159/000235976 |
| 11. | Bhana A., Mellins C. A., Petersen I., Alicea S., Myeza N., Holst H., McKay M. (2014). The VUKA family program: Piloting a family-based psychosocial intervention to promote health and mental health among HIV infected early adolescents in South Africa. AIDS Care − Psychological and Socio-Medical Aspects of AIDS/HIV, 26(1), 1–11. doi:10.1080/09540121.2013.806770 |
| 12. | Blanchet K., Ramesh A., Frison S., Warren E., Hossain M., Smith J., Roberts B. (2017). Evidence on public health interventions in humanitarian crises. The Lancet, 390(10109), 2287–2296. doi:10.1016/S0140-6736(16)30768-1 |
| 13. | Brown F. L., de Graaff A. M., Annan J., Betancourt T. S. (2017). Annual research review: Breaking cycles of violence − A systematic review and common practice elements analysis of psychosocial interventions for children and youth affected by armed conflict. Journal of Child Psychology and Psychiatry, 58(4), 507–524. doi:10.1111/jcpp.12671 |
| 14. | Bryant R. A., Schafer A., Dawson K. S., Anjuri D., Mulili C., Ndogoni L., van Ommeren M. (2017). Effectiveness of a brief behavioural intervention on psychological distress among women with a history of gender-based violence in urban Kenya: A randomised clinical trial. PLOS Medicine, 14(8), e1002371. doi:10.1371/journal.pmed.1002371 |
| 15. | Chaudhury S., Brown F. L., Kirk C. M., Mukunzi S., Nyirandagijimana B., Mukandanga J., Betancourt T. S. (2016). Exploring the potential of a family-based prevention intervention to reduce alcohol use and violence within HIV-affected families in Rwanda. AIDS Care, 28 Suppl 2, 118–129. doi:10.1080/09540121.2016.1176686 |
| 16. | Chowa G., Masa R., Ansong D., Despard M., Wu S., Hughes D. (2015). Impacts of financial inclusion on youth development: Findings from the Ghana YouthSave experiment (CSD Research Report No. 15-35). St. Louis, MO. doi:10.7936/K7HQ3ZD7 |
| 17. | Covidence Systematic Review Software. (n.d.). Melbourne, Australia: Veritas Health Innovation. Retrieved from www.covidence.org |
| 18. | Diab M., Peltonen K., Qouta S. R., Palosaari E., Punamaki R. L. (2015). Effectiveness of psychosocial intervention enhancing resilience among war-affected children and the moderating role of family factors. Child Abuse & Neglect, 40, 24–35. doi:10.1016/j.chiabu.2014.12.002 |
| 19. | Dybdahl R. (2001). A psychosocial support programme for children and mothers in war. Clinical Child Psychology and Psychiatry, 6(3), 425–436. doi:10.1177/1359104501006003010 |
| 20. | Eloff I., Finestone M., Makin J. D., Boeving-Allen A., Visser M., Ebersohn L., Forsyth B. W. C. (2014). A randomized clinical trial of an intervention to promote resilience in young children of HIV-positive mothers in South Africa. (Special Issue: Children born into families affected by HIV). AIDS, 28, S347–S357. doi:10.1097/QAD.0000000000000335 |
| 21. | Fleischmann A., Bertolote J. M., Wasserman D., De Leo D., Bolhari J., Botega N. J., Thanh H. T. T. (2008). Effectiveness of brief intervention and contact for suicide attempters: A randomized controlled trial in five countries. Bulletin of the World Health Organization, 86(9), 703–709. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/18797646 |
| 22. | Hossain M., Zimmerman C., Kiss L., Abramsky T., Kone D., Bakayoko-Topolska M., Watts C. (2014). Working with men to prevent intimate partner violence in a conflict-affected setting: A pilot cluster randomized controlled trial in rural Cote d’Ivoire. BMC Public Health, 14, 339. doi:10.1186/1471-2458-14-339 |
| 23. | |
| 24. | |
| 25. | |
| 26. | |
| 27. | Jordans M. J., Komproe I. H., Tol W. A., Kohrt B. A., Luitel N. P., Macy R. D., de Jong J. T. (2010). Evaluation of a classroom-based psychosocial intervention in conflict-affected Nepal: A cluster randomized controlled trial. Journal of Child Psychology and Psychiatry, 51(7), 818–826. doi:10.1111/j.1469-7610.2010.02209.x |
| 28. | Jordans M. J. D., Pigott H., Tol W. A. (2016). Interventions for children affected by armed conflict: A systematic review of mental health and psychosocial support in low- and middle-income countries. Current Psychiatry Reports, 18(1), 9. doi:10.1007/s11920-015-0648-z |
| 29. | Kumakech E., Cantor-Graae E., Maling S., Bajunirwe F. (2009). Peer-group support intervention improves the psychosocial well-being of AIDS orphans: Cluster randomized trial. Social Science & Medicine, 68(6), 1038–1043. doi:10.1016/j.socscimed.2008.10.033 |
| 30. | Lee C., Nguyen A. J., Haroz E. E., Tol W. A., Bolton P. (2019). Stakeholder engagement to prioritize psychosocial support programs for further research in humanitarian settings. Global Mental Health, 6(e23). |
| 31. | L’Engle K. L., Mwarogo P., Kingola N., Sinkele W., Weiner D. H. (2014). A randomized controlled trial of a brief intervention to reduce alcohol use among female sex workers in Mombasa, Kenya. JAIDS Journal of Acquired Immune Deficiency Syndromes, 67(4), 446–453. doi:10.1097/QAI. 0000000000000335 |
| 32. | Li L., Lee S., Jiraphongsa C., Khumtong S., Iamsirithaworn S., Thammawijaya P., Rotheram-Borus M. J. (2010). Improving the health and mental health of people living with HIV/AIDS: 12-month assessment of a behavioral intervention in Thailand. American Journal of Public Health, 100(12), 2418–2425. doi:10.2105/AJPH.2009.185462 |
| 33. | Li L., Ji G., Liang L.-J., Ding Y., Tian J., Xiao Y. (2011). A multilevel intervention for HIV-affected families in China: Together for Empowerment Activities (TEA). Social Science & Medicine, 73(8), 1214–1221. doi:10.1016/j.socscimed.2011.07.013 |
| 34. | Li X., Harrison S. E., Fairchild A. J., Chi P., Zhao J., Zhao G. (2017). A randomized controlled trial of a resilience-based intervention on psychosocial well-being of children affected by HIV/AIDS: Effects at 6- and 12-month follow-up. Social Science and Medicine. doi:10.1016/j.socscimed.2017.02.007 |
| 35. | |
| 36. | Mawar N., Katendra T., Bagul R., Bembalkar S., Vedamurthachar A., Tripathy S., Paranjape R. S. (2015). Sudarshan Kriya yoga improves quality of life in healthy people living with HIV (PLHIV): Results from an open label randomized clinical trial. Indian Journal of Medical Research, 141(1), 90–99. |
| 37. | |
| 38. | Moher D., Shamseer L., Clarke M., Ghersi D., Liberati A., Petticrew M., Stewart L. A. (2015). Preferred reporting items for systematic review and meta-analysis protocols (PRISMA-P) 2015 statement. Systematic Reviews, 4(1), 1. doi:10.1186/2046-4053-4-1 |
| 39. | Nabunya P., Ssewamala F. M., Ilic V. (2014). Family economic strengthening and parenting stress among caregivers of AIDS-orphaned children: Results from a cluster randomized clinical trial in Uganda. Children and Youth Services Review, 44, 417–421. doi:10.1016/j.childyouth.2014.07.018 |
| 40. | Nyangara F., Thurman T. R., Hutchinson P., Obiero W. (2009). Effects of programs supporting Orphans and Vulnerable Children: Key findings, Emerging Issues, and Future directions from evaluation of four projects in Kenya and Tanzania. Washington, DC: Public Health. |
| 41. | O’Callaghan P., Branham L., Shannon C., Betancourt T. S., Dempster M., McMullen J. (2014). A pilot study of a family focused, psychosocial intervention with war-exposed youth at risk of attack and abduction in north-eastern Democratic Republic of Congo. Child Abuse & Neglect, 38(7), 1197–1207. doi:10.1016/j.chiabu.2014.02.004 |
| 42. | O’Callaghan P., McMullen J., Shannon C., Rafferty H. (2015). Comparing a trauma focused and non trauma focused intervention with war affected Congolese youth: A preliminary randomised trial. Intervention: Journal of Mental Health and Psychosocial Support in Conflict Affected Areas, 13(1), 28–44. doi:10.1097/WTF. 0000000000000054 |
| 43. | Olley B. O. (2006). Improving well-being through psycho-education among voluntary counseling and testing seekers in Nigeria: A controlled outcome study. AIDS Care, 18(8), 1025–1031. doi:10.1080/09540120600568756 |
| 44. | Pairojkul S., Siripul P., Prateepchaikul L., Kusol K., Puytrakul T. (2010). Psychosocial first aid: Support for the child survivors of the Asian Tsunami. Journal of Developmental & Behavioral Pediatrics, 31(9), 723–727. doi:10.1097/DBP.0b013e3181f46de2 |
| 45. | Paluck E. L. (2009). Reducing intergroup prejudice and conflict using the media: A field experiment in Rwanda. Journal of Personality and Social Psychology, 96(3), 574–587. doi:10.1037/a0011989 |
| 46. | Panter-Brick C., Dajani R., Eggerman M., Hermosilla S., Sancilio A., Ager A. (2018). Insecurity, distress and mental health: Experimental and randomized controlled trials of a psychosocial intervention for youth affected by the Syrian crisis. Journal of Child Psychology and Psychiatry, 59(5), 523–541. doi:10.1111/jcpp.12832 |
| 47. | Peltonen K., Qouta S., El Sarraj E., Punamäki R.-L. (2012). Effectiveness of school-based intervention in enhancing mental health and social functioning among war-affected children. Traumatology, 18(4), 37–46. doi:10.1177/1534765612437380 |
| 48. | Pronyk P. M., Harpham T., Busza J., Phetla G., Morison L. A., Hargreaves J. R., Porter J. D. (2008). Can social capital be intentionally generated? A randomized trial from rural South Africa. Social Science & Medicine, 67(10), 1559–1570. doi:10.1016/j.socscimed.2008.07.022 |
| 49. | Puffer E. S., Green E. P., Sikkema K. J., Broverman S. A., Ogwang-Odhiambo R. A., Pian J. (2016). A church-based intervention for families to promote mental health and prevent HIV among adolescents in rural Kenya: Results of a randomized trial. Journal of Consulting and Clinical Psychology, 84(6), 511–525. doi:10.1037/ccp0000076 |
| 50. | Purgato M., van Ommeren M., Tol W., Barbui C. (2018a). Addressing stress, depression, and anxiety in people exposed to traumatic events in humanitarian settings: A systematic review and meta-analysis of psychosocial interventions. Journal of Psychosomatic Research, 109, 127. doi:10.1016/J.JPSYCHORES. 2018.03.122 |
| 51. | Purgato M., Gross A. L., Betancourt T., Bolton P., Bonetto C., Gastaldon C., Barbui C. (2018b). Focused psychosocial interventions for children in low-resource humanitarian settings: A systematic review and individual participant data meta-analysis. The Lancet Global Health, 6(4), e390–e400. doi:10.1016/S2214-109X(18)30046-9 |
| 52. | Qouta S. R., Palosaari E., Diab M., Punamäki R. (2012). Intervention effectiveness among war‐affected children: A cluster randomized controlled trial on improving mental health. Journal of Traumatic Stress, 25(3), 288–298. doi:10.1002/jts.21707 |
| 53. | Qouta S. R., Peltonen K., Diab S. Y., Anttila S., Palosaari E., Punamäki R.-L. (2016). Psychosocial intervention and dreaming among war-affected Palestinian children. Dreaming, 26(2), 95–118. doi:10.1037/drm0000025 |
| 54. | Rahman A., Hamdani S. U., Awan N. R., Bryant R. A., Dawson K. S., Khan M. F., van Ommeren M. (2016). Effect of a multicomponent behavioral intervention in adults impaired by psychological distress in a conflict-affected area of Pakistan. JAMA, 316(24), 2609. doi:10.1001/jama.2016.17165 |
| 55. | Riblet N. B. V., Shiner B., Young-Xu Y., Watts B. V. (2017). Strategies to prevent death by suicide: Meta-analysis of randomised controlled trials. British Journal of Psychiatry, 210(06), 396–402. doi:10.1192/bjp.bp.116.187799 |
| 56. | Richards J., Foster C., Townsend N., Bauman A. (2014). Physical fitness and mental health impact of a sport-for-development intervention in a post-conflict setting: Randomised controlled trial nested within an observational study of adolescents in Gulu, Uganda. BMC Public Health, 14(1), 619. doi:10.1186/1471-2458-14-619 |
| 57. | Rotheram-Borus M. J., Richter L. M., van Heerden A., van Rooyen H., Tomlinson M., Harwood J. M., Stein A. (2014). A cluster randomized controlled trial evaluating the efficacy of peer mentors to support South African women living with HIV and their infants. PLoS One, 9(1). doi:10.1371/journal.pone.0084867 |
| 58. | Samarasekera U., Horton R. (2017). Improving evidence for health in humanitarian crises. Lancet (London, England), 390(10109), 2223–2224. doi:10.1016/S0140-6736(17)31353-3 |
| 59. | Segatto M. L., Andreoni S., de Souza e Silva R., Diehl A., Pinsky I. (2011). Brief motivational interview and educational brochure in emergency room settings for adolescents and young adults with alcohol-related problems: A randomized single-blind clinical trial. Revista Brasileira de Psiquiatria, 33(3), 225–233. |
| 60. | Singla D. R., Kohrt B. A., Murray L. K., Anand A., Chorpita B. F., Patel V. (2017). Psychological treatments for the world: Lessons from low- and middle-income countries. Annual Review of Clinical Psychology, 13(1), 149–181. doi:10.1146/annurev-clinpsy-032816-045217 |
| 61. | Soleimani S., Sharif F., Mani A., Keshavarzi S. (2014). The effect of conflict resolution training on children’s behavioral problems in Shiraz, Southern Iran: A randomized controlled trial. International Journal of Community Based Nursing & Midwifery, 2(3), 185–195. |
| 62. | Sparling J., Dragomir C., Ramey S. L., Florescu L. (2005). An educational intervention improves developmental progress of young children in a Romanian orphanage. Infant Mental Health Journal, 26(2), 127–142. doi:10.1002/imhj.20040 |
| 63. | Sphere Association. (2018). The Sphere handbook: Humanitarian charter and minimum standards in humanitarian response (Vol. 1). Geneva, Switzerland. ISBN 978-1-908176-00-4. |
| 64. | Ssewamala F. M., Neilands T. B., Waldfogel J., Ismayilova L. (2012). The impact of a comprehensive microfinance intervention on depression levels of AIDS-orphaned children in Uganda. Journal of Adolescent Health, 50(4), 346–352. doi:10.1016/j.jadohealth.2011.08.008 |
| 65. | Thurman T. R., Kidman R., Taylor T. M. (2015). Bridging the gap: The impact of home visiting programs for orphans and vulnerable children on social grant uptake in South Africa. Children and Youth Services Review, 48, 111–116. doi:10.1016/J.CHILDYOUTH. 2014.12.002 |
| 66. | Tol W. A., Komproe I. H., Susanty D., Jordans M. J., Macy R. D., De Jong J. T. (2008). School-based mental health intervention for children affected by political violence in Indonesia: A cluster randomized trial. JAMA, 300(6), 655–662. doi:10.1001/jama.300.6.655 |
| 67. | Tol Wietse A., Barbui C., Galappatti A., Silove D., Betancourt T. S., Souza R., van Ommeren M. (2011a). Mental health and psychosocial support in humanitarian settings: linking practice and research. Lancet, 378(9802), 1581–1591. doi:10.1016/S0140-6736(11)61094-5 |
| 68. | Tol Wietse A., Patel V., Tomlinson M., Baingana F., Galappatti A., Panter-Brick C., van Ommeren M. (2011b). Research priorities for mental health and psychosocial support in humanitarian settings. PLoS Medicine, 8(9), e1001096. doi:10.1371/journal.pmed.1001096 |
| 69. | Tol W. A., Komproe I. H., Jordans M. J., Vallipuram A., Sipsma H., Sivayokan S., de Jong J. T. (2012). Outcomes and moderators of a preventive school-based mental health intervention for children affected by war in Sri Lanka: A cluster randomized trial. World Psychiatry, 11(2), 114–122. |
| 70. | Tol W. A., Komproe I. H., Jordans M. J., Ndayisaba A., Ntamutumba P., Sipsma H., de Jong J. T. (2014). School-based mental health intervention for children in war-affected Burundi: a cluster randomized trial. BMC Medicine, 12, 56. doi:10.1186/1741-7015-12-56 |
| 71. | Tol Wietse A., Purgato M., Bass J. K., Galappatti A., Eaton W. (2015). Mental health and psychosocial support in humanitarian settings: a public mental health perspective. Epidemiology and Psychiatric Sciences, 24(06), 484–494. doi:10.1017/S2045796015000827 |
| 72. | Torrente C., Johnston B., Starkey L., Seidman E., Shivshanker A., Weisenhorn N., Aber J. L. (2015). Improving the quality of school interactions and student well-being: Impacts of one year of a school-based program in the Democratic Republic of Congo. Journal on Education in Emergencies, 1(1), 48–91. Retrieved from https://archive.nyu.edu/handle/2451/39654 |
| 73. | Train K. J., Butler N. (2013). A pilot study to test psychophonetics methodology for self-care and empathy in compassion fatigue, burnout and secondary traumatic stress. African Journal of Primary Health Care & Family Medicine, 5(1). doi:10.4102/phcfm.v5i1.497 |
| 74. | Tsai L. C., Carlson C. E., Aira T., Pala A. N., Riedel M., Witte S. S. (2016). The impact of a microsavings intervention on reducing violence against women engaged in sex work: A randomized controlled study. BMC International Health and Human Rights, 16(27). |
| 75. | UNICEF. (2018). Community-based mental health and psychosocial support in humanitarian settings: Three-tiered support for children and families. New York, NY. |
| 76. | |
| 77. | Vijayakumar L., Mohanraj R., Kumar S., Jeyaseelan V., Sriram S., Shanmugam M. (2017). CASP − An intervention by community volunteers to reduce suicidal behaviour among refugees. International Journal of Social Psychiatry, 63(7), 589–597. doi:10.1177/0020764017723940 |
| 78. | Weinstein N., Khabbaz F., Legate N. (2016). Enhancing need satisfaction to reduce psychological distress in Syrian refugees. Journal of Consulting and Clinical Psychology, 84(7), 645–650. doi:10.1037/ccp0000095 |
| 79. | Weiss W. M., Ugueto A. M., Mahmooth Z., Murray K., Hall B. J., Nadison M., Bass J. (2016). Mental health interventions and priorities for research for adult survivors of torture and systematic violence: A review of the literature. Torture, 26(1), 17–44. |
| 80. | Widmann M., Apondi B., Musau A., Warsame A. H., Isse M., Mutiso V., Odenwald M. (2017). Comorbid psychopathology and everyday functioning in a brief intervention study to reduce khat use among Somalis living in Kenya: Description of baseline multimorbidity, its effects of intervention and its moderation effects on substance use. Soc Psychiatry and Psychiatric Epidemiology. doi:10.1007/s00127-017-1368-y |
| 81. | Young S. D., Konda K., Caceres C., Galea J., Sung-Jae L., Salazar X., Coates T. (2011). Effect of a community popular opinion leader HIV/STI intervention on stigma in urban, coastal Peru. AIDS and Behavior, 15(5), 930–937. doi:10.1007/s10461-010-9826-5 |
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3]
|
Search Pubmed for