Participate!
Disability Inclusive Monitoring, Evaluation and Learning

Monitoring and evaluation of disability inclusion is an integral part of disability inclusive programming and essential for enhancing the effectiveness of interventions towards disability inclusion within a program.

Reading Time: 8 minutes

Disability Inclusive Monitoring, Evaluation and Learning

Monitoring and evaluation of disability inclusion is an integral part of disability inclusive programming and essential for enhancing the effectiveness of interventions towards disability inclusion within a program.

Set Targets

Setting targets or milestones and including specific indicators around disability inclusion works to set the pace for other actions needed to achieve and track the set targets. A target of at least 5% participation of persons with disabilities is an ideal place to get started. Once a general target has been set, additional targets may be set around inclusion of certain types of disabilities such as people with high support needs who may be easily left behind.

Disability-disaggregated data

Disaggregating data goes beyond stating whether an individual does or does not have a disability, types of impairments are noted in a way that allows data on this to be analyzed for other intersectional factors. This can include additional codes for different types of disabilities) as a multiplier to all individual-based indicators for program activities, outputs, and outcomes – this is often the best way to reflect outcomes related to equal access and inclusion during program monitoring. 

Inclusion as a key focus area for learning 

A number of wins and pitfalls will be experienced along the journey to disability inclusion which present plenty of opportunities for learning on best practices, or solutions to identified challenges. These are all very important for program adaptations to maximise impact, and for the benefit of other industry peers, and the sector as a whole. Systematically collecting these lessons through a learning agenda topic focused on disability inclusion will ensure sustained attention on the same throughout the program’s duration. In addition to this, organizing learning meetings either focused on inclusion, or integrating the topic of inclusion as part of other learning events ensures the knowledge generated from practice is shared, and utilized.

Participatory research, monitoring, evaluation and learning

Meaningfully including young women and men with disabilities in research, monitoring, evaluation and learning will help you to better meet your disability inclusion learning needs.  Allowing for this meaningful participation will enable program participants with disabilities to directly tell you the challenges they are facing, and help you to get the information you need to develop solutions together, while ensuring their specific experience in the program is captured. Embedding inclusion in programming this way will ensure that it is not an afterthought, but a well thought-out, integral part of what you do.  

Budgeting

To support these activities, it is advisable to allocate a portion of the disability inclusion budget specifically for monitoring, evaluation and learning. A good practice is to set aside 7% of this budget to cover activities like disability disaggregated data collection, analysis and sensemaking. This investment ensures that the program can measure its impact, spot gaps early, and adjust as needed, making disability inclusion realizable.

Tools

A useful tool for collecting disability data is the Washington Group Set of Questions.  This tool can be helpful in the recruitment phase of the program to ensure you are getting disability disaggregated data from the start.  

To learn more about deliberate mobilization in recruitment, click here (Link to Deliberate mobilization article)

Best Practice Spotlight

Rather than including a specific indicator such as “number of persons with disabilities that access digital skills training”, it may be more efficient to include a broader count of “number of people that access digital skills training”, but ensure that this data is disaggregated by age, gender, disability, and type of impairment. This allows a program to compare and monitor equal access for people with and without disability, as well as other intersecting identities such as age, gender, and type of impairment.

What Are the Washington Group Set of Questions?

The Washington Group on Disability Statistics (WG) is a city group established under the United Nations Statistical Commission. The major objective is to provide basic necessary information on disability that is comparable throughout the world. The WG has developed data collection tools for use in national censuses and surveys that produce internationally comparable data on disability. 

These tools include:

  • Short Set on Functioning (WG-SS)
  • Extended Set on Functioning (WG-ES)
  • Short Set on Functioning – Enhanced (WG-SS Enhanced)
  • UNICEF Child Functioning Module (CFM)
  • Washington Group / ILO Labor Force Survey (WG-LFS)

The Rationale Behind the Washington Group Questions

  • The term ‘disability’ can be understood differently among individuals, across cultures and communities.
  • In some languages, ‘disability’ may instead come off as a derogatory term that many individuals may not want to be associated with.
  • Concepts on disability have also evolved over time.
  • Asking “do you have a disability?”, often leads to unreliable data: 
  • Apart from the respondents being reluctant to reveal their functional limitations, this also leaves it up to the researcher/data collector to define what they would consider as a disability or not and can be influenced by their knowledge on disability.

It is generally more useful and reliable to ask about difficulties with aspects of functioning as recommended by the Washington Group and the International Classification of Functioning, Disability and Health (ICF).

The ICF classifies problems with human functioning as follows:
  • Impairment: any loss or abnormality in body function or alterations in body structure – for example, paralysis or blindness.
  • Activity limitations: difficulties in executing activities of daily living – for example, dressing, bathing or eating.
  • Participation restrictions: inability to perform an activity in a manner or within the range considered normal for a human being, mostly resulting from an impairment (ICF, 2001).

 

Under the classification, two general measures of impairment were constructed: basic action difficulty (mobility difficulty, seeing and hearing difficulty, and cognitive difficulty) and complex activity limitation which is more related to a person’s restriction in full participation in social role activities (self‐care, communication limitations).

Source:
Washington Group on Disability Statistics
WHO (2001). The International Classification of Functioning (ICF) Disability and Health.

Short Set-Enhanced Questions

VISION: 

VIS_1 [Do/Does] [you/he/she] have difficulty seeing, even when wearing [your/his/her] glasses]? Would you say… [Read response categories]

HEARING:

HEAR_1 [Do/Does] [you/he/she] have difficulty hearing, even when using a hearing aid(s)]? Would you say… [Read response categories]

MOBILITY:

[Do/Does] [you/he/she] have difficulty walking or climbing steps? Would you say… [Read response categories]

COMMUNICATION:

COM_1 Using [your/his/her] usual language, [do/does] [you/he/she] have difficulty communicating, for example understanding or being understood? Would you say… [Read response categories]

COGNITION (REMEMBERING): 

COG_1 [Do/does] [you/he/she] have difficulty remembering or concentrating? Would you say… [Read response categories] 

SELF-CARE:

SC_SS [Do/does] [you/he/she] have difficulty with self care, such as washing all over or dressing? Would you say… [Read response categories]

UPPER BODY:

UB_1 [Do/Does] [you/he/she] have difficulty raising a 2-liter bottle of water or soda from waist to eye level? Would you say… [Read response categories]

UB_2 [Do/Does] [you/he/she] have difficulty using [your/his/her] hands and fingers, such as picking up small objects, for example, a button or pencil, or opening or closing containers or bottles? Would you say… [Read response categories]

Response Categories

  1. No difficulty
  2. Some difficulty
  3. A lot of difficulty
  4. Cannot do at all
  5. Refused
  6. Don’t know

AFFECT (ANXIETY AND DEPRESSION):

Interviewer: If respondent asks whether they are to answer about their emotional states after taking mood-regulating medications, say: “Please answer according to whatever medication [you were/he was/she was] taking.”

ANXIETY:

ANX_1 How often [do/does] [you/he/she] feel worried, nervous or anxious? Would you say…[Read response categories]

  1. Daily
  2. Weekly
  3. Monthly
  4. A few times a year
  5. Never
  6. Refused
  7. Don’t know

ANX_2 Thinking about the last time [you/he/she] felt worried, nervous or anxious, how would [you/he/she] describe the level of these feelings? Would [you/he/she] say… [Read response categories]

  1. A little
  2. A lot
  3. Somewhere in between a little and a lot
  4. Refused
  5. Don’t know

DEPRESSION:

DEP_1 How often [do/does] [you/he/she] feel depressed? Would [you/he/she] say… [Read response categories]

  1. Daily
  2. Weekly
  3. Monthly
  4. A few times a year
  5. Never
  6. Refused
  7. Don’t know

DEP_2 Thinking about the last time [you/he/she] felt depressed, how depressed did [you/he/she] feel? Would you say… [Read response categories]

  1. A little
  2. A lot
  3. Somewhere in between a little and a lot
  4. Refused
  5. Don’t know

Response Validation/Data Analysis

*Guidance note for data analysis using SPSS, STATA Available

Level of Difficulty

  1. No difficulty
  2. Some difficulty
  3. A lot of difficulty
  4. Cannot do at all
  5. Refused
  6. Don’t know

Frequency

  1. Daily
  2. Weekly
  3. Monthly
  4. A few times a year
  5. Never
  6. Refused

Magnitude

  1. Don’t know
  2. A little
  3. A lot
  4. Somewhere in between a little and a lot
  5. Refused
  6. Don’t know

Using the Washington Group Short-Set on Functioning

  • Used as part of larger surveys, data collection exercises 
  • Inserted in the demographic section
  • Data collectors need to be trained on disability awareness and proper use of the WG Set of Questions
  • Field testing to observe data collectors; quality of interview, correct errors 
  • Consultative Translation before interviews as opposed to on-spot translations

Do’s and Don’ts

  • Do – treat all respondents with respect and dignity
  • Do – read the response options aloud, at least until the respondent becomes familiar with the answer categories
  • Do - record the answers given by the respondent exactly
  • Do – be sensitive to the situation of the person
  • Don’t - use the word disability when asking the questions
  • Don’t - change the questions, or add your own interpretation
  • Don’t - give examples
  • Don’t – make observations
  • Don’t - try to ‘diagnose’ or go beyond the question
  • Don’t - translate as you go

Reference Documents and Other Resource Links