January 2013
Report: | Legal Capability for Everyday Life Evaluation Report |
Report commissioned by: | Lisa Wintersteiger, Law for Life |
Report author: | Liz Mackie, The Gilfillan Partnership |
1.4 About the project evaluation
The Gilfillan Partnership was commissioned to carry out an external evaluation of the Legal Capability for Everyday Life project. The external evaluation was required to evaluate the effectiveness and impact of the Legal Capability for Everyday Life project, and to test the practicalities of using the PLE evaluation framework to evaluate a PLE initiative.
To evaluate the effectiveness and impact of the Legal Capability for Everyday Life project, the evaluation assesses whether the project achieved its aims and intended outcomes. To test the PLE evaluation framework, the evaluation was carried out using (or adapting) indicators, methods and guidance detailed in the PLE evaluation framework document.
The project evaluation used the following methods:
- Use of ‘before’ and ‘after’ questionnaires with all participants. Each participant was asked to complete a questionnaire at the beginning of the first session. This questionnaire was intended to establish a baseline for participants of their level of legal capability before taking part in the PLE course. Participants were then asked to complete a similar questionnaire at the end of the last session, designed to measure any increases in legal capability over the course of the programme. The before and after questionnaires used 13 indicators of legal capability which were derived from the PLE evaluation framework. The indicators are shown in Table II, which also shows how they fit within the PLE evaluation framework
- Use of control groups. Each partner organisation agreed to identify a control group of about 10 service users who were similar to the project participants but who were not taking part in the project. The control groups were to complete the same before and after questionnaires at around the same time as the project participants. The intention was to provide a control for any changes to legal capability that may have occurred over the project period, in order to ensure that any changes for the participant group could be attributed to their participation in the project.
- Observation of training session. The evaluators participated in one session of each programme; the first session of the course which took place at Attend/DLS, and the final sessions of the courses with Paiwand and Community Links. As well as observing the sessions, we talked informally with participants, staff, volunteers and trainers.
- Interviews with partner organisations. These interviews took place after the PLE courses had ended and invited partners to reflect on the experience of participating in this pilot initiative and what they had learned from their involvement.
- Review session with the Law for Life project team. This session took place midway through the programme delivery period. It focused on the project experience to date and lessons emerging from this.
Table II: PLE evaluation framework and indicators used in the Legal Capability for Everyday
Life evaluation
PLE evaluation framework: evaluation measures |
Indicators used in the Legal Capability for Everyday Life evaluation |
Domain 1: Recognising and framing the legal dimension of issues and situations | |
1. Through PLE, participants learn more about the concepts of rights and obligations as a means of recognising and framing law-related issues. |
1. How confident are you that you understand your legal rights and obligations? 2. Have you ever had to deal with a law-related issue? |
2. Participants are able to apply learning about the concepts of rights and obligations to law-related issues in their lives. |
|
3. Through PLE, participants learn to identify that civil law might be relevant to deal with issues in their lives. |
3. Do you understand the difference between civil and criminal law? |
4. Through PLE, participants feel more confident about explaining an everyday issue and asking/answering questions about it. |
Domain 2: Finding out more about the legal dimension of issues and situations | |
5. Through PLE, participants learn what rights and obligations apply to a particular issue. |
|
6. Through PLE, participants learn how to find out what steps are involved in dealing with a law-related issue. |
|
7. Through PLE, participants learn how to find out more about a law-related issue. |
4. What sources of information would you use to find out more about a law-related issue? |
8. Through PLE, participants learn how to critically assess different sources of information. |
5. When you are deciding what sources of information or advice to use about law-related issues, how important are each of the following to you? [cost, access, impartiality, recommendation] |
9. Through PLE, participants learn how to assess when they can deal with a law related issue themselves and when they need expert advice. |
6. How confident are you that you know when you need to get expert legal help to deal with a situation? |
10. Through PLE, participants learn about different sources of advice and how to choose one that meets their needs. |
7. Do you agree with this statement: “If I had a law-related issue that I needed help to deal with, I would know where to get advice.”? |
11. Through PLE, participants are better able to identify the obstacles, risks and consequences of law-related issues |
Domain 3: Dealing with law-related issues | |
12. Participants are able to apply information or advice to law-related issues. |
8. How well have you dealt with the law-related issues that you have experiences in the pa |
PLE evaluation framework: evaluation measures |
Indicators used in the Legal Capability for Everyday Life evaluation |
9. If the same law-related issues happened in the future, how well would you be able to deal with them? 10. How confident are you that you can get the best result out of a legal situation? |
|
13. Through PLE, participants learn about different courses of action for dealing with law-related issues. |
|
14. Through PLE, participants learn how to critically assess different courses of action for dealing with law-related issues. |
|
15. Through PLE, participants feel able to plan and follow-through a course of action to deal with a law-related issue |
11. Do you agree with this statement: “I am good at taking action and seeing things through to the end.”? |
16. Through PLE, participants learn to organise and keep track. |
12. Do you agree with this statement: “I am well organised and can easily find relevant paperwork and correspondence when I need it.”? |
17. Through PLE, participants learn how to communicate appropriately and effectively as a way of dealing with law related issues. |
|
18. Through PLE, participants learn how to manage their feelings appropriately and effectively. |
Domain 4: Engaging and influencing | |
19. Through PLE, participants become aware of issues that might impact positively or negatively on their lives and the lives of others. |
|
20. Through PLE, participants learn to engage with and influence the world in which they live. |
|
21. Through PLE, participants learn how to seek changes in the laws and procedures that affect their lives and the lives of others. |
13. How much do you know about ways of seeking to make changes in laws and legal procedures? |