LEGAL CAPABILITY FOR EVERYDAY LIFE ( PART 9 )

January 2013

Report: Legal Capability for Everyday Life Evaluation Report
Report commissioned by: Lisa Wintersteiger, Law for Life
Report author: Liz Mackie, The Gilfillan Partnership

 

THREE: CONCLUSIONS

3.1 Improving individual legal capability

The project has fully met its aim of improving individual legal capability. The outcome that learners would be enabled to cope with the law in their everyday lives by improving their ability to deal with common law-related issues and situations has also been fully achieved.

The Legal Capability for Everyday Life project has developed and tested a set of resources for delivering a six session PLE course to advice service users. The testing carried out through the project found that people significantly increased their legal capability through their participation in the PLE course. The improvements in individual legal capability are across all four of the legal capability domains identified in the legal capability matrix. The clearest improvements in individual legal capability were in Domain 1 (recognising and framing the legal dimension of issues and situations) and Domain 3 (dealing with law related issues). These are the domains which partner agencies saw as the priority for their service users, and are therefore the areas in which the course content was focused. The strong improvements in individual legal capability in these domains reflects the emphasis of the course content.

Individuals who took part in the PLE courses were better able to recognise the legal dimension of day to day issues and more confident that they could tackle these issues, or seek appropriate help when necessary. Several participants had already started to act on legal issues as a result of the knowledge and confidence they had gained from the course. One project partner reported that some participants were significantly less likely to use the agency’s advice service since taking part in the PLE course, as they now felt more confident to deal on their own with many advice and information issues.

Based on feedback from individual participants and partner agencies, the following factors can be identified as having contributed to the success of the PLE courses:

  • Excellent trainers. The trainers were very highly praised for pitching the course at a level which was basic enough for those with no legal knowledge while being sufficiently engaging for people with varying levels of education and experience.
  • Useful hand outs. Information sheets about sources of advice on legal issues were handed out during the sessions. Participants found these very useful.
  • Course length. Individuals and partner agencies felt that attending a course over three or six weeks was very beneficial. It helped participants to become more confident to join in discussions or ask questions as they got to know the trainers and the other course participants. Several participants commented that they would have liked the course to run for a few more weeks so that more legal issues could be covered.
  • Tailoring to meet participant needs. The structure and basic content were the same for the three courses but the substantive areas of law, discussions and exercises varied to reflect the issues of greatest concern to the different course participants. For example, the Attend/DLS participants were particularly interested in legal issues concerning employment and discrimination, so course materials for some sessions were focused on these.
  • A legal expert participant. In every session on all three courses a legal expert was on hand to support the trainer with advice on particular points of law or their practical experience of dealing with legal issues. In some sessions the legal expert was from Law for Life and in others volunteers from the partner agencies took this role. The legal experts added great value to the sessions.

3.2 Testing the conceptual model of legal capability

The project aim of testing the conceptual model of legal capability has been fully achieved. The conceptual model of legal capability set out in the legal capability matrix proved a valuable framework for planning the PLE courses. The evaluation framework provided a helpful reference to ensure that the key elements of legal capability were addressed.

The outcome that partners would have a better understanding of the domains and elements of legal capability and how they apply to different groups of learners has been partially achieved. Only one interviewee from the three project partners seemed familiar with the PLE evaluation framework or the legal capability matrix. For agencies which are approaching PLE for the first time, the PLE evaluation framework document may be slightly daunting. It would be useful for Law for Life to produce a short guide which introduces the legal capability matrix to agencies that are new to PLE.

The testing which took place through the Legal Capability for Everyday Life project focused on Domains 1 and 3 of the evaluation framework, to reflect the priorities of the partner agencies and their service users. However, the same approach could easily be extended to cover Domains 2 and 4 in more depth, perhaps through a follow up course.

3.3 Developing delivery of public legal education within the advice sector The project aim of testing the PLE evaluation framework in the context of advice agencies has been fully achieved. The three advice agencies which took part in the Legal Capability for Everyday Life project provided a useful testing ground for developing the delivery of PLE within the advice sector, particularly with community based advice providers such as Paiwand and Community Links.

It would be useful for Law for Life to extend this pilot to include a wider range of advice agencies, particularly agencies working with communities that were not covered in this initial pilot project, such as young people or older people. The project aim of improving advice agencies’ ability to undertake PLE within their local communities has been achieved. All the partner agencies for this project now have a better understanding of the value of PLE within their own communities and experience of organising and managing the delivery of this. While some of the partner agencies are highly likely to develop their own PLE provision, others are looking to Law for Life for continued support in this area. This suggests that involvement in one PLE project does not in itself give all advice agencies sufficient experience and confidence to plan and deliver further PLE provision without continuing support. There is an important continuing role for central support from Law for Life to organisations wanting to deliver PLE. This support role includes promoting the value of PLE, developing resources for PLE, promoting high quality in PLE, sharing good practice from PLE provision and between PLE providers, evidencing the impacts of PLE through robust, systematic and co-ordinated evaluation of PLE provision.

The outcome that the PLE evaluation framework should be tailored to adapt to advice agency delivery of PLE has been fully achieved. The legal capability matrix has proved a helpful and flexible framework for planning and delivering PLE that is tailored to the needs of advice agencies and their users.

The outcome that advice agencies would better understand the skills and methodologies that are suited to delivering PLE in their communities has been fully achieved. The partner agencies all have a better understanding of what works in delivering PLE within their communities as a result of their involvement in this project.

The advice agencies which took part in the project consider that PLE is becoming increasingly essential to help ordinary people to cope as funding cuts reduce the availability of legal and general advice services. The PLE course developed and piloted through the Legal Capability for Everyday Life project represents an accessible and relatively low-cost measure which advice agencies can use to help their users to become better able to manage their everyday lives without recourse to increasingly limited advice service provision.

3.4 Measuring the impact of public legal education

The project aim of beginning to assess the potential wider socio-economic impact of PLE projects has been partially achieved. The project has provided an opportunity to test the PLE evaluation framework as a guide for evaluating the impacts of PLE. The PLE evaluation framework itself, however, can only evaluate these impacts in terms of individual benefits and does not extend to measuring wider socio-economic impacts.

The methods used to evaluate the impacts of the PLE courses delivered through this project found significant improvements in the legal capability of project participants.

The key lessons learned from the project evaluation are:

  • Paper-based before and after questionnaires are an affordable way to evaluate project impact. To minimise the errors that can occur in printing, distributing and collecting paper-based questionnaires, they should be kept as short as possible, ideally no more than two sides of A4 and with perhaps no more than six or seven questions.
  • Of the 13 indicators tested in this evaluation, eight worked well or very well and five did not work at all well. Any evaluation resources or guidance developed for future PLE projects should use the indicators identified in this report as working well and avoid or re-work those which did not work well.
  • There is a strong likelihood of cognitive bias in the responses to the before questionnaires, with many respondents giving positive responses to questions that they believe they know about when in fact, as the PLE course helps them to recognise, they do not have any knowledge in this area at all. This means that the evaluation has probably under-estimated the improvements made in individual legal capability by PLE, as the before position for participants is likely to be lower than has been measured.
  • To overcome this cognitive bias, future evaluations should introduce additional questions to test the knowledge of those who respond positively. For example, responses to the evaluation indictor ‘Do you understand the difference between civil and criminal law?’ could be tested by the inclusion of quiz questions asking respondents to identify whether various actions would fall under civil or criminal law.
  • The use of control groups is essential to confirm the results achieved from testing the participant groups. However, it proved particularly difficult for partner agencies to manage the process of completing before and after questionnaires with control groups.
  • It would be useful to explore alternative ways of managing the evaluation process in order to minimise errors arising from lack of capacity within partner organisations. It may be more effective to put the evaluation questionnaires online, with Law for Life collecting and analysing responses. This would remove the administrative burden from partner agencies and would enable centrally coordinated data analysis from local PLE initiatives to build a robust evidence base for the benefits and impacts of PLE.
  • It may be helpful to more fully integrate the questionnaire completion within the course sessions, perhaps including a quiz element to make it more engaging and as a means of verifying the accuracy of some question responses.
  • Evaluating the socio-economic impacts of PLE will require longitudinal research which looks at the longer term outcomes for individuals who have participated in PLE.

FOUR: RECOMMENDATIONS

1. The conceptual framework for legal capability provides a valuable tool for planning and delivering PLE. The full PLE evaluation framework document may be too long and complex for community advice agencies which are new to this topic. It would be useful for Law for Life to produce a short guide which introduces the legal capability matrix to agencies that are new to PLE.

2. It would be useful for Law for Life to extend the Legal Capability for Everyday Life pilot to include a wider range of advice agencies, particularly agencies working with communities that were not covered in the initial pilot project, such as young people and older people.

3. There is an important continuing role for central support from Law for Life to organisations delivering PLE. It seems unlikely that many agencies will initiate and deliver good quality PLE without such support. This support role includes promoting the value of PLE, developing resources for PLE, promoting high quality in PLE, sharing good practice between PLE providers, and evidencing the impacts of PLE through robust, systematic and co-ordinated evaluation of PLE provision.

4. Paper-based before and after questionnaires are an affordable and useful way to evaluate the impact of PLE initiatives. The questionnaires developed for this project should be adapted to become templates for other agencies to use when evaluating PLE initiatives.

5. To overcome the cognitive bias in responses to the before questionnaires, future evaluations should introduce additional questions which are designed to test the knowledge of those who respond positively.

6. Further work is needed to find ways of helping agencies delivering PLE initiatives to manage the implementation of robust evaluation methods. It may be more effective to put the evaluation questionnaires on-line, with Law for Life collecting and analysing responses. It might also help to more fully integrate the questionnaire completion within the course sessions and to use quiz questions to verify the accuracy of some question responses.

7. Law for Life should explore the possibility of securing funding for a longitudinal research project to measure the longer term outcomes for individuals who have participated in PLE and to identify any causal relationship between PLE and wider socio-economic benefits.

 

 

Disclaimer:

The information contains in this web-site is prepared for educational purpose. This site may be used by the students, faculties, independent learners and the learned advocates of all over the world. Researchers all over the world have the access to upload their writes up in this site. In consideration of the people’s participation in the Web Page, the individual, group, organization, business, spectator, or other, does hereby release and forever discharge the Lawyers & Jurists, and its officers, board, and employees, jointly and severally from any and all actions, causes of actions, claims and demands for, upon or by reason of any damage, loss or injury, which hereafter may be sustained by participating their work in the Web Page. This release extends and applies to, and also covers and includes, all unknown, unforeseen, unanticipated and unsuspected injuries, damages, loss and liability and the consequences thereof, as well as those now disclosed and known to exist.  The provisions of any state’s law providing substance that releases shall not extend to claims, demands, injuries, or damages which are known or unsuspected to exist at this time, to the person executing such release, are hereby expressly waived. However the Lawyers & Jurists makes no warranty expressed or implied or assumes any legal liability or responsibility for the accuracy, completeness or usefulness of any information, apparatus, product or process disclosed or represents that its use would not infringe privately owned rights. Reference herein to any specific commercial product process or service by trade name, trade mark, manufacturer or otherwise, does not necessarily constitute or imply its endorsement, recommendation or favouring by the Lawyers & Jurists. The views and opinions of the authors expressed in the Web site do not necessarily state or reflect those of the Lawyers & Jurists. Above all, if there is any complaint drop by any independent user to the admin for any contents of this site, the Lawyers & Jurists would remove this immediately from its site.