I1.0 Evaluation of Testing and Data Collection Plan

Introduction

What is the purpose of the test? Is it to validate your predictions or is it to answer a question? How do you know that a test is successful? These are only a few of the questions that designers answer when testing a prototype or system and evaluating their test results.

A design solution may solve the technical problem at hand, but what if the test isn’t accurate or repeatable? Does the engineer immediately revisit the design process? No, the engineer would seek input from other experts to troubleshoot the testing results. This is exactly what your team is going to do . . . run a test, collect data, and seek the advice of experts on the evaluation of the testing results.

Equipment

• Engineering notebook
• Computer and printer
• Camera

Procedure

For this project, your team will test your prototype, collect data, and evaluate the results.

Part I

Perform the testing procedure(s) at least three times and collect data for each test on the forms that you created in Project 5.1.2 Test Procedure. Take pictures throughout the test procedure. Document your test administration, insert copies of your data, and reflect on the test results in your engineering notebook.

If your test results reflect your predictions for the testing procedure, answer the following questions with your team:

• Are your test results accurate?
• Is your testing procedure repeatable?

If you answered no to either of the questions above, proceed with your team to Part II. If you answered yes to both questions, then your team is ready to send your data to experts and/or stakeholders. Create a summary of the testing procedure. Submit the data tables and summary to the experts for their input.

After receiving feedback from the experts and/or stakeholders, determine with your team whether your design solution was a success, failure, or somewhere in between. As a team discuss the test results and write a summary of proposed corrections or modifications to the following:

• Prototype’s design
• Testing procedures
• Data collected

Revisit the document that you created in Project 5.1.2 Test Procedure. Revise the document such that the information provided in that Project is provided under a “Test Procedure” heading. Then add the remaining sections from the list that follows to the document for each different testing procedure that you performed.

I. Test Procedure: Include the information you documented in Project 5.1.2 Test Procedure. Include images of your tests to illustrate the procedure.

II. Results: Include the data that you collected as well as graphs and/or charts that are appropriate to display your data for easy consumption.

III. Reflection: Explain the results of your test. Discuss the quality of the test data. Why do you believe it is accurate and repeatable? Did the limitations of the test affect the results? What implications does the data have on the product design? Did the design solution perform successfully?

IV. Expert Feedback: Summarize the feedback that you received from experts regarding your test results and the implications of the test data.

V. Recommendations: Based on the test results, what recommendations would you make for improving the design of the product, the testing procedures, or the type or method of data collection?

Part II

Answer the following questions. You may choose to consult with your experts prior to answering:

• Do the results reflect a problem with the testing procedure?
• Do the results reflect a problem with the testing criteria?
• Do the results reflect a problem with the materials used for the prototype?
• Do the results reflect a problem with the quality of the building process of the prototype?
• Do the results reflect a problem with the design of the prototype?

If your team answered yes to any question above, revisit the document that you created in Project 5.1.2 Test Procedure. Revise the document such that the information provided in that Project is provided under a “Test Procedure” heading. Then add the remaining sections from the list that follows to the document for each different testing procedure that you performed.

I. Test Procedure: Include the information you documented in Project 5.1.2 Test Procedure.

II. Results: Include the data that you collected. Then discuss the results of your test. Explain why the data is not accurate or not repeatable.

III. Expert Feedback (if applicable): Summarize the feedback that you received from experts regarding your test results and the implications of the test data. Be sure to document your correspondence with experts and stakeholders and indicate specific questions, concerns, and opinions regarding the testing procedure and data analysis.

IV. Recommendations: Based on the test results, what recommendations would you make for improving the design of the product or prototype, changing the testing criteria (which may require a change in the design specifications), and/or revising the testing procedures.

Implement your recommended changes and create new documents (design specifications, technical drawings, prototype build procedure, and/or test procedure) as necessary. Be sure to document refinements in your engineering notebook. Retest your prototype using the instructions in Part I of this document. Be sure to document the second round of testing as required. If time does not allow you to incorporate the necessary refinements to your design solution, meet with your instructor to discuss options. You may only be able to create a summary of proposed improvements and receive expert feedback on your recommendations.

Conclusion

1. Why is a design solution to a technical problem an iterative process?
2. What is the purpose of evaluating your test results?