validators.reward.test_task_validator
#
Module Contents#
Classes#
This class contains unit tests for the TaskValidator class. |
- class validators.reward.test_task_validator.TaskValidatorTestCase(methodName='runTest')#
Bases:
unittest.TestCase
This class contains unit tests for the TaskValidator class.
The tests cover different scenarios for the reward method of the TaskValidator class. The reward method is expected to return a reward based on the task name and the completion text.
- setUp()#
Hook method for setting up the test fixture before exercising it.
- test_augment_with_answer_keyword()#
Test if the reward method returns 0 when the task “name” starts with ‘augment’ (summarization) and the completion contains the ‘Answer:’ keyword.
- test_followup_with_answer_keyword()#
Test if the reward method returns 0 when the task “name” starts with ‘followup’ (question generation) and the completion contains the ‘Answer:’ keyword.
- test_augment_with_question_keyword()#
Test if the reward method returns 0 when the task “name” starts with ‘augment’ (summarization) and the completion contains the ‘Question:’ keyword.
- test_answer_with_question_keyword()#
Test if the reward method returns 0 when the task “name” is ‘answer’ (answer generation) and the completion contains the ‘Question:’ keyword.
- test_followup_and_answer_with_summary_keyword()#
Test if the reward method returns 0 when the task “name” is different from “augment” (summarization) and the completion contains the ‘Summary:’ keyword.
- test_reward_valid_followup()#
Test if the reward method returns 1 when the task “name” starts with ‘followup’ (question generation) and the completion contains a question
- test_reward_valid_answer()#
Test if the reward method returns 1 when the task “name” is ‘answer’ (answer generation) and the completion contains an answer
- test_reward_valid_augment()#
Test if the reward method returns 1 when the task “name” is ‘augment’ (summarization) and the completion contains the a summary.
- test_reward_valid_other()#
Test if the reward method returns 1 when the task “name” is different from “augment”, “followup”, and “answer” and the completion does not contain the ‘Summary:’, ‘Answer:’, and ‘Question:’ keywords.