Paper Key : IRJ************132
Author: Kartik Ashok Hawelikar
Date Published: 16 Oct 2023
Abstract
In the domain of Natural Language Processing (NLP), the technique of prompt engineering is a strategic method utilized to guide the responses of models such as ChatGPT. This research explores the intricacies of prompt engineering, with a specific focus on its effects on the quality of summaries generated by ChatGPT 3.5, an openly accessible chatbot developed by OpenAI. The study encompasses a comprehensive examination of 110 summaries produced from ten diverse paragraphs, employing eleven distinct summarization prompts under zero-shot setting. Evaluation is conducted using the BERT Score, a metric that offers a more contextually relevant assessment of summary quality. This study introduces an innovative approach to appraising the quality of summaries, setting it apart from prior investigations and delivering valuable insights into the nuances of prompt engineering's role within the NLP landscape. Ultimately, this inquiry illuminates the strengths and weaknesses associated with various prompts and their influence on ChatGPT 3.5's summarization capabilities, thereby making a significant contribution to the constantly evolving field of NLP and automated text summarization.
DOI LINK : 10.56726/IRJMETS45268 https://www.doi.org/10.56726/IRJMETS45268