Authors: Dr. Sunil B. Joshi, Dr. Ashwini Chavan, Mr. Sushil Kulkarni, and Mr. Balchandra N. Doddi

Editor: Mr. Rajat Anil Hedav

ISBN: 978-81-989434-5-3

DOI: https://doi.org/10.59646/pe/396

Date of Publication: June 30, 2025

About the Book:

“Prompt Engineering: Theory and Practice” is a comprehensive guide to understanding the concepts, techniques, and applications of prompt engineering within large language models (LLMs). The book begins with a foundational overview of LLMs, exploring their architecture, historical evolution, and differences from traditional machine learning models. It delves into the core components such as tokens, embeddings, attention mechanisms, and transformer architecture. The book further introduces prompt engineering, focusing on the importance of prompts in generative AI, as well as techniques such as zero-shot and few-shot learning. Advanced strategies for effective prompt design, including role prompting, chain-of-thought, and self-consistency, are covered in detail. Additionally, the book emphasizes practical applications for various industries, including content generation, legal document summarization, and educational tools. With chapters on prompt evaluation, optimization, and the integration of open-source LLMs, this book serves as a valuable resource for those interested in mastering prompt engineering and its role in driving AI-driven innovations.

References

1. Brown, T. B., & Kaplan, J. (2021). Scaling laws for neural language models. Proceedings of NeurIPS (pp. 6111–6120). https://arxiv.org/abs/2001.08361
2. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., & Kaplan, J. (2020). Language models are few-shot learners. arXiv preprint arXiv:2005.14165. https://arxiv.org/abs/2005.14165
3. Devlin, J., Chang, M. W., Lee, K., & Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. Proceedings of NAACL (pp. 4171–4186). https://doi.org/10.18653/v1/N19-1423
4. He, H., & Cho, K. (2020). Prompt tuning for general-purpose language understanding. Proceedings of the 37th International Conference on Machine Learning (pp. 1111–1124). https://arxiv.org/abs/2006.08324
5. Lee, J., Shin, S., & Kim, T. (2022). Few-shot learning with prompt engineering for large language models. Proceedings of ICLR. https://openreview.net/forum?id=H2VbM4p5p9h
6. Li, J., & Liang, P. (2022). Learning to prompt for vision-language models. Proceedings of NeurIPS. https://arxiv.org/abs/2201.06783
7. Liu, B., & Su, C. (2022). Prompt engineering for generation tasks in natural language processing. IEEE Transactions on Neural Networks and Learning Systems, 33(8), 3592–3603. https://doi.org/10.1109/TNNLS.2021.3070348
8. Liu, Y., Ott, M., Goyal, N., Du, J., & Joshi, M. (2019). RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692. https://arxiv.org/abs/1907.11692
9. Radford, A., Narasimhan, K., Salimans, T., & Sutskever, I. (2018). Improving language understanding by generative pre-training. OpenAI Blog. https://openai.com/research/language-unsupervised
10. Raffel, C., Shinn, M., Roberts, A., Lee, D., & Narang, S. (2020). Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 21(140), 1–67. https://jmlr.org/papers/volume21/20-074/20-074.pdf
11. Schick, T., & Schütze, H. (2021). Exploiting cloze questions for few-shot text classification and natural language inference. Proceedings of EMNLP (pp. 6548–6561). https://doi.org/10.18653/v1/2021.emnlp-main.512
12. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. A., Kaiser, Ł., & Polosukhin, I. (2017). Attention is all you need. Proceedings of NeurIPS (pp. 5998–6008). https://doi.org/10.5555/3295222.3295349
13. Wei, J., Bosma, M., & Shinn, M. (2022). Chain of thought prompting elicits reasoning in large language models. Proceedings of the 40th International Conference on Machine Learning. https://arxiv.org/abs/2201.11903
14. Yang, Z., Dai, Z., Yang, Y., Salakhutdinov, R., & Cohen, W. (2019). Xlnet: Generalized autoregressive pretraining for language understanding. Proceedings of NeurIPS (pp. 5754–5764). https://arxiv.org/abs/1906.08237
15. Ziegler, Z., Stiennon, N., & Radford, A. (2021). Exploring transfer learning with GPT-3. Proceedings of the 38th International Conference on Machine Learning, 40, 678–690. https://arxiv.org/abs/2102.05460