PDPC: Plant disease phenotype captioning via semantic correction and trait description dependency grammar based on LLM.

As the cornerstone of global food security, agriculture sustains billions through staple crops like rice, wheat, and maize—yet devastating diseases such as wheat rust (causing annual losses over $2.9 billion) threaten this vital system. To enable precise disease diagnosis, we develop PDPC: a zero-shot framework that transforms visual symptoms into expert-level descriptions without model training. By integrating agricultural knowledge with semantic structuring, PDPC outperforms existing models including GPT-4 in accuracy across 300+ diseases in 60 crops, validated through 20,943 expert-annotated cases. This innovation pioneers intelligent crop protection by turning complex phenotypes into actionable insights for enhanced yield security.

Advantages of Multimodal Methods

In plant disease diagnosis, the multimodal method combining images and text reports shows significant advantages. This method can not only identify diseases more accurately but also provide more comprehensive diagnostic information.

Improvement in Early Detection

This method has been verified in medical diagnostics and also provides a better solution for the early detection of plant diseases. By integrating multimodal data, we can detect issues earlier and take action, thus reducing damage to crops.

Solving Single-Modality Problems

Similar to the advancements in smart medical care, solving plant disease identification requires the development of multimodal technology that can generate descriptive text. This not only improves the accuracy of diagnosis but also provides researchers and farmers with more detailed disease information.

Architecture

We designed the PDPC framework,an innovative framework designed for zero-shot image captioning that aims to enhance description quality. First, the PDPC framework leverages an extensive descriptive corpus to lay a solid linguistic foundation for plant disease descriptions, ensuring they are detailed and contextually relevant. Second, it employs syntactic analysis to dissect sentence structures, identifying key elements and their interrelations, which is crucial for constructing accurate and meaningful descriptions. Finally, by integrating semantic analysis with syntactic structure optimization, the PDPC framework enriches the descriptions, capturing the subtleties of plant disease characteristics with greater precision.

Figure 1: The framework of the proposed method. First, textual descriptions related to the input images are obtained from the augmented database (AD); Second, a concept map is generated by a concept map builder to identify and correlate the key features; Furthermore, the concept import optimization technique is applied to refine the concepts through substitution and insertion operations to improve the accuracy of the descriptions; Last, the filtered key concepts are combined using the LLAMA-3 model, which leads to an effective description text Correction.

Experimental Result

Through experimental validation, our approach outperforms multiple cutting-edge models in testing. Compared to larger models, our approach outperforms GPT-4 and other multimodal large models on multiple feature descriptors. These results show that our approach can provide more accurate disease diagnosis and phenotypic description while maintaining high efficiency.

Large Model Visualization

Figure 2: Examples of our method with other Large Language Models. These examples illustrate that PDPC excels in integrating detailed information, outperforming other large models in this critical aspect.